Skip to content

fix(gemma4): handle JSON-formatted keys in tool call parser#1065

Closed
afanty2021 wants to merge 9 commits into
Blaizzy:mainfrom
afanty2021:fix/gemma4-tool-parser-json-keys
Closed

fix(gemma4): handle JSON-formatted keys in tool call parser#1065
afanty2021 wants to merge 9 commits into
Blaizzy:mainfrom
afanty2021:fix/gemma4-tool-parser-json-keys

Conversation

@afanty2021
Copy link
Copy Markdown

Summary

Gemma 4 models may output standard JSON format for tool call arguments:

call:func_name{"key": "value"}  ← Standard JSON

Instead of the Gemma 4-specific format:

call:func_name{key<|"|>value<|"|>}  ← Gemma 4 format

The parser's _parse_object function was directly using the raw key text,
which included the JSON quotes, resulting in keys like '"action"' instead
of 'action'.

Problem

When the parser received JSON-formatted input:

  1. _parse_object extracted the key as "action" (with quotes)
  2. This created a dict: {'"action"': 'add'}
  3. json.dumps escaped the existing quotes: '{"\"action\"": "add"}'
  4. Downstream tools received malformed parameter names

Solution

Modified _parse_object to detect and handle JSON-formatted keys:

if key.startswith('"') and key.endswith('"'):
    try:
        key = json.loads(key)  # Properly unescape
    except (json.JSONDecodeError, ValueError):
        key = key[1:-1]  # Fallback: strip quotes

Test Cases

Added comprehensive tests covering:

  • ✅ Simple JSON-formatted keys
  • ✅ Multiple JSON keys
  • ✅ Nested objects/arrays with JSON keys
  • ✅ Unicode escape sequences in keys
  • ✅ Mixed Gemma 4 and JSON formats

All 12 tests pass.

Impact

  • Fixes tool call parameter parsing for Gemma 4 models
  • Maintains backward compatibility with existing Gemma 4 format
  • No changes needed to model code or chat templates

Co-Authored-By: Claude Opus 4.6 [email protected]

afanty2021 and others added 9 commits April 14, 2026 15:35
- 添加根级 CLAUDE.md 项目架构文档
- 添加 computer_use、evals、models 模块文档
- 为 38 个模型创建 README 文档
- 模型文档覆盖率从 33% 提升到 100%

Co-Authored-By: Claude Opus 4.6 <[email protected]>
Merge changes from upstream:
- Blaizzy#1056: hunyuan_vl/gemma3n cache-offset optimization
- Blaizzy#1053: Fix DFlash speculative decoding (GPU hang, performance)
- Blaizzy#1050: Thread-local generation stream (port mlx-lm#1090)
- Blaizzy#1055: Close batch_generate/server decode gap + VLM fixes

Conflict resolution:
- requirements.txt: Mixed approach - mlx>=0.31.2 with transformers<5.4.0
  to maintain omlx compatibility while accepting mlx update

Co-Authored-By: Claude Opus 4.6 <[email protected]>
Gemma 4 models may output standard JSON format for tool call arguments:
call:func_name{"key": "value"} instead of call:func_name{key<|"|>value<|"|>}

The parser's _parse_object function was directly using the raw key text,
which included the JSON quotes, resulting in keys like '"action"' instead
of 'action'. This caused downstream issues when the dict was serialized
to JSON, as the quotes were escaped again: '{"\"action\"": "value"}'

Fix:
- In _parse_object, detect JSON-formatted keys (surrounded by quotes)
- Use json.loads to properly unescape the key
- Fallback to stripping outer quotes if json.loads fails

Add comprehensive test cases covering:
- Simple JSON-formatted keys
- Multiple JSON keys
- Nested objects/arrays with JSON keys
- Unicode escape sequences in keys
- Mixed Gemma 4 and JSON formats

Fixes tool call parameter parsing for Gemma 4 models when they output
standard JSON format instead of the Gemma 4-specific escape format.

Co-Authored-By: Claude Opus 4.6 <[email protected]>
@Blaizzy
Copy link
Copy Markdown
Owner

Blaizzy commented May 6, 2026

Hey @afanty2021, thanks!

Please open a issue first

@Blaizzy Blaizzy closed this May 6, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants