fix(gemma4): handle JSON-formatted keys in tool call parser#1065
Closed
afanty2021 wants to merge 9 commits into
Closed
fix(gemma4): handle JSON-formatted keys in tool call parser#1065afanty2021 wants to merge 9 commits into
afanty2021 wants to merge 9 commits into
Conversation
- 添加根级 CLAUDE.md 项目架构文档 - 添加 computer_use、evals、models 模块文档 - 为 38 个模型创建 README 文档 - 模型文档覆盖率从 33% 提升到 100% Co-Authored-By: Claude Opus 4.6 <[email protected]>
Merge changes from upstream: - Blaizzy#1056: hunyuan_vl/gemma3n cache-offset optimization - Blaizzy#1053: Fix DFlash speculative decoding (GPU hang, performance) - Blaizzy#1050: Thread-local generation stream (port mlx-lm#1090) - Blaizzy#1055: Close batch_generate/server decode gap + VLM fixes Conflict resolution: - requirements.txt: Mixed approach - mlx>=0.31.2 with transformers<5.4.0 to maintain omlx compatibility while accepting mlx update Co-Authored-By: Claude Opus 4.6 <[email protected]>
Gemma 4 models may output standard JSON format for tool call arguments:
call:func_name{"key": "value"} instead of call:func_name{key<|"|>value<|"|>}
The parser's _parse_object function was directly using the raw key text,
which included the JSON quotes, resulting in keys like '"action"' instead
of 'action'. This caused downstream issues when the dict was serialized
to JSON, as the quotes were escaped again: '{"\"action\"": "value"}'
Fix:
- In _parse_object, detect JSON-formatted keys (surrounded by quotes)
- Use json.loads to properly unescape the key
- Fallback to stripping outer quotes if json.loads fails
Add comprehensive test cases covering:
- Simple JSON-formatted keys
- Multiple JSON keys
- Nested objects/arrays with JSON keys
- Unicode escape sequences in keys
- Mixed Gemma 4 and JSON formats
Fixes tool call parameter parsing for Gemma 4 models when they output
standard JSON format instead of the Gemma 4-specific escape format.
Co-Authored-By: Claude Opus 4.6 <[email protected]>
Owner
|
Hey @afanty2021, thanks! Please open a issue first |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Gemma 4 models may output standard JSON format for tool call arguments:
Instead of the Gemma 4-specific format:
The parser's
_parse_objectfunction was directly using the raw key text,which included the JSON quotes, resulting in keys like
'"action"'insteadof
'action'.Problem
When the parser received JSON-formatted input:
_parse_objectextracted the key as"action"(with quotes){'"action"': 'add'}json.dumpsescaped the existing quotes:'{"\"action\"": "add"}'Solution
Modified
_parse_objectto detect and handle JSON-formatted keys:Test Cases
Added comprehensive tests covering:
All 12 tests pass.
Impact
Co-Authored-By: Claude Opus 4.6 [email protected]