fix: increase signal in git diff, git log, and json filters (#621)#708
Open
pszymkowiak wants to merge 1 commit intodevelopfrom
Open
fix: increase signal in git diff, git log, and json filters (#621)#708pszymkowiak wants to merge 1 commit intodevelopfrom
pszymkowiak wants to merge 1 commit intodevelopfrom
Conversation
0b45e2d to
9fe7879
Compare
- git diff: raise max_hunk_lines from 30 to 100 (LLMs need full hunks) - git log: show 3 body lines instead of 1 (preserves BREAKING CHANGE, migration notes) - json: show values by default (LLMs need values for config debugging), add --schema for types-only Tested with phi4:14b on local LLM — all 3 fixes improve comprehension. Signed-off-by: Patrick szymkowiak <[email protected]>
9fe7879 to
272534b
Compare
Collaborator
Author
LLM Comprehension Test — 57 RTK CommandsTested all 57 RTK Rust commands with 3 local LLMs to verify filtered output is actionable. ProtocolFor each command: send RTK output to LLM with "You ran Results
0% confusion rate — all RTK filtered outputs correctly understood by LLMs as small as 7B. No command blocks the AI from taking the correct next action. Commands testedGit (5), Files (8), Data (4), Runners (4), GitHub (3), Network (2), Meta (7), Python (4), Go (2), Docker (2), TOML filters (12), JS/TS (4) = 57 total Also fixed in this PR
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #621
Summary
Three filters were cutting signal that LLMs need:
--schemafor types-onlyWhy
git diffat 30 lines: LLMs miss the end of refactors and produce incorrect codegit logat 1 line: BREAKING CHANGE notes, migration instructions lostjsonwithout values: useless for config debugging (the primary use case)Test plan
rtk git diff HEAD~3— shows full hunks up to 100 linesrtk git log -5— shows 3 body lines per commitrtk json— shows values ("rtk","4.5", etc.)rtk json --schema— shows types only (old behavior)