Guard nvcf 500 error parsing against non-JSON bodies#1945
Open
chuenchen309 wants to merge 1 commit into
Open
Conversation
On an HTTP 500, NvcfChat._call_model ran
json.loads(response.content)["detail"].startswith("Input value error")
unconditionally. A 500 body is not guaranteed to be JSON (a gateway/proxy in
front of the NVCF function can return an HTML/text error page) or to contain a
string 'detail', so this raised an uncaught JSONDecodeError / KeyError /
AttributeError. None of those is in the backoff on_exception tuple, so it
propagated out of _call_model and aborted the whole scan instead of being
retried or skipped.
Extract the check into a defensive _is_input_value_error helper: anything that
can't be confirmed as an input-value error returns False and falls through to
the existing raise_for_status path (a genuine server error, retried by
backoff) rather than crashing. Behavior for a well-formed
'Input value error' 500 (skip -> [None]) is unchanged.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Signed-off-by: Andrew Chen <48723787+chuenchen309@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
On an HTTP 500,
NvcfChat._call_modelparsed the error body unconditionally:A 500 body is not guaranteed to be JSON — a gateway/proxy/load balancer in front of the NVCF function commonly returns an HTML or plain-text error page — and even when JSON it may not carry a string
detail. In those cases this line raises an uncaughtjson.JSONDecodeError(aValueError),KeyError, orAttributeError.None of those is in the
@backoff.on_exceptiontuple (AttributeErroris, butJSONDecodeError/KeyErrorare not, and the string.startswithon a non-strdetailis theAttributeErrorpath), so on a long scan a single such 500 propagates out of_call_modeland aborts the whole run instead of being retried or skipped — the top-level handler just logs and exits.Fix
Extract the check into a defensive
_is_input_value_error(response)helper. Anything that can't be confirmed as an input-value error (non-JSON body, missing/non-stringdetail, or a different message) returnsFalseand falls through to the existingraise_for_status()path — i.e. it's treated as a genuine server error and retried by backoff, exactly as any other unrecognized 5xx already is. Behaviour for a well-formed"Input value error"500 (skip →[None]) is unchanged.Verification
test_nvcf_is_input_value_error_parses_defensively, parametrized over: a well-formed input-value error (→True), a different message, a non-JSON body, an empty body, JSON withoutdetail, and a non-stringdetail(all →False). Each of the last four would previously have raised inside the parse. Fulltests/generators/test_nvcf.pypasses;black --checkclean.garak -t <target_type> -n <model_name>— not run: this is a unit-level fix and I have no live target configured; verified via the tests above rather than ticking a box I didn't exercise.Disclosure: this contribution was prepared with the assistance of an AI agent (Claude Code). The analysis, fix, and tests were reviewed by me before submission.