Skip to content

fix(probes): correct inverted target_lang_name_en guard message#1934

Open
anxkhn wants to merge 1 commit into
NVIDIA:mainfrom
anxkhn:patch-10
Open

fix(probes): correct inverted target_lang_name_en guard message#1934
anxkhn wants to merge 1 commit into
NVIDIA:mainfrom
anxkhn:patch-10

Conversation

@anxkhn

@anxkhn anxkhn commented Jul 10, 2026

Copy link
Copy Markdown
Contributor

Tell us what this change does:

garak.probes.latentinjection.TranslationMixin.__init__ raises a ValueError
when target_lang_name_en is None, i.e. the mixin requires
target_lang_name_en to be set and the guard fires precisely when it is not.
The message, however, read:

Probe cannot operate with target_lang_name_en being set

That states the opposite of the guarded condition. Anyone extending the mixin
(for example a new translation-based probe subclass that forgets to set
target_lang_name_en) who hits this error is told to unset the value, which
guarantees the failure persists.

This changes one word so the diagnostic matches the code's actual requirement:

Probe cannot operate without target_lang_name_en being set

It is a message-only change with no behavioural effect. There is exactly one
occurrence of this string in the codebase, so the fix is complete.

A regression test is added in the existing
tests/probes/test_probes_latentinjection.py: it instantiates a
TranslationMixin subclass without target_lang_name_en, asserts the raised
message states the value is required (without target_lang_name_en), and
asserts it no longer describes the inverse condition
(operate with target_lang_name_en). No target/model backend is needed.

Not a duplicate

gh pr list --state open searches on latentinjection, target_lang_name_en,
and TranslationMixin return no PR that touches
garak/probes/latentinjection.py, target_lang_name_en, or TranslationMixin.
The only two open PRs that mention the latentinjection module (1776 AgentHarm,
1627 0DIN JEF) do not modify this file or symbol (confirmed via
gh pr diff --name-only); they reference it incidentally. No existing issue
covers this message.

AI assistance

AI assistance (Claude) was used to help identify and fix this bug; this is
disclosed here and attributed in the commit via a Co-authored-by: trailer, as
garak's contribution guide requests. Every changed line has been reviewed and is
understood and defended by the human submitter.

Verification

  • Run the tests and ensure they pass: python -m pytest tests/probes/test_probes_latentinjection.py -q -> 20 passed
  • Broader probe suite unaffected: python -m pytest tests/probes/test_probes.py -q -> 1148 passed, 1 failed; the single failure is probes.audio.AudioAchillesHeel needing optional deps soundfile/librosa (not installed here), unrelated to this change and pre-existing
  • Formatting: python -m black --check garak/probes/latentinjection.py tests/probes/test_probes_latentinjection.py -> clean
  • Red -> green proof: reverting the source to the old "with" message makes the new test fail with a clear assertion; restoring the fix makes it pass
  • Verify the thing does what it should: a TranslationMixin subclass with no target_lang_name_en now raises a ValueError whose message correctly says the value must be set
  • Verify the thing does not do what it should not: no behavioural change; only the error string wording differs, and only the one guard message is touched

(No generator config, target run, or special hardware/software is relevant: this
is a message-only correctness fix with a unit-level regression test.)

`TranslationMixin.__init__` raises a `ValueError` when
`target_lang_name_en` is `None`, but the message read "Probe cannot
operate with target_lang_name_en being set". That states the opposite of
the guarded condition: the probe requires `target_lang_name_en` to be
set and the check fires precisely when it is not. Anyone extending the
mixin who hits this error was told to unset the value, which guarantees
the failure persists.

Reword the message to "cannot operate without target_lang_name_en being
set" so the diagnostic matches the code's actual requirement. This is a
message-only change with no behavioural effect.

Add a regression test that instantiates a `TranslationMixin` subclass
without `target_lang_name_en` and asserts the raised message states the
value is required (and no longer describes the inverse condition).

Co-authored-by: Claude <noreply@anthropic.com>
Signed-off-by: Anas Khan <83116240+anxkhn@users.noreply.github.com>
@jmartin-tech jmartin-tech self-assigned this Jul 10, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants