[Bug]: `permission_mode: "auto"` makes `cupcake eval` exit non-zero after deciding — Claude Code hook fails open

[jtonl]

Bug Description

Claude Code sends permission_mode: "auto" in its hook events. cupcake eval --harness claude rejects that value during deserialization, and the process exits non-zero with empty stdout after it has already evaluated the policies and synthesized a decision.

Claude Code treats any hook exit code other than 0 (allow) or 2 (block) as a non-blocking error: it warns the user and runs the tool anyway. Because cupcake exits 1 with nothing on stdout, the decision it computed — including a deny/halt — never reaches Claude Code. The guardrail therefore fails open: tool calls that policy would block are permitted whenever the session's permission mode is auto.

This is a security-relevant failure, not a cosmetic one. A policy that denies an operation is silently ineffective under auto.

Steps to Reproduce

With any project that enables a blocking builtin (e.g. git_block_no_verify):

# 1. An event that policy SHOULD block, with the accepted mode "default":
printf '%s' '{"session_id":"x","transcript_path":"/tmp/t.jsonl","cwd":"'"$PWD"'","hook_event_name":"PreToolUse","tool_name":"Bash","tool_input":{"command":"git commit --no-verify -m x"},"permission_mode":"default"}' \
  | cupcake eval --harness claude --policy-dir .cupcake
echo "exit=$?"
# -> exit 0, stdout contains "permissionDecision":"deny"   (correct)

# 2. The SAME event, permission_mode "auto":
printf '%s' '{"session_id":"x","transcript_path":"/tmp/t.jsonl","cwd":"'"$PWD"'","hook_event_name":"PreToolUse","tool_name":"Bash","tool_input":{"command":"git commit --no-verify -m x"},"permission_mode":"auto"}' \
  | cupcake eval --harness claude --policy-dir .cupcake
echo "exit=$?"
# -> exit 1, stdout EMPTY, stderr: Error: unknown variant `auto`, ...
#    The deny decision is lost; Claude Code would run the command.

Mode matrix, same event otherwise:

permission_mode	exit	decision emitted
default / plan / acceptEdits / bypassPermissions	0	yes
(field absent)	0	yes
auto	1	no — stdout empty

Expected Behavior

1. Fail closed, not open. An internal error while handling a PreToolUse event must never result in the tool being permitted. If cupcake eval cannot produce a decision, it should exit 2 (block) rather than a non-blocking code with empty stdout — a policy engine that cannot evaluate should deny, not allow.
2. Tolerate unknown/future permission_mode values. New Claude Code modes (here, auto) should be accepted and ignored if no policy consumes the field, rather than aborting deserialization. A lenient/open enum (e.g. #[serde(other)] or Option<String>) would prevent every future mode from reintroducing this.

Actual Behavior

cupcake eval parses the event, matches policies, synthesizes a decision, and then fails deserialization on the unrecognized enum variant, exiting 1 with empty stdout:

Error: unknown variant `auto`, expected one of `default`, `plan`, `acceptEdits`, `bypassPermissions`

Claude Code reports Failed with non-blocking status code: No stderr output (the message is often empty in practice because operators redirect stderr — see Additional Context) and proceeds with the tool call. The computed decision is discarded.

Harness

Claude Code

Cupcake Version

cupcake 0.3.0 (from cupcake --version), installed from release v0.5.1.

Operating System

Linux

OS Version

Ubuntu 24.04

Cupcake Configuration

Reproduces with any enabled blocking builtin; minimal rulebook.yml:

builtins:
  git_block_no_verify:
    enabled: true

Relevant Logs

RUST_LOG=info trace — note the decision is synthesized before the crash, confirming the failure is in output/deserialization, not evaluation:

INFO evaluate{event_name="PreToolUse" tool_name="Bash"}: Found N matching policies
INFO evaluate{...}: synthesize{...}: Synthesizing decision from 1 total decisions
INFO evaluate{...}: Synthesized final decision: Deny { reason: "...", agent_messages: [] }
Error: unknown variant `auto`, expected one of `default`, `plan`, `acceptEdits`, `bypassPermissions`

Additional Context

Two things make this easy to miss and worth addressing alongside the fix:

1. cupcake eval writes several KB of INFO logs and a "STANDARD mode" banner to stderr on every successful call, even with RUST_LOG=error. This pushes operators to add 2>/dev/null to the hook command, which then suppresses the unknown variant error entirely — so the only symptom is a silent, unexplained fail-open. Consider gating the banner/INFO chatter behind a flag or RUST_LOG so stderr on success is quiet and stderr on failure is meaningful.

2. The broader hardening: for the Claude Code harness specifically, any non-0/2 outcome on PreToolUse is a fail-open by the hook contract. Treating "engine could not decide" as a 2 (deny) would make cupcake robust against this whole class of issue, not just the auto variant.