Postmortem: three_digit_zip_code removal breaks ACA PTC

[MaxGhenis]

Postmortem: three_digit_zip_code formula removal breaks ACA PTC (PR #7695)

Timeline

• Mar 3: PR Remove dead ZIP_CODE_DATASET infrastructure #7695 opened to remove dead ZIP_CODE_DATASET infrastructure
• Mar 4: Reviewed by Pavel (Claude Code agents), approved with suggestions
• Mar 6: Merged. three_digit_zip_code formula removed as collateral damage
• Mar 6 → Mar 12: Versions 1.588 through 1.596.5 all carry the regression
• Mar 12: Paul Huntsberger (Amplifi) reports ACA PTC returning $0 for LA County households in #partner-amplifi
• Mar 12: Fix PR Restore three_digit_zip_code formula (fixes ACA PTC regression) #7774 opened, restoring the formula with regression tests (TDD)

Impact

• Who was affected: Any API caller providing zip_code but not three_digit_zip_code for an LA County household got aca_ptc = 0 instead of the correct value (~$5,055 for Paul's test case)
• Production household API: Still on v1.587.1 (the last working version). The weekly update bot created PRs to bump to broken versions (OR integration test #1433–Texas TANF #1442) but none were merged. Production users were NOT affected.
• Local/pip users: Anyone installing policyengine-us >= 1.588 from pip (like Amplifi) got the broken behavior.

Root cause

PR #7695 correctly identified that ZIP_CODE_DATASET was dead infrastructure (loaded from a file that never existed). The cleanup removed the zip_code formula (which did depend on the dataset) and the three_digit_zip_code formula (which did NOT — it was pure arithmetic: zip_code // 100).

The three_digit_zip_code formula got swept up because it lived in the same zip_code/ directory, and the PR treated both formulas as equivalent. The commit message says it "removes formulas from zip_code and three_digit_zip_code (kept as input-only variables)."

Why review didn't catch it

Pavel's review (generated by Claude Code agents) was thorough — it validated:

• Dead code removal was complete with no dangling references
• New guard logic in slcsp_age_0 and slcsp_rating_area_la_county was correct
• Test coverage for new guard paths
• Cosmetic issues

But it never questioned whether three_digit_zip_code's formula should have been removed. The review validated the implementation without challenging the scope.

The review's S2 finding was inches away from catching it — it analyzed what happens when three_digit_zip_code is empty — but didn't connect the dots: removing the formula is what makes it empty for normal API users.

Why tests didn't catch it

• All 132 ACA YAML tests pass because they provide slcsp or intermediate values directly
• The 17 Medicaid tests provide three_digit_zip_code as direct input
• No test exercised the full derivation chain: zip_code → three_digit_zip_code → slcsp_rating_area_la_county → slcsp → aca_ptc

How we fixed it (TDD)

1. Wrote 3 failing tests:
   • 90019 → "900" (unit test)
   • 01234 → "012" (leading zero, unit test)
   • LA County zip 90019 with eligible individual → nonzero aca_ptc (integration test)
2. Confirmed all 3 fail on the current codebase
3. Restored the formula (with a minor improvement: // 100 instead of // 1e2 to avoid unnecessary float promotion)
4. Confirmed all 3 pass

PR: #7774

Systemic fixes

Added to CLAUDE.md:

BEFORE DELETING ANY CODE, VERIFY IT IS ACTUALLY UNUSED

• Grep for all callers
• Code that lives near dead code is not necessarily dead — verify each piece independently
• Existing tests may bypass the code being removed — passing tests ≠ safe to delete

Added to policyengine-claude review patterns skill:

Code Removal / Cleanup PRs — Deletions are the highest-risk changes to review

1. Verify each deletion independently
2. Grep for downstream callers
3. Existing tests may not protect you
4. Question scope, not just correctness

General lessons

1. Proximity ≠ dependency — code co-located with dead code is not necessarily dead. Verify each piece independently.
2. Passing tests ≠ safe to delete — tests that provide intermediate values as direct input bypass derivation logic entirely.
3. Review scope, not just correctness — "Is the cleanup done correctly?" is the wrong question. "Does each removed piece actually belong in scope?" is the right one.