Static Test Fail Tracker for EIP-8037 #2601
Description
Context
EIP-8037 adds a state gas dimension at Amsterdam for SSTORE, CREATE, and code-deposit operations. This causes test fill failures for the ported static tests. 3423 tests still fail and likely need deeper fixes.
Test fixes should be applied to the eips/amsterdam/eip-8037 branch. Note this branch needs a merge once #2563 lands, but this work can be done in parallel.
Once all of the ported static tests are passing on eips/amsterdam/eip-8037 we must confirm that we can fill all with the following:
uv run fill --generate-all-formats --until Amsterdam
For debugging and local testing the recommendation is to fill with -m blockchain_test with the exact test path only.
When the table below is complete, we then run the above to check it all works for all the formats (enginex/pre-alloc). Some issues came up before that were previously skipped to get devnet releases out, so these will still be present.
Once we have all of these passing, we can then recreate the devnets/bal/3 branch and fix up the integration issues.
Failure Tracking
| Suite | Failures | Assignee | PR | Fixed |
|---|---|---|---|---|
tests/ported_static/stTimeConsuming |
2919 | @spencer-tb | WIP | ❌ |
tests/ported_static/stSStoreTest |
120 | @kclowes | WIP | ❌ |
tests/ported_static/stCreateTest |
59 | ❌ | ||
tests/ported_static/stEIP2930 |
43 | ❌ | ||
tests/ported_static/stCreate2 |
39 | ❌ | ||
tests/ported_static/stPreCompiledContracts |
29 | ❌ | ||
tests/ported_static/stEIP150singleCodeGasPrices |
28 | ❌ | ||
tests/ported_static/stEIP5656_MCOPY |
23 | ❌ | ||
tests/ported_static/stZeroKnowledge |
19 | ❌ | ||
tests/ported_static/stCallCreateCallCodeTest |
18 | ❌ | ||
tests/ported_static/stCallCodes |
17 | ❌ | ||
tests/ported_static/stInitCodeTest |
11 | ❌ | ||
tests/ported_static/stCallDelegateCodesCallCodeHomestead |
11 | ❌ | ||
tests/ported_static/stNonZeroCallsTest |
10 | ❌ | ||
tests/ported_static/stCallDelegateCodesHomestead |
10 | ❌ | ||
tests/ported_static/stWalletTest |
8 | ❌ | ||
tests/ported_static/stRefundTest |
7 | ❌ | ||
tests/ported_static/stPreCompiledContracts2 |
7 | ❌ | ||
tests/ported_static/stEIP150Specific |
7 | ❌ | ||
tests/ported_static/stRevertTest |
4 | ❌ | ||
tests/ported_static/stCodeSizeLimit |
4 | ❌ | ||
tests/ported_static/stBadOpcode |
4 | @kclowes | #2618 | ❌ |
tests/ported_static/stTransactionTest |
3 | ❌ | ||
tests/ported_static/stMemExpandingEIP150Calls |
3 | ❌ | ||
tests/ported_static/stEIP3860_limitmeterinitcode |
3 | ❌ | ||
tests/ported_static/stDelegatecallTestHomestead |
3 | ❌ | ||
tests/ported_static/stSystemOperationsTest |
2 | ❌ | ||
tests/ported_static/stMemoryTest |
2 | ❌ | ||
tests/ported_static/stHomesteadSpecific |
2 | ❌ | ||
tests/ported_static/stEIP3855_push0 |
2 | ❌ | ||
tests/ported_static/vmArithmeticTest |
1 | ❌ | ||
tests/ported_static/stSpecialTest |
1 | ❌ | ||
tests/ported_static/stMemoryStressTest |
1 | ❌ | ||
tests/ported_static/stEIP158Specific |
1 | ❌ | ||
tests/ported_static/stEIP1559 |
1 | ❌ | ||
tests/ported_static/stAttackTest |
1 | ❌ |
Additional failures on devnets/bal/3 (+29) To Be Confirmed, after the above are all fixed and merged in
On devnets/bal/3, the total rises from 3,423 to 3,452 due to EIP-8037 integration with other Amsterdam EIPs. These are secondary fixes tracked on branch devnets/bal/3/fixes.
| Suite | eip-8037 |
devnets/bal/3 |
Delta |
|---|---|---|---|
tests/ported_static/stStaticCall |
0 | 28 | +28 |
tests/ported_static/stEIP3860_limitmeterinitcode |
3 | 5 | +2 |
tests/ported_static/stAttackTest |
1 | 2 | +1 |
tests/ported_static/stEIP2930 |
43 | 41 | -2 |
Branch
eips/amsterdam/eip-8037
Why a flat gas bump doesn't work for these
-
Tests with many chained SSTORE ops (e.g.
stSStoreTest): each zero→nonzero SSTORE costs ~37.5k state gas (32 × 1174). Tests with 16+ SSTOREs need 600k+ extra — the 2M bump covers the tx-level gas but inner CALL gas allocations are unchanged. -
Exact-gas tests (e.g.
stEIP150singleCodeGasPrices,stPreCompiledContracts): these test precise gas accounting. State gas changes the gas available to inner calls, so the expected storage outcomes change even with sufficient top-level gas. -
CREATE-heavy tests (e.g.
stCreateTest,stEIP2930): CREATE state gas is ~131k per new account (112 × 1174). Tests that create multiple contracts or have tight gas need per-test calibration. -
OOG boundary tests (e.g.
stWalletTest,stCallCreateCallCodeTest): tests that deliberately operate near the gas boundary. Bumping gas changes which operations succeed/fail, altering expected post-state. -
Slow SSTORE combination tests (
stTimeConsuming): inner CALL opcodes pass hardcoded gas amounts in the bytecode. Top-level gas bumps don't propagate to subcalls, so the inner SSTOREs still OOG.
Approach for remaining fixes
Each failing test needs individual analysis:
- Determine which state operations cause the gas shortfall
- Calculate the exact extra gas needed
- For exact-gas tests: may need fork-conditional expected post-state
- For OOG boundary tests: may need separate Amsterdam test parameters
- For slow tests: may need bytecode-level changes or fork-conditional test splitting