Skip to content

Stop the terminal-shutdown harness from loading the whole app graph to test worker respawn - #2366

Open
kriszyp wants to merge 1 commit into
mainfrom
claude/agitated-wozniak-19743c
Open

Stop the terminal-shutdown harness from loading the whole app graph to test worker respawn#2366
kriszyp wants to merge 1 commit into
mainfrom
claude/agitated-wozniak-19743c

Conversation

@kriszyp

@kriszyp kriszyp commented Aug 27, 2026

Copy link
Copy Markdown
Member

unitTests/server/threads/terminalShutdown.test.js is flaky on Windows — one case per run, a different one each time, dies on Timeout of 30000ms exceeded. It is the last thing keeping the unit-test-windows gate from passing.

Root cause

Not a too-short condition wait. Both waitFor() calls in the harness find their condition true on the first poll, well inside their 5s budget.

The cost is restartWorkers()'s lazy require at server/threads/manageThreads.js:512:

const { loadRootComponents } = require('../loadRootComponents.js');

loadRootComponents.js's module top-level pulls in the entire application graph — resources/databases, componentLoader, Application, security/keys, and transitively the AWS SDK, systeminformation, pkijs, rxjs, joi. I instrumented Module._load: 5,442 synchronous module loads, ~34s on Windows with a warm file cache, measured at 34.2s / 33.3s / 32.3s across three consecutive runs. That is steady-state Windows module-resolution cost, not a cold OS cache — no single hot module, just ~6ms × 5,400 files.

The default harness mode therefore ran ~46s end to end against a 30s guard. Only one test failed per run because the block sits right on that boundary: whichever of the two restartWorkers cases landed on the slow side blew it.

That same blocked event loop explains the two red herrings in the failure output:

  • JavaScript execution has taken too long and is not allowing proper event queue cycling
  • ITC broadcast (type schema) not acknowledged by worker thread(s) 1 within 30000ms

The ack timer was armed before the block and fired the instant it ended, so it cost no additional wall clock. I confirmed this by making the stub worker acknowledge ITC broadcasts: the warning disappeared and the runtime changed by nothing, so that change is not in this PR.

Changes

unitTests/server/threads/fixtures/terminalShutdownHarness.cjs — seed the require cache with a no-op loadRootComponents before loading manageThreads. Nothing this harness asserts depends on root components being reloaded; every assertion is about manageThreads' own respawn/shutdown bookkeeping, and the three modes that already passed never reach that require at all. The restart path itself is untouched.

The stub resolves its cache key relative to manageThreads' own directory, exactly the way manageThreads resolves it. Verified both layouts land on the right file:

condition resolves to
default dist/server/loadRootComponents.js
--conditions=typestrip server/loadRootComponents.js

If either file moves, require.resolve throws here rather than quietly restoring the 34s load.

unitTests/server/threads/terminalShutdown.test.js — hoist the timeout to the suite and raise it to 120s. Worth noting that .mocharc.json sets "timeout": 0 globally, so this suite's 30s was a self-imposed hang guard, not the repo default. It sat too close to the real cost: I measured the same case at 2.7s and at 18s on one machine depending on system load. 120s still catches a wedged harness with real headroom.

Verification

Default harness mode: 46s → 15.5s cold, ~2.6s once warm, with byte-identical output. Five consecutive suite runs, all green:

5 passing (12s) / (13s) / (12s) / (11s) / (12s)

Full unitTests/server/**/*.test.js group: 837 passing, 21 failing. All 21 are pre-existing Windows unix-domain-socket failures (listen EACCES on .sock paths under Temp) in udsMirror.test.js, uwsAdapter, and threadServer listenOnPorts — untouched by this change and unrelated to it.

Prettier clean on both files.

Notes

  • No EXCLUDED entry is needed for this test. unitTests/windowsGate.mjs does not exist on main yet (ci: give Windows unit-level test coverage #2361 is unmerged), so I could not run npm run test:unit:windows directly — I verified against the equivalent glob instead.
  • Out of scope here, but worth a separate look: restartWorkers() paying a ~34s blocking module load on Windows is not only a test problem. A real component-reload restart on a Windows deployment eats that same load on the main thread the first time through.

🤖 Generated with Claude Code

…o test worker respawn

terminalShutdown.test.js was flaky on Windows: one case per run — a different one
each time — died on `Timeout of 30000ms exceeded`.

The cost is not a too-short condition wait. Both `waitFor()` calls in the harness
find their condition true on the first poll. It is `restartWorkers()`'s lazy
`require('../loadRootComponents.js')`, whose module top-level pulls in the entire
application graph (databases, componentLoader, Application, certificates, and
transitively the AWS SDK, systeminformation, pkijs, rxjs, joi). That is ~5,400
synchronous module loads costing ~34s on Windows with a warm file cache — measured
at 34.2s/33.3s/32.3s across three consecutive runs, so it is steady-state
resolution cost, not a cold OS cache. The default harness mode ran ~46s against a
30s guard, and whichever `restartWorkers` case landed on the slow side of that
boundary failed.

The same blocked event loop produced the two red herrings in the output: the
"JavaScript execution has taken too long" warning, and the "ITC broadcast (type
schema) not acknowledged within 30000ms" warning — that ack timer was armed before
the block and fired the instant it ended, costing no additional wall clock.

Nothing this harness asserts depends on root components being reloaded; every
assertion is about manageThreads' own respawn/shutdown bookkeeping, and the three
modes that already passed never reach that require at all. So seed the require
cache with a no-op, resolved relative to manageThreads' own directory the same way
manageThreads resolves it — that keeps the key correct in both the dist and
typestrip layouts, and throws rather than silently restoring the 34s load if either
file moves.

Also hoist the timeout to the suite and raise it to 120s. .mocharc.json sets
`"timeout": 0` globally, so this suite's 30s was a self-imposed hang guard rather
than the repo default, and it sat too close to the real cost: the same case
measured 2.7s and 18s on one machine depending on load. 120s still catches a wedged
harness with room to spare.

Default harness mode goes from 46s to 15.5s cold and ~2.6s warm, with identical
output. Five consecutive suite runs pass in 11-13s.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request optimizes the terminal worker shutdown test suite by stubbing out the loading of root components in the test harness. This prevents loading the entire application module graph, which was causing significant delays and timeouts (especially on Windows). Additionally, the test suite is refactored to use a suite-level timeout of 120 seconds instead of individual 30-second timeouts, and the test cases are simplified to use arrow functions. There are no review comments, and I have no additional feedback to provide.

@claude

claude Bot commented Aug 27, 2026

Copy link
Copy Markdown
Contributor

Reviewed; no blockers found.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant