Stop the terminal-shutdown harness from loading the whole app graph to test worker respawn - #2366
Open
kriszyp wants to merge 1 commit into
Open
Stop the terminal-shutdown harness from loading the whole app graph to test worker respawn#2366kriszyp wants to merge 1 commit into
kriszyp wants to merge 1 commit into
Conversation
…o test worker respawn
terminalShutdown.test.js was flaky on Windows: one case per run — a different one
each time — died on `Timeout of 30000ms exceeded`.
The cost is not a too-short condition wait. Both `waitFor()` calls in the harness
find their condition true on the first poll. It is `restartWorkers()`'s lazy
`require('../loadRootComponents.js')`, whose module top-level pulls in the entire
application graph (databases, componentLoader, Application, certificates, and
transitively the AWS SDK, systeminformation, pkijs, rxjs, joi). That is ~5,400
synchronous module loads costing ~34s on Windows with a warm file cache — measured
at 34.2s/33.3s/32.3s across three consecutive runs, so it is steady-state
resolution cost, not a cold OS cache. The default harness mode ran ~46s against a
30s guard, and whichever `restartWorkers` case landed on the slow side of that
boundary failed.
The same blocked event loop produced the two red herrings in the output: the
"JavaScript execution has taken too long" warning, and the "ITC broadcast (type
schema) not acknowledged within 30000ms" warning — that ack timer was armed before
the block and fired the instant it ended, costing no additional wall clock.
Nothing this harness asserts depends on root components being reloaded; every
assertion is about manageThreads' own respawn/shutdown bookkeeping, and the three
modes that already passed never reach that require at all. So seed the require
cache with a no-op, resolved relative to manageThreads' own directory the same way
manageThreads resolves it — that keeps the key correct in both the dist and
typestrip layouts, and throws rather than silently restoring the 34s load if either
file moves.
Also hoist the timeout to the suite and raise it to 120s. .mocharc.json sets
`"timeout": 0` globally, so this suite's 30s was a self-imposed hang guard rather
than the repo default, and it sat too close to the real cost: the same case
measured 2.7s and 18s on one machine depending on load. 120s still catches a wedged
harness with room to spare.
Default harness mode goes from 46s to 15.5s cold and ~2.6s warm, with identical
output. Five consecutive suite runs pass in 11-13s.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Contributor
There was a problem hiding this comment.
Code Review
This pull request optimizes the terminal worker shutdown test suite by stubbing out the loading of root components in the test harness. This prevents loading the entire application module graph, which was causing significant delays and timeouts (especially on Windows). Additionally, the test suite is refactored to use a suite-level timeout of 120 seconds instead of individual 30-second timeouts, and the test cases are simplified to use arrow functions. There are no review comments, and I have no additional feedback to provide.
Contributor
|
Reviewed; no blockers found. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
unitTests/server/threads/terminalShutdown.test.jsis flaky on Windows — one case per run, a different one each time, dies onTimeout of 30000ms exceeded. It is the last thing keeping theunit-test-windowsgate from passing.Root cause
Not a too-short condition wait. Both
waitFor()calls in the harness find their condition true on the first poll, well inside their 5s budget.The cost is
restartWorkers()'s lazy require atserver/threads/manageThreads.js:512:loadRootComponents.js's module top-level pulls in the entire application graph —resources/databases,componentLoader,Application,security/keys, and transitively the AWS SDK,systeminformation,pkijs,rxjs,joi. I instrumentedModule._load: 5,442 synchronous module loads, ~34s on Windows with a warm file cache, measured at 34.2s / 33.3s / 32.3s across three consecutive runs. That is steady-state Windows module-resolution cost, not a cold OS cache — no single hot module, just ~6ms × 5,400 files.The default harness mode therefore ran ~46s end to end against a 30s guard. Only one test failed per run because the block sits right on that boundary: whichever of the two
restartWorkerscases landed on the slow side blew it.That same blocked event loop explains the two red herrings in the failure output:
JavaScript execution has taken too long and is not allowing proper event queue cyclingITC broadcast (type schema) not acknowledged by worker thread(s) 1 within 30000msThe ack timer was armed before the block and fired the instant it ended, so it cost no additional wall clock. I confirmed this by making the stub worker acknowledge ITC broadcasts: the warning disappeared and the runtime changed by nothing, so that change is not in this PR.
Changes
unitTests/server/threads/fixtures/terminalShutdownHarness.cjs— seed the require cache with a no-oploadRootComponentsbefore loadingmanageThreads. Nothing this harness asserts depends on root components being reloaded; every assertion is aboutmanageThreads' own respawn/shutdown bookkeeping, and the three modes that already passed never reach that require at all. The restart path itself is untouched.The stub resolves its cache key relative to
manageThreads' own directory, exactly the waymanageThreadsresolves it. Verified both layouts land on the right file:dist/server/loadRootComponents.js--conditions=typestripserver/loadRootComponents.jsIf either file moves,
require.resolvethrows here rather than quietly restoring the 34s load.unitTests/server/threads/terminalShutdown.test.js— hoist the timeout to the suite and raise it to 120s. Worth noting that.mocharc.jsonsets"timeout": 0globally, so this suite's 30s was a self-imposed hang guard, not the repo default. It sat too close to the real cost: I measured the same case at 2.7s and at 18s on one machine depending on system load. 120s still catches a wedged harness with real headroom.Verification
Default harness mode: 46s → 15.5s cold, ~2.6s once warm, with byte-identical output. Five consecutive suite runs, all green:
Full
unitTests/server/**/*.test.jsgroup: 837 passing, 21 failing. All 21 are pre-existing Windows unix-domain-socket failures (listen EACCESon.sockpaths under Temp) inudsMirror.test.js,uwsAdapter, andthreadServer listenOnPorts— untouched by this change and unrelated to it.Prettier clean on both files.
Notes
EXCLUDEDentry is needed for this test.unitTests/windowsGate.mjsdoes not exist onmainyet (ci: give Windows unit-level test coverage #2361 is unmerged), so I could not runnpm run test:unit:windowsdirectly — I verified against the equivalent glob instead.restartWorkers()paying a ~34s blocking module load on Windows is not only a test problem. A real component-reload restart on a Windows deployment eats that same load on the main thread the first time through.🤖 Generated with Claude Code