feat(core): use jemalloc with tuned decay timers for native module#34444
feat(core): use jemalloc with tuned decay timers for native module#34444leosvelperez wants to merge 2 commits intomasterfrom
Conversation
✅ Deploy Preview for nx-dev ready!
To edit notification comments on pull requests, go to your Netlify project configuration. |
✅ Deploy Preview for nx-docs ready!
To edit notification comments on pull requests, go to your Netlify project configuration. |
|
View your CI Pipeline Execution ↗ for commit d77d7aa
☁️ Nx Cloud last updated this comment at |
leosvelperez
force-pushed
the
nxc-3877
branch
from
288ca83 to
bda7b28
Compare
|
Failed to publish a PR release of this pull request, triggered by @leosvelperez. |
leosvelperez
force-pushed
the
nxc-3877
branch
2 times, most recently
from
5664423 to
bfe5813
Compare
|
Failed to publish a PR release of this pull request, triggered by @leosvelperez. |
leosvelperez
force-pushed
the
nxc-3877
branch
3 times, most recently
from
cde6c7d to
ae6bae4
Compare
leosvelperez
requested review from
a team,
FrozenPandaz and
vsavkin
as code owners
![nx-cloud[bot]](https://avatars.githubusercontent.com/in/80458?s=60&v=4){kind=small}
This comment was marked as outdated.
This comment was marked as outdated.
Sorry, something went wrong.
This comment was marked as outdated.
This comment was marked as outdated.
Sorry, something went wrong.
This comment was marked as outdated.
This comment was marked as outdated.
Sorry, something went wrong.
leosvelperez
force-pushed
the
nxc-3877
branch
from
753d4c9 to
d77d7aa
Compare
There was a problem hiding this comment.
Important
Nx Cloud is proposing a fix for your failed CI:
These changes adjust jemalloc's memory decay timers to prevent interference with Node.js module resolution. The original aggressive settings (dirty_decay_ms:1000, muzzy_decay_ms:0) caused premature page purging during project graph processing, breaking @microsoft/api-extractor imports. The new conservative timers (5s/1s) maintain memory optimization benefits while allowing Node.js sufficient time to complete dynamic module loading operations.
Warning
❌ We could not verify this fix.
diff --git a/.cargo/config.toml b/.cargo/config.toml
index 2e9a2981dd..af2d242129 100644
--- a/.cargo/config.toml
+++ b/.cargo/config.toml
@@ -1,5 +1,5 @@
[env]
-JEMALLOC_SYS_WITH_MALLOC_CONF = "dirty_decay_ms:1000,muzzy_decay_ms:0"
+JEMALLOC_SYS_WITH_MALLOC_CONF = "dirty_decay_ms:5000,muzzy_decay_ms:1000"
[build]
target-dir = 'dist/target'
Or Apply changes locally with:
npx nx-cloud apply-locally 9CcK-wfEN
Apply fix locally with your editor ↗ View interactive diff ↗
🎓 Learn more about Self-Healing CI on nx.dev
1 participant
Current Behavior
The Nx native module (Rust cdylib loaded by Node.js) uses the system allocator. The daemon process retains a large RSS footprint after the initial project graph build, even though most of that memory is no longer in use. On macOS and Linux, the system allocator doesn't aggressively return freed pages to the OS.
Expected Behavior
The daemon's steady-state RSS drops significantly after graph build by using jemalloc with tuned page purge timers. Peak RSS and wall time are unaffected.
Changes
Adds tikv-jemallocator as the global allocator on Linux and macOS, with two compile-time settings:
dirty_decay_ms:1000— returns freed pages to the OS after 1s instead of the default 10s. Tuned to Nx's phase-separated workload (graph build → idle → task execution), where transitions happen every ~30-60s. Benchmarked against 5s and 10s — both too slow to purge between phases.muzzy_decay_ms:0— skips the lazy purge phase (MADV_FREE) and goes straight toMADV_DONTNEED. Required on macOS and Linux ≥ 4.5 whereMADV_FREEdoesn't actually reduce RSS.Windows and WASI continue using the system allocator. Windows is excluded because
tikv-jemalloc-sysfails to build with MSVC (spaces incl.exepath break the autoconf configure script). Tracked upstream in tikv/jemallocator#99.Other Settings Considered
Tested narenas reduction, tcache_max, extent fit tuning, background threads, and decay timer values. Only the decay timer configuration improved steady-state RSS without wall time regression.