chore: add nightly Node.js variant MONGOSH-2969 by nbbeeken · Pull Request #2725 · mongodb-js/mongosh

nbbeeken · 2026-04-28T16:34:40Z

Introduce a new nightly Node.js build variant

Copilot

Pull request overview

Adds an Evergreen “Node.js nightly” buildvariant to run mongosh e2e tests against the latest Node nightly as an early-warning signal, plus supporting build/test adjustments for prerelease versions.

Changes:

Add a new Evergreen task + buildvariant (linux_node_nightly) that installs and runs against the latest Node.js nightly via nvm.
Adjust platform warnings and e2e tests to better handle Node prerelease/nightly behavior (including targeted skips).
Add a Node.js source patch and make Node patch application only pick up *.patch files.

Reviewed changes

Copilot reviewed 6 out of 7 changed files in this pull request and generated 1 comment.

Show a summary per file

File	Description
`scripts/nodejs-patches/002-workaround-node-bug-52229.patch`	New Node source patch to work around upstream issue 52229.
`packages/e2e-tests/test/e2e.spec.ts`	Skips a few flaky assertions on nightly builds while investigation is ongoing.
`packages/cli-repl/src/cli-repl.ts`	Includes prereleases in the Node version “recommended” warning check.
`packages/build/src/compile/signable-compiler.ts`	Applies only `.patch` files from `scripts/nodejs-patches/`.
`.evergreen/setup-env.sh`	Adds `NODE_JS_VERSION=nightly` handling via nvm + pins resolved nightly version within the task.
`.evergreen/evergreen.yml.in`	Adds `test_e2e_node_nightly` task and `linux_node_nightly` buildvariant in the template.
`.evergreen.yml`	Adds the rendered `test_e2e_node_nightly` task and `linux_node_nightly` buildvariant.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Copilot · 2026-04-28T21:19:07Z

+    # nightly variant prefer the host's system compiler — RHEL 10 ships
+    # gcc 14.2 / clang 19.1 stock, both new enough.
+    if [ -n "$USE_NIGHTLY_NODE" ] && [ -x /usr/bin/gcc ]; then
+      export CC=/usr/bin/gcc
+      export CXX=/usr/bin/g++


In the nightly branch this unconditionally switches CC/CXX to /usr/bin/gcc+/usr/bin/g++ whenever /usr/bin/gcc exists. On the new linux_node_nightly variant (ubuntu2004), /usr/bin/gcc is typically older than the pinned /opt/devtools gcc (and likely below the stated GCC >= 13.2 requirement), so this can downgrade the toolchain and cause native addon builds to fail. Consider selecting the compiler based on an actual version check (only override if the system compiler meets the minimum) or updating the buildvariant to run on an image that provides a sufficiently new compiler toolchain.

Suggested change

# nightly variant prefer the host's system compiler — RHEL 10 ships

# gcc 14.2 / clang 19.1 stock, both new enough.

if [ -n "$USE_NIGHTLY_NODE" ] && [ -x /usr/bin/gcc ]; then

export CC=/usr/bin/gcc

export CXX=/usr/bin/g++

# nightly variant prefer the host's system compiler only when it meets

# that minimum requirement.

version_ge() {

[ "$(printf '%s\n%s\n' "$2" "$1" | sort -V | head -1)" = "$2" ]

}

if [ -n "$USE_NIGHTLY_NODE" ] && [ -x /usr/bin/gcc ] && [ -x /usr/bin/g++ ]; then

MIN_GCC_VERSION=13.2.0

SYSTEM_GCC_VERSION=$(/usr/bin/gcc -dumpfullversion -dumpversion 2>/dev/null)

SYSTEM_GXX_VERSION=$(/usr/bin/g++ -dumpfullversion -dumpversion 2>/dev/null)

if [ -n "$SYSTEM_GCC_VERSION" ] && [ -n "$SYSTEM_GXX_VERSION" ] && \

version_ge "$SYSTEM_GCC_VERSION" "$MIN_GCC_VERSION" && \

version_ge "$SYSTEM_GXX_VERSION" "$MIN_GCC_VERSION"; then

export CC=/usr/bin/gcc

export CXX=/usr/bin/g++

else

echo "Keeping pinned /opt/devtools compiler for nightly: system gcc/g++ ($SYSTEM_GCC_VERSION / $SYSTEM_GXX_VERSION) do not meet minimum GCC $MIN_GCC_VERSION"

fi

addaleax · 2026-05-01T00:18:42Z

+  export NODE_JS_VERSION
+  export NODE_JS_VERSION_OVERRIDE="$NODE_JS_VERSION"
+  export USE_NIGHTLY_NODE=1
+fi


nvm feels a bit like overkill for downloading and extracting from https://nodejs.org/download/nightly/ for a single architecture and target OS

(aside: the ticket itself wasn't asking for nightlies, but I don't mind using them as long as the team is okay with that overall)

Node.js latest or nightly builds as an early warning system for breakages

I guess I just went with the second half of the OR and didn't think too much about it 😅 latest node would def be easier, but I'm wondering if we're a good canary in the coal mine with how deeply we integrate with the platform that nightlies could be more interesting, could also be more noisy...

addaleax · 2026-05-01T00:19:33Z

+    if [ -n "$USE_NIGHTLY_NODE" ] && [ -x /usr/bin/gcc ]; then
+      export CC=/usr/bin/gcc
+      export CXX=/usr/bin/g++
+    fi


Any reason not to update the toolchain?

This isn't needed anymore, we can still file a follow up for a toolchain update

addaleax · 2026-05-01T00:23:32Z

+      // under the nightly's prebuilt binary and re-enable.
+      if (process.version.includes('-nightly')) {
+        return this.skip();
+      }


Do we create separate tickets for these?

Yes, created!

addaleax · 2026-05-01T00:25:50Z

+          // executionAsyncId() === 0 inside the REPL context, where stable
+          // Node returns > 1. Investigate whether this is an intentional
+          // async_hooks change in V8/Node or whether the REPL setup needs
+          // adjustment, and re-enable once classified.


This is from nodejs/node#61227

We can probably get away with just adjusting the test here

The file is tracked because it carries the shared workspace setup, but personal color/peacock additions slipped in. Reverting; the file is now marked skip-worktree locally so future per-user tweaks don't show up in diffs.

Node.js v26 requires GCC >= 13.2 (per BUILDING.md) for C++20 features in V8 turboshaft, but /opt/devtools/bin/gcc on rhel80 builders is 12.4. Source the newest gcc-toolset-N/enable available on the host when USE_NIGHTLY_NODE is set so the build picks up a compliant compiler if one is installed.

Node.js v26 needs GCC >= 13.2 (C++20 in V8 turboshaft); rhel80-build only ships GCC 12.4. RHEL 10 brings a new-enough toolchain stock, so build on rhel10.0-large and run e2e on rhel10.0-small (matching glibc) instead of trying to bolt a newer compiler onto rhel80.

Confirmed via spawn host that current Node.js v26 nightlies (as of 2026-04) fail to build out of the box: V8 has multiple non-dependent static_asserts inside discarded `if constexpr` branches in turboshaft and typed array iteration paths. Both gcc 14.2 and clang 19.1 reject these per the C++ spec; pointer compression doesn't help because some occurrences live in non-template functions where there is no template parameter to defer the check on. Disable the variant for now while keeping all the supporting infrastructure in place: per-major-version patch directories, semver-based nodeVersionRange parsing in SignableCompiler, nvm-based nightly install in setup-env.sh, and v26 patch refreshes for the patches we already had. Re-enabling is a one-line flip on `disabled: true` once V8 v26 stabilizes.

…pile) Reframe: the goal is to keep verifying mongosh works on whatever Node.js release is coming next, forever. Compiling our own mongosh against nightly chains us to upstream V8 source bugs (e.g. current v26 nightlies fail to build on any toolchain because of non-dependent static_asserts in discarded `if constexpr` branches). Skip the compile entirely: install the prebuilt nightly Node.js binary via nvm and run the e2e suite against mongosh source. The Node.js team owns the V8 toolchain; we only need to validate that mongosh's source + native addons work on the next major. Concretely: drop the boxednode-nightly machinery (multi-version patch dirs, signable-compiler major-version routing, compile/e2e variant pair) and replace with a single linux test_e2e_node_nightly task that runs `npm run test-ci` with MONGOSH_RUN_ONLY_IN_PACKAGE=e2e-tests on a host where setup-env.sh has nvm-installed the latest nightly. Verified on a rhel10 spawn host: `node packages/cli-repl/bin/mongosh.js --version` returns 2.8.2 under v26.0.0-nightly202604284e3a873286 with native addons loaded cleanly.

Verified on a rhel10 spawn host: e2e suite runs 372 passing, 77 pending, 0 failing under v26.0.0-nightly20260428... (was 373/73/3 before these changes). * cli-repl: include prereleases when checking the recommended Node.js version. `semver.satisfies` excludes prereleases by default, so running on a Node.js nightly / RC (e.g. v26.0.0-nightly...) was incorrectly emitting "Using mongosh with Node.js versions lower than 24.0.0 is deprecated" even though the major version is 26. Real product bug, fix is one option flag. * e2e: skip the `async_hooks.executionAsyncId()` REPL test on `-nightly` builds. Stable Node returns >1, nightly v26 returns 0; needs investigation of whether async_hooks behavior changed in V8/Node or whether the REPL setup needs updating. TODO(MONGOSH-2969). * e2e: skip the update-notification test on `-nightly` builds. The expected notification line doesn't surface in captured output under nightly; the test fixture's HTTP/fetch interaction needs a closer look. TODO(MONGOSH-2969).

CI surfaced two more nightly-only failures that don't reproduce on the rhel10 spawn host: * --build-info: mongosh reports runtimeGlibcVersion = 'N/A' while Node.js process.report sees the host's actual glibc. The glibc-version addon's dlsym for gnu_get_libc_version returns null under the prebuilt nightly binary on some builders (e.g. ubuntu2004-small). * sets device ID for telemetry: native-machine-id throws under the same conditions, so the telemetry deviceId falls back to 'unknown'. Both are real but loader/binary-distribution-related, not regressions in mongosh source. Skip on '-nightly' with TODO(MONGOSH-2969) so the nightly e2e variant can land green; reproduce/fix as a follow-up.

nbbeeken marked this pull request as ready for review April 28, 2026 21:14

nbbeeken requested a review from a team as a code owner April 28, 2026 21:14

Copilot AI review requested due to automatic review settings April 28, 2026 21:14

Copilot started reviewing on behalf of nbbeeken April 28, 2026 21:14 View session

Copilot AI reviewed Apr 28, 2026

View reviewed changes

addaleax reviewed May 1, 2026

View reviewed changes

nbbeeken added 9 commits April 30, 2026 23:49

wip: nightly node variant + v26 patches

08889a9

fix(e2e-tests): update TODO references for Node.js nightly issues

0ccad22

nbbeeken force-pushed the MONGOSH-2969-node-latest branch from 1d08145 to 0ccad22 Compare May 1, 2026 03:49

nbbeeken requested a review from addaleax May 1, 2026 13:43

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

chore: add nightly Node.js variant MONGOSH-2969#2725

chore: add nightly Node.js variant MONGOSH-2969#2725
nbbeeken wants to merge 9 commits intomainfrom
MONGOSH-2969-node-latest

nbbeeken commented Apr 28, 2026

Uh oh!

Copilot AI left a comment

Uh oh!

Copilot AI Apr 28, 2026

Uh oh!

addaleax May 1, 2026

Uh oh!

nbbeeken May 1, 2026

Uh oh!

addaleax May 1, 2026

Uh oh!

nbbeeken May 1, 2026

Uh oh!

addaleax May 1, 2026

Uh oh!

nbbeeken May 1, 2026

Uh oh!

addaleax May 1, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

-    # nightly variant prefer the host's system compiler — RHEL 10 ships
-    # gcc 14.2 / clang 19.1 stock, both new enough.
-    if [ -n "$USE_NIGHTLY_NODE" ] && [ -x /usr/bin/gcc ]; then
-      export CC=/usr/bin/gcc
-      export CXX=/usr/bin/g++
+    # nightly variant prefer the host's system compiler only when it meets
+    # that minimum requirement.
+    version_ge() {
+      [ "$(printf '%s\n%s\n' "$2" "$1" | sort -V | head -1)" = "$2" ]
+    }
+    if [ -n "$USE_NIGHTLY_NODE" ] && [ -x /usr/bin/gcc ] && [ -x /usr/bin/g++ ]; then
+      MIN_GCC_VERSION=13.2.0
+      SYSTEM_GCC_VERSION=$(/usr/bin/gcc -dumpfullversion -dumpversion 2>/dev/null)
+      SYSTEM_GXX_VERSION=$(/usr/bin/g++ -dumpfullversion -dumpversion 2>/dev/null)
+      if [ -n "$SYSTEM_GCC_VERSION" ] && [ -n "$SYSTEM_GXX_VERSION" ] && \
+         version_ge "$SYSTEM_GCC_VERSION" "$MIN_GCC_VERSION" && \
+         version_ge "$SYSTEM_GXX_VERSION" "$MIN_GCC_VERSION"; then
+        export CC=/usr/bin/gcc
+        export CXX=/usr/bin/g++
+      else
+        echo "Keeping pinned /opt/devtools compiler for nightly: system gcc/g++ ($SYSTEM_GCC_VERSION / $SYSTEM_GXX_VERSION) do not meet minimum GCC $MIN_GCC_VERSION"
+      fi

Conversation

nbbeeken commented Apr 28, 2026

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Copilot AI Apr 28, 2026

Choose a reason for hiding this comment

Uh oh!

addaleax May 1, 2026

Choose a reason for hiding this comment

Uh oh!

nbbeeken May 1, 2026

Choose a reason for hiding this comment

Uh oh!

addaleax May 1, 2026

Choose a reason for hiding this comment

Uh oh!

nbbeeken May 1, 2026

Choose a reason for hiding this comment

Uh oh!

addaleax May 1, 2026

Choose a reason for hiding this comment

Uh oh!

nbbeeken May 1, 2026

Choose a reason for hiding this comment

Uh oh!

addaleax May 1, 2026

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants