Skip to content

feat(onboard): add hosted inference to portable profile - #8578

Open
ericksoa wants to merge 15 commits into
mainfrom
feat/portable-remote-inference-bootstrap
Open

feat(onboard): add hosted inference to portable profile#8578
ericksoa wants to merge 15 commits into
mainfrom
feat/portable-remote-inference-bootstrap

Conversation

@ericksoa

@ericksoa ericksoa commented Aug 7, 2026

Copy link
Copy Markdown
Contributor

Summary

The hidden portable profile now selects the existing OpenAI-compatible hosted inference path when a validated credential descriptor is available. It consumes an activated owner-only descriptor first, falls back to bounded object-storage retrieval, and stops before onboarding when neither hosted source is available.

Changes

  • Read an activated descriptor from a fixed runtime path so image activation can prepare credentials without network startup latency.
  • Fall back to the current AWS CLI credential context for object retrieval when no activated descriptor is present.
  • Bound and validate source locations, file ownership and permissions, Base64/JSON shape, credential data, HTTPS endpoints, and model IDs before onboarding mutates sandbox state.
  • Route validated values through the existing compatible-endpoint provider, restore scoped environment state after onboarding, and fail closed instead of selecting local inference.
  • Accept a persistent image-prepared activation file and skip the optional preset-policy upload in this hidden profile.
  • Add focused tests for activation precedence, file security, source selection, fallback, restoration, malformed input, ambiguous aliases, and redacted failures.
  • Keep the in-sandbox gateway intact while disabling its host-side dashboard forward for portable installs, including reuse, recovery, restart, stop, doctor, and deployment verification.

Type of Change

  • Code change (feature, bug fix, or refactor)
  • Code change with doc updates
  • Doc only (prose changes, no code sample modifications)
  • Doc only (includes code sample changes)

Quality Gates

  • Tests added or updated for changed behavior
  • Existing tests cover changed behavior — justification:
  • Tests not applicable — justification:
  • Docs updated for user-facing behavior changes
  • Docs not applicable — justification: The change remains confined to an existing hidden experimental profile and adds no public command, flag, environment variable, default, or supported configuration surface.
  • Sensitive paths changed (security, policy, credentials, preflight, onboarding, inference, runner, sandbox, or messaging)
  • Sensitive-path review completed or maintainer-approved waiver recorded — reviewer/approval link/justification: Credential-boundary review completed against the repository security rubric. Descriptor acquisition is bounded, source validation fails closed, secret-bearing errors are redacted, environment state is restored, and focused tests cover local precedence, ownership and permissions, malformed data, and failure paths.
  • Non-success, skipped, or missing CI check accepted by maintainer — check name, approval link, and follow-up issue:

Documentation Writer Review

  • Documentation writer subagent reviewed the completed changes
  • Result: no-docs-needed
  • Evidence: The change remains limited to the existing hidden portable experimental profile. Hosted inference is required from an image-prepared activation file or object source, missing activation fails closed, and optional preset-policy upload is skipped. No public interface or supported workflow changes.
  • Agent: Codex Desktop

DGX Station Hardware Evidence

  • Tested on DGX Station
  • Tested commit:
  • Station profile/scenario:
  • Result:
  • Supporting evidence:

Verification

  • PR description includes a Signed-off-by: line and every commit appears as Verified in GitHub
  • Normal pre-commit, commit-msg, and pre-push hooks passed, or npm run validate:pr passed after refreshing origin/main when hooks were skipped or unavailable
  • Targeted behavior tests pass for the current change set, or tests are marked not applicable above — command/result or justification: Hosted-only inference and policy-selection tests passed 73/73; dashboard-forward unit tests passed 151/151; the built-CLI recovery integration passed 5/5; npm run typecheck:cli and repository checks passed.
  • Applicable broad gate passed — npm test for broad runtime/test-harness changes; npm run check for repo-wide validation/coverage changes — command/result:
  • Quality Gates section completed with required justifications or waivers
  • No secrets, API keys, or credentials committed
  • npm run docs builds without warnings (doc changes only)
  • Doc pages follow the style guide (doc changes only)
  • New doc pages include SPDX header and frontmatter (new pages only)

Signed-off-by: Aaron Erickson aerickson@nvidia.com

Summary by CodeRabbit

  • New Features

    • Added hosted portable inference support during onboarding.
    • Automatically applies configured provider, model, credentials, and endpoint.
    • Supports local activation descriptors and hosted storage configuration.
    • Separates dashboard forwarding from dashboard lifecycle management.
    • Portable profiles use gateway and inference readiness checks without host dashboard forwarding.
  • Bug Fixes

    • Improved validation of incomplete or invalid inference settings.
    • Provides clear onboarding errors instead of unexpected failures.
    • Preserves environment settings and redacts credentials from errors.
    • Skips dashboard forwarding recovery and verification when disabled.

Signed-off-by: Aaron Erickson <aerickson@nvidia.com>
Signed-off-by: Aaron Erickson <aerickson@nvidia.com>
@ericksoa ericksoa self-assigned this Aug 7, 2026
@coderabbitai

coderabbitai Bot commented Aug 7, 2026

Copy link
Copy Markdown
Contributor

Review Change Stack

Note

Reviews paused

It looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the reviews.auto_review.auto_pause_after_reviewed_commits setting.

Use the following commands to manage reviews:

  • @coderabbitai resume to resume automatic reviews.
  • @coderabbitai review to trigger a single review.

Use the checkboxes below for quick actions:

  • ▶️ Resume reviews
  • 🔍 Trigger review
📝 Walkthrough

Walkthrough

Portable onboarding now resolves hosted inference descriptors from bootstrap credentials and S3. It validates credentials, endpoints, and models. Dashboard lifecycle management is separate from host dashboard forwarding. Recovery and deployment verification honor persisted forwarding state.

Changes

Portable inference onboarding

Layer / File(s) Summary
Portable inference source resolution
src/lib/onboard/experimental/portable-inference-source.ts, src/lib/onboard/experimental/portable-inference-source.test.ts
Loads and validates bootstrap credentials and S3 descriptors. Normalizes inference sources and sanitizes errors.
Portable onboarding environment integration
src/lib/onboard/command.ts, src/lib/onboard/command.test.ts
Applies resolved hosted inference settings, restores environment state, and reports invalid sources as bounded input errors.
Dashboard-forward policy and onboarding wiring
src/lib/onboard/dashboard-runtime.ts, src/lib/onboard/agent-dashboard-forward.ts, src/lib/onboard.ts, src/lib/onboard/machine/handlers/finalization.ts, src/lib/onboard/sandbox-registration.ts, src/lib/onboard/sandbox-reuse.ts, src/lib/state/registry/types.ts, src/lib/onboard/forward-cleanup.ts, src/lib/onboard/*test.ts
Separates dashboard management from host forwarding, disables forwarding for portable profiles, persists forwarding state, and cleans up existing forwards during reuse.
Dashboard-forward recovery and verification
src/lib/actions/sandbox/*.ts, src/lib/verify-deployment.ts, src/lib/verify-deployment.test.ts, test/recover-port-forward.test.ts, src/lib/onboard/inference-selection-validation.ts, src/lib/security/trusted-private-endpoint.ts
Skips host-forward probing and recovery when forwarding is disabled. Controls explicit loopback access through caller permission. Gateway and inference checks remain active.

Estimated code review effort: 4 (Complex) | ~60 minutes

Sequence Diagram(s)

sequenceDiagram
  participant Onboarding
  participant SandboxRegistry
  participant Recovery
  participant DeploymentVerification
  Onboarding->>SandboxRegistry: persist dashboardForwardEnabled
  Recovery->>SandboxRegistry: read dashboardForwardEnabled
  alt Dashboard forwarding enabled
    Recovery->>Recovery: recover dashboard/API forward
    Recovery->>DeploymentVerification: verify dashboard forward
  else Dashboard forwarding disabled
    Recovery->>Recovery: skip dashboard/API forward recovery
    Recovery->>DeploymentVerification: verify gateway and inference
  end
Loading

Possibly related PRs

Suggested labels: area: sandbox

Suggested reviewers: apurvvkumaria, cv

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 31.58% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title clearly summarizes the primary change: adding hosted inference support to the portable profile.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
✨ Finishing Touches 💡 3
📝 Generate docstrings 💡
  • Create stacked PR
  • Commit on current branch
⚔️ Resolve merge conflicts 💡
  • Resolve merge conflict in branch feat/portable-remote-inference-bootstrap
🛠️ Fix failing CI checks 💡
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch feat/portable-remote-inference-bootstrap

Comment @coderabbitai help to get the list of available commands.

@github-code-quality

github-code-quality Bot commented Aug 7, 2026

Copy link
Copy Markdown
Contributor

Code Coverage Overview

Languages: TypeScript

TypeScript / code-coverage/plugin

The overall coverage in commit beb7b2b in the feat/portable-remote... branch remains at 96%, unchanged from commit cdaefe8 in the main branch.


Updated August 07, 2026 20:24 UTC

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🧹 Nitpick comments (1)
src/lib/onboard/command.test.ts (1)

532-540: 🎯 Functional Correctness | 🔵 Trivial | ⚡ Quick win

Assert removal of initially unset portable variables.

toMatchObject permits extra keys after runOnboardCommand returns. The test does not detect leaked NEMOCLAW_OLLAMA_NO_AUTOSTART, policy variables, or NEMOCLAW_EXPERIMENTAL_PROFILE.

Use toEqual with the initial environment object, or assert that each initially unset portable variable is absent after onboarding.

Proposed test change
-    expect(env).toMatchObject({
+    expect(env).toEqual({
       S3_BUCKET: "portable-inference",
       S3_KEY: "path/credential.b64",
       COMPATIBLE_API_KEY: "previous-compatible-key",

The PR objective requires scoped environment restoration.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@src/lib/onboard/command.test.ts` around lines 532 - 540, Update the
environment assertion in the onboarding test to verify exact restoration rather
than a subset: replace toMatchObject with toEqual against the initial
environment object, or explicitly assert absence of every initially unset
portable variable, including NEMOCLAW_OLLAMA_NO_AUTOSTART, policy variables, and
NEMOCLAW_EXPERIMENTAL_PROFILE.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@src/lib/onboard/experimental/portable-inference-source.test.ts`:
- Around line 35-39: Stub readActivatedDescriptor in every listed
source-selection test in
src/lib/onboard/experimental/portable-inference-source.test.ts (lines 35-39,
113-127, 129-152, 154-171, and 173-184) by passing a shared
noActivatedDescriptor fake returning null to resolvePortableInferenceSource,
ensuring tests do not depend on the external activation file.

---

Nitpick comments:
In `@src/lib/onboard/command.test.ts`:
- Around line 532-540: Update the environment assertion in the onboarding test
to verify exact restoration rather than a subset: replace toMatchObject with
toEqual against the initial environment object, or explicitly assert absence of
every initially unset portable variable, including NEMOCLAW_OLLAMA_NO_AUTOSTART,
policy variables, and NEMOCLAW_EXPERIMENTAL_PROFILE.
🪄 Autofix

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: 4919853f-21ea-448c-9ba7-262c243ea2c1

📥 Commits

Reviewing files that changed from the base of the PR and between a7d5842 and 27e493f.

📒 Files selected for processing (4)
  • src/lib/onboard/command.test.ts
  • src/lib/onboard/command.ts
  • src/lib/onboard/experimental/portable-inference-source.test.ts
  • src/lib/onboard/experimental/portable-inference-source.ts

Comment thread src/lib/onboard/experimental/portable-inference-source.test.ts Outdated
Signed-off-by: Aaron Erickson <aerickson@nvidia.com>
@senthilr-nv senthilr-nv added feature PR adds or expands user-visible functionality area: onboarding Onboarding FSM, provider setup, sandbox launch, or first-run flow area: inference Inference routing, serving, model selection, or outputs security v0.0.105 Release target labels Aug 7, 2026
@github-actions

github-actions Bot commented Aug 7, 2026

Copy link
Copy Markdown
Contributor

PR Review Advisor — No blocking findings reported

Advisor assessment: No blocking advisor findings reported
Next action: Review the warnings below.
Findings: 0 blockers · 2 warnings · 0 suggestions

Model lanes

  • GPT-5.6 Terra (primary): Completed · medium confidence · 0 blockers · 2 warnings · 0 suggestions
  • Nemotron 3 Ultra (second opinion): Failed after a partial review · low confidence · 0 blockers · 0 warnings · 0 suggestions

Second-opinion terminology and E2E selections are advisory. Live E2E does not run automatically for pull requests.

4 semantic terminology decisions

Terminology decisions are advisory. They affect the assessment only when a separate finding identifies concrete semantic impact.

  • established — hosted inference at src/lib/onboard/command.ts:397: Keep `hosted inference` for the portable remote provider path.
  • justified — bootstrap credential at src/lib/onboard/experimental/portable-inference-source.ts:59: Keep `bootstrap credential`; its modifier identifies its retrieval role and trust boundary.
  • define — credential descriptor at src/lib/onboard/experimental/portable-inference-source.ts:191: Define `credential descriptor` at its first explanatory user-facing use if this hidden profile becomes a supported surface.
  • established — dashboard forward at src/lib/verify-deployment.ts:533: Keep `dashboard forward` for the dashboard-specific host-side port forward.

E2E guidance

Advisory only. A maintainer can dispatch the default E2E suite against this exact revision.

Recommended E2E: cloud-inference, cloud-onboard, managed-image-multiarch-startup, managed-image-protected-runtime, security-posture, inference-routing, network-policy, onboard-repair, onboard-resume

1 optional E2E recommendation
  • sandbox-operations
2 warnings · 0 suggestions

Warnings

Warnings do not block.

PRA-1 Warning — Update the portable policy-tier assertion

  • Location: src/lib/onboard/command.test.ts:480
  • Category: correctness
  • Problem: The portable-profile environment test still expects the old `personal` policy tier after the implementation changed the value to `balanced`.
  • Impact: The changed test fails and cannot provide reliable regression evidence for the portable policy tier.
  • Recommendation: Change the test expectation to `balanced`, unless the intended portable policy tier is `personal`, in which case restore that implementation value.
  • Verification: Read `applyPortableEnvironment` and the portable-profile test object, then run the focused command test in the repository validation environment.
  • Test coverage: The existing `prepares and scopes hosted-only portable profile defaults around onboarding` test should assert the selected `balanced` policy tier and restoration of the prior value.
  • Evidence: src/lib/onboard/command.ts:407 sets `NEMOCLAW_POLICY_TIER: "balanced"`. src/lib/onboard/command.test.ts:480 expects `NEMOCLAW_POLICY_TIER: "personal"` for the same portable onboarding flow.

PRA-2 Warning — Cover policy-set HTTP/2 reset recovery

  • Location: src/lib/policy/index.ts:618
  • Category: tests
  • Problem: The new policy transport-reset recovery branch has no focused automated coverage.
  • Impact: A transport reset can be misclassified as success, retry unnecessarily after a completed policy update, or fail to preserve the intended fail-closed outcome without a test detecting the regression.
  • Recommendation: Add focused policy tests with mocked policy-set and policy-get results for the verified-reset, retry-success, and repeated-reset failure paths.
  • Verification: Inspect policy tests with mocked runner output and confirm assertions for policy-set call count and final boolean result for each reset outcome.
  • Test coverage: Test an H2 `PROTOCOL_ERROR` followed by matching active policy without a second set; a mismatching read followed by a successful second set; and two reset responses with no matching active policy returning false.
  • Evidence: src/lib/policy/index.ts:618-653 adds `isPolicySetH2TransportReset`, `activePolicyMatches`, and `setPolicyFileWithH2Recovery`. The policy test inventory contains no test referencing `H2` or this recovery branch.

Workflow run details

This automated review informs maintainers. Warnings and suggestions do not require a response. A maintainer decides whether to merge.

Signed-off-by: Aaron Erickson <aerickson@nvidia.com>

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (1)
src/lib/onboard/machine/handlers/finalization.ts (1)

246-272: 🎯 Functional Correctness | 🟠 Major | ⚡ Quick win

Pass the dashboard-forward policy into deployment verification.

manageDashboardForward only controls forward creation and output. deps.verifyDeployment(sandboxName, verifyChain) still uses the default host-forward check. A portable run therefore verifies an intentionally absent host forward and pauses onboarding.

  • src/lib/onboard/machine/handlers/finalization.ts#L246-L272: Extend deps.verifyDeployment to accept verification options. Pass { verifyDashboardForward: manageDashboardForward }.
  • src/lib/onboard/machine/handlers/finalization.test.ts#L219-L236: Assert that portable finalization passes verifyDashboardForward: false to the dependency.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@src/lib/onboard/machine/handlers/finalization.ts` around lines 246 - 272,
Update src/lib/onboard/machine/handlers/finalization.ts lines 246-272 so
verifyDeployment accepts and receives the options object {
verifyDashboardForward: manageDashboardForward }, preserving the existing
verification flow. Update src/lib/onboard/machine/handlers/finalization.test.ts
lines 219-236 to assert that portable finalization passes
verifyDashboardForward: false to the dependency.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@src/lib/onboard/sandbox-reuse.ts`:
- Line 108: Update the sandbox reuse flow around dashboardForwardEnabled so a
true-to-false manageDashboard transition stops any existing host forward before
persisting false. Only write the disabled state after the stop succeeds;
propagate or handle stop failures without recording false. Add a reuse test
covering this transition and verifying the persisted lifecycle state.

---

Outside diff comments:
In `@src/lib/onboard/machine/handlers/finalization.ts`:
- Around line 246-272: Update src/lib/onboard/machine/handlers/finalization.ts
lines 246-272 so verifyDeployment accepts and receives the options object {
verifyDashboardForward: manageDashboardForward }, preserving the existing
verification flow. Update src/lib/onboard/machine/handlers/finalization.test.ts
lines 219-236 to assert that portable finalization passes
verifyDashboardForward: false to the dependency.
🪄 Autofix

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: 1bd38c9d-2a61-43df-848c-da57e81dfbd0

📥 Commits

Reviewing files that changed from the base of the PR and between 442e8c7 and 75766ca.

📒 Files selected for processing (17)
  • src/lib/actions/sandbox/doctor.ts
  • src/lib/actions/sandbox/forward-recovery.ts
  • src/lib/actions/sandbox/gateway-restart.ts
  • src/lib/actions/sandbox/process-recovery.ts
  • src/lib/onboard.ts
  • src/lib/onboard/agent-dashboard-forward.test.ts
  • src/lib/onboard/agent-dashboard-forward.ts
  • src/lib/onboard/dashboard-runtime.ts
  • src/lib/onboard/machine/handlers/finalization.test.ts
  • src/lib/onboard/machine/handlers/finalization.ts
  • src/lib/onboard/sandbox-registration.ts
  • src/lib/onboard/sandbox-reuse.test.ts
  • src/lib/onboard/sandbox-reuse.ts
  • src/lib/state/registry/types.ts
  • src/lib/verify-deployment.test.ts
  • src/lib/verify-deployment.ts
  • test/recover-port-forward.test.ts

Comment thread src/lib/onboard/sandbox-reuse.ts
Signed-off-by: Aaron Erickson <aerickson@nvidia.com>
@senthilr-nv

Copy link
Copy Markdown
Collaborator

Security review on e2a45b8: the fail-closed hosted-inference requirement is good, but the new persistent /var/lib/nemoclaw/portable-inference.b64 source needs a credential-lifecycle guarantee. Please establish that it is injected after image creation and never retained in an image layer, registry, build cache, or artifact; define its cleanup/rotation behavior; and test precedence so a stale persistent descriptor cannot silently shadow rotated S3 credentials.

PRA-1’s portable-to-SSRF regression and a default AWS CLI subprocess-boundary test also remain outstanding.

Signed-off-by: Aaron Erickson <aerickson@nvidia.com>

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 5

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (1)
src/lib/onboard/sandbox-reuse.ts (1)

125-125: 🗄️ Data Integrity & Integration | 🟠 Major | ⚡ Quick win

Persist the disabled-forward state before reused metadata.

Line 114 writes dashboardPort: 0 through updateReusedSandboxMetadata, but Line 125 writes dashboardForwardEnabled: false only afterward. If the process stops between these writes, the registry can retain dashboardForwardEnabled: true with dashboardPort: 0. Recovery can then treat a stopped forward as active.

Write dashboardForwardEnabled and dashboardPort together before calling updateReusedSandboxMetadata. Add an ordering test for the disabled path.

Proposed fix
-  input.updateReusedSandboxMetadata(
-    input.sandboxName,
-    input.agent,
-    input.model,
-    input.provider,
-    dashboardPort,
-    input.selectionVerified,
-    input.sandboxGpuConfig,
-  );
   (input.updateSandbox ?? registry.updateSandbox)(input.sandboxName, {
     ...getHermesDashboardRegistryFields(hermesDashboardState),
     dashboardForwardEnabled: manageDashboard,
+    dashboardPort,
     gatewayName: input.gatewayName,
     gatewayPort: input.gatewayPort,
   });
+  input.updateReusedSandboxMetadata(
+    input.sandboxName,
+    input.agent,
+    input.model,
+    input.provider,
+    dashboardPort,
+    input.selectionVerified,
+    input.sandboxGpuConfig,
+  );

As per path instructions, persisted state must converge across resume and recovery paths.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@src/lib/onboard/sandbox-reuse.ts` at line 125, Update the reused-sandbox
metadata flow around updateReusedSandboxMetadata so the disabled path persists
dashboardForwardEnabled: false and dashboardPort: 0 in the same write before any
subsequent metadata update. Preserve the enabled path behavior, and add an
ordering test that verifies the disabled state is written atomically before
updateReusedSandboxMetadata completes.

Source: Path instructions

🧹 Nitpick comments (2)
src/lib/onboard/experimental/portable-inference-source.ts (1)

137-140: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick win

Use overloads on hmac to remove the type assertions.

hmac returns Buffer | string. Lines 161-165 then need five as Buffer / as string assertions. Overloads make the return type follow the encoding argument, so the assertions disappear and a wrong usage becomes a compile error.

♻️ Proposed refactor
-function hmac(key: Buffer | string, value: string, encoding?: "hex"): Buffer | string {
-  const digest = createHmac("sha256", key).update(value);
-  return encoding === "hex" ? digest.digest("hex") : digest.digest();
-}
+function hmac(key: Buffer | string, value: string): Buffer;
+function hmac(key: Buffer | string, value: string, encoding: "hex"): string;
+function hmac(key: Buffer | string, value: string, encoding?: "hex"): Buffer | string {
+  const digest = createHmac("sha256", key).update(value);
+  return encoding === "hex" ? digest.digest("hex") : digest.digest();
+}

Then drop the as Buffer and as string assertions at lines 161-165.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@src/lib/onboard/experimental/portable-inference-source.ts` around lines 137 -
140, Update the hmac function with overloads that return Buffer when encoding is
omitted and string when encoding is "hex", while retaining the existing
implementation behavior. Then remove the related as Buffer and as string
assertions from the hmac call sites around the derived key and digest values so
TypeScript infers the correct types.
src/lib/onboard/experimental/portable-inference-source.test.ts (1)

171-177: 🔒 Security & Privacy | 🔵 Trivial | ⚡ Quick win

Assert the HTTPS-only curl guard.

The test verifies the Authorization header, secret redaction, and the process limits. It does not verify proto = "=https". If that line is removed from the curl config, this test stays green while the scheme restriction is gone. The scheme restriction is the control that keeps descriptor retrieval on HTTPS.

Add an assertion on the config text.

💚 Proposed addition
     expect(String(options.input)).toContain("Authorization: AWS4-HMAC-SHA256");
     expect(String(options.input)).not.toContain(TEST_SECRET_ACCESS_KEY);
+    expect(String(options.input)).toContain('proto = "=https"');
+    expect(String(options.input)).toContain("https://");
+    expect(String(options.input)).not.toContain("http://");
     expect(options).toMatchObject({

As per path instructions, "Migration tests must prove the superseded path is unreachable" and tests should flag assertions that "make a test pass without exercising its claim."

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@src/lib/onboard/experimental/portable-inference-source.test.ts` around lines
171 - 177, Extend the curl configuration assertions in the portable inference
source test to require the HTTPS-only setting `proto = "=https"` in the config
text. Keep the existing Authorization, secret-redaction, and process-limit
assertions unchanged.

Source: Path instructions

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@src/lib/onboard/experimental/portable-inference-source.test.ts`:
- Around line 184-204: Update the redaction test around
resolvePortableInferenceSource to use a captureError helper that throws if the
invocation returns normally, ensuring both branches always execute assertions.
For the bootstrap credential branch, assert the captured error contains the
generic message and excludes bootstrapSecret; for the credential descriptor
branch, retain the generic-message assertion and descriptorSecret exclusion.
- Around line 67-82: Make readPortableInferenceBootstrapFile accept an optional
runtime bootstrap path parameter, defaulting to the existing production path,
and use that parameter when selecting the runtime candidate. Update the test
around readPortableInferenceBootstrapFile to pass a non-existent temporary path
so it always exercises the Desktop branch without depending on host filesystem
state.

In `@src/lib/onboard/experimental/portable-inference-source.ts`:
- Around line 13-15: Document the accepted scope, owning team, object-content
rotation procedure, retirement/relocation plan, and compatibility, security, and
validation expectations for the fixed target constants
PORTABLE_INFERENCE_S3_REGION, PORTABLE_INFERENCE_S3_BUCKET, and
PORTABLE_INFERENCE_S3_KEY; include a link to the tracking GitHub issue and
ensure the lifecycle decision covers bucket or account changes.
- Around line 72-84: Update the bootstrap-reading function around the buffer
allocation and return path to clear the full buffer before throwing for empty or
oversized credentials, and return an independent copy of the bytes rather than
buffer.subarray(0, offset). Ensure resolvePortableInferenceSource’s existing
rawBootstrap.fill(0) clears all credential data, including any trailing byte
read past offset.

In `@src/lib/security/trusted-private-endpoint.ts`:
- Around line 275-284: Update the isOpenShellManagedHost() branch to return the
same rejection when options.allowExplicitLoopback is false, covering
inference.local and host.*.internal aliases while preserving existing acceptance
for non-portable flows. Add regression coverage for portable rejection of both
alias forms and allowed behavior when loopback access is enabled, keeping
deny-by-default and fail-closed handling intact.

---

Outside diff comments:
In `@src/lib/onboard/sandbox-reuse.ts`:
- Line 125: Update the reused-sandbox metadata flow around
updateReusedSandboxMetadata so the disabled path persists
dashboardForwardEnabled: false and dashboardPort: 0 in the same write before any
subsequent metadata update. Preserve the enabled path behavior, and add an
ordering test that verifies the disabled state is written atomically before
updateReusedSandboxMetadata completes.

---

Nitpick comments:
In `@src/lib/onboard/experimental/portable-inference-source.test.ts`:
- Around line 171-177: Extend the curl configuration assertions in the portable
inference source test to require the HTTPS-only setting `proto = "=https"` in
the config text. Keep the existing Authorization, secret-redaction, and
process-limit assertions unchanged.

In `@src/lib/onboard/experimental/portable-inference-source.ts`:
- Around line 137-140: Update the hmac function with overloads that return
Buffer when encoding is omitted and string when encoding is "hex", while
retaining the existing implementation behavior. Then remove the related as
Buffer and as string assertions from the hmac call sites around the derived key
and digest values so TypeScript infers the correct types.
🪄 Autofix

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: c8704d47-459d-410f-9065-82aa1ec6c1d1

📥 Commits

Reviewing files that changed from the base of the PR and between e2a45b8 and ab85412.

📒 Files selected for processing (11)
  • src/lib/onboard.ts
  • src/lib/onboard/command.test.ts
  • src/lib/onboard/command.ts
  • src/lib/onboard/experimental/portable-inference-source.test.ts
  • src/lib/onboard/experimental/portable-inference-source.ts
  • src/lib/onboard/forward-cleanup.test.ts
  • src/lib/onboard/forward-cleanup.ts
  • src/lib/onboard/inference-selection-validation.ts
  • src/lib/onboard/sandbox-reuse.test.ts
  • src/lib/onboard/sandbox-reuse.ts
  • src/lib/security/trusted-private-endpoint.ts
🚧 Files skipped from review as they are similar to previous changes (3)
  • src/lib/onboard.ts
  • src/lib/onboard/command.ts
  • src/lib/onboard/command.test.ts

Comment thread src/lib/onboard/experimental/portable-inference-source.test.ts
Comment thread src/lib/onboard/experimental/portable-inference-source.test.ts
Comment on lines +13 to +15
const PORTABLE_INFERENCE_S3_REGION = "us-east-2";
const PORTABLE_INFERENCE_S3_BUCKET = "gfn-ld-ai-poc-355178295565-us-east-2-an";
const PORTABLE_INFERENCE_S3_KEY = "GFNClawV2/secrets/nvcf-llm.b64";

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔒 Security & Privacy | 🟡 Minor | ⚡ Quick win

Confirm ownership and lifecycle for the fixed object-storage target.

Lines 13-15 hardcode a region, a bucket name that embeds an AWS account ID, and an object key. This pins the product to one externally owned location with no rotation or relocation path. If the bucket is retired, renamed, or its account changes, every portable onboarding run fails with a generic descriptor error.

Record the accepted scope decision, the owning team, the rotation procedure for the object contents, and the retirement plan. Link the tracking issue in GitHub.

As per coding guidelines, "Before implementing or approving a new supported integration, recipe, custom image, third-party stack, or other product surface, confirm an accepted scope decision and defined ownership, lifecycle, compatibility, security, and validation expectations."

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@src/lib/onboard/experimental/portable-inference-source.ts` around lines 13 -
15, Document the accepted scope, owning team, object-content rotation procedure,
retirement/relocation plan, and compatibility, security, and validation
expectations for the fixed target constants PORTABLE_INFERENCE_S3_REGION,
PORTABLE_INFERENCE_S3_BUCKET, and PORTABLE_INFERENCE_S3_KEY; include a link to
the tracking GitHub issue and ensure the lifecycle decision covers bucket or
account changes.

Source: Coding guidelines

Comment thread src/lib/onboard/experimental/portable-inference-source.ts
Comment on lines +275 to +284
// An explicit loopback host is valid only for flows that can select local inference.
if (isLoopbackHostname(hostname)) {
return options.allowExplicitLoopback !== false
? { ok: true, addresses: [] }
: {
ok: false,
reason: `endpoint host "${hostname}" is a private/internal address`,
reasonCode: "rejected",
};
}

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔒 Security & Privacy | 🟠 Major | ⚡ Quick win

Apply the portable loopback policy to OpenShell-managed aliases.

This branch rejects localhost, 127.0.0.1, and ::1 when allowExplicitLoopback is false. The following isOpenShellManagedHost() branch still accepts inference.local and host.*.internal without checking that option. A portable hosted-only flow can therefore still select a local managed endpoint.

Reject OpenShell-managed aliases when allowExplicitLoopback is false. Add regression coverage for portable rejection of inference.local and host.*.internal, while preserving allowed loopback behavior for non-portable flows.

Proposed fix
-  if (isOpenShellManagedHost(hostname)) return { ok: true, addresses: [] };
+  if (isOpenShellManagedHost(hostname)) {
+    return options.allowExplicitLoopback !== false
+      ? { ok: true, addresses: [] }
+      : {
+          ok: false,
+          reason: `endpoint host "${hostname}" is a private/internal address`,
+          reasonCode: "rejected",
+        };
+  }

As per path instructions, preserve deny-by-default and fail-closed handling at this security boundary.

📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
// An explicit loopback host is valid only for flows that can select local inference.
if (isLoopbackHostname(hostname)) {
return options.allowExplicitLoopback !== false
? { ok: true, addresses: [] }
: {
ok: false,
reason: `endpoint host "${hostname}" is a private/internal address`,
reasonCode: "rejected",
};
}
// An explicit loopback host is valid only for flows that can select local inference.
if (isLoopbackHostname(hostname)) {
return options.allowExplicitLoopback !== false
? { ok: true, addresses: [] }
: {
ok: false,
reason: `endpoint host "${hostname}" is a private/internal address`,
reasonCode: "rejected",
};
}
if (isOpenShellManagedHost(hostname)) {
return options.allowExplicitLoopback !== false
? { ok: true, addresses: [] }
: {
ok: false,
reason: `endpoint host "${hostname}" is a private/internal address`,
reasonCode: "rejected",
};
}
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@src/lib/security/trusted-private-endpoint.ts` around lines 275 - 284, Update
the isOpenShellManagedHost() branch to return the same rejection when
options.allowExplicitLoopback is false, covering inference.local and
host.*.internal aliases while preserving existing acceptance for non-portable
flows. Add regression coverage for portable rejection of both alias forms and
allowed behavior when loopback access is enabled, keeping deny-by-default and
fail-closed handling intact.

Source: Path instructions

@copy-pr-bot

copy-pr-bot Bot commented Aug 7, 2026

Copy link
Copy Markdown

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

Signed-off-by: Aaron Erickson <aerickson@nvidia.com>
Signed-off-by: Aaron Erickson <aerickson@nvidia.com>
Signed-off-by: Aaron Erickson <aerickson@nvidia.com>
Signed-off-by: Aaron Erickson <aerickson@nvidia.com>
Signed-off-by: Aaron Erickson <aerickson@nvidia.com>
Signed-off-by: Aaron Erickson <aerickson@nvidia.com>
Signed-off-by: Aaron Erickson <aerickson@nvidia.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area: inference Inference routing, serving, model selection, or outputs area: onboarding Onboarding FSM, provider setup, sandbox launch, or first-run flow feature PR adds or expands user-visible functionality security v0.0.105 Release target

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants