Repository navigation
test(e2e): make snapshot commands inference hermetic #6621
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
Changes from 5 commits
Commits
Show all changes
12 commits
Select commit
Hold shift + click to select a range
71b0013
test(e2e): make snapshot commands inference hermetic
harjothkhara 2419a7c
test(e2e): prove snapshot inference auth in sandbox
harjothkhara be308d3
test(e2e): use secret-shaped snapshot fixture key
harjothkhara 05f191c
Merge branch 'main' into codex/hermetic-snapshot-e2e
harjothkhara 9e77269
test(e2e): cover snapshot hermetic inference env guard
harjothkhara 79fe326
test(e2e): assert snapshot env drops ambient NVIDIA credentials
harjothkhara 228dd07
Merge branch 'main' into codex/hermetic-snapshot-e2e
cv 39370a0
Merge branch 'main' into codex/hermetic-snapshot-e2e
cv 5c627d4
Merge branch 'main' into codex/hermetic-snapshot-e2e
cv be98731
test(e2e): map snapshot commands mock parity
cv ecc9351
Merge branch 'main' into codex/hermetic-snapshot-e2e
cv a704634
Merge branch 'main' into codex/hermetic-snapshot-e2e
cv File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,45 @@ | ||
| // SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved. | ||
| // SPDX-License-Identifier: Apache-2.0 | ||
|
|
||
| import { buildAvailabilityProbeEnv } from "../fixtures/availability-env.ts"; | ||
|
|
||
| export interface SnapshotInferenceFixture { | ||
| apiKey: string; | ||
| endpointUrl: string; | ||
| model: string; | ||
| } | ||
|
|
||
| /** | ||
| * Builds the child env for the snapshot-commands live target. | ||
| * | ||
| * `NEMOCLAW_E2E_USE_HOSTED_INFERENCE` is in the fixture env allowlist, so an | ||
| * ambient value is forwarded into the child. `stageHostedInferenceSourceSecretEnv` | ||
| * treats that flag alone as sufficient to force hosted-custom staging even when | ||
| * `COMPATIBLE_API_KEY` names an explicit endpoint, which would silently route | ||
| * this target back at hosted inference. Strip it so the target stays hermetic. | ||
| */ | ||
| export function buildSnapshotCommandEnv( | ||
| sandboxName: string, | ||
| inference?: SnapshotInferenceFixture, | ||
| ): NodeJS.ProcessEnv { | ||
| const env: NodeJS.ProcessEnv = { | ||
| ...buildAvailabilityProbeEnv(), | ||
| NEMOCLAW_ACCEPT_THIRD_PARTY_SOFTWARE: "1", | ||
| NEMOCLAW_NON_INTERACTIVE: "1", | ||
| NEMOCLAW_RECREATE_SANDBOX: "1", | ||
| NEMOCLAW_SANDBOX_NAME: sandboxName, | ||
| OPENSHELL_GATEWAY: process.env.OPENSHELL_GATEWAY ?? "nemoclaw", | ||
| }; | ||
| delete env.NEMOCLAW_E2E_USE_HOSTED_INFERENCE; | ||
| if (inference) { | ||
| Object.assign(env, { | ||
| COMPATIBLE_API_KEY: inference.apiKey, | ||
| NEMOCLAW_COMPAT_MODEL: inference.model, | ||
| NEMOCLAW_ENDPOINT_URL: inference.endpointUrl, | ||
| NEMOCLAW_MODEL: inference.model, | ||
| NEMOCLAW_PREFERRED_API: "openai-completions", | ||
| NEMOCLAW_PROVIDER: "custom", | ||
| }); | ||
| } | ||
| return env; | ||
| } |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,77 @@ | ||
| // SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved. | ||
| // SPDX-License-Identifier: Apache-2.0 | ||
|
|
||
| import { afterEach, describe, expect, it } from "vitest"; | ||
|
|
||
| import { buildAvailabilityProbeEnv } from "../fixtures/availability-env.ts"; | ||
| import { buildSnapshotCommandEnv } from "../live/snapshot-commands-helpers.ts"; | ||
|
|
||
| const HOSTED_FLAG = "NEMOCLAW_E2E_USE_HOSTED_INFERENCE"; | ||
| const SANDBOX_NAME = "e2e-snapshot"; | ||
|
|
||
| const INFERENCE = { | ||
| apiKey: "nvapi-snapshot-commands-fixture-credential", | ||
| endpointUrl: "http://host.openshell.internal:31337/v1", | ||
| model: "snapshot-commands-model", | ||
| }; | ||
|
|
||
| afterEach(() => { | ||
| delete process.env[HOSTED_FLAG]; | ||
| }); | ||
|
|
||
| describe("snapshot commands live env helper", () => { | ||
| it("forwards an ambient hosted-inference flag through the shared probe env", () => { | ||
| process.env[HOSTED_FLAG] = "1"; | ||
|
|
||
| // Negative control. The strip below only matters while the flag is | ||
| // allowlisted for forwarding; if this ever stops holding, the strip | ||
| // assertion would pass vacuously and the hermeticity guard would rot. | ||
| expect(buildAvailabilityProbeEnv()[HOSTED_FLAG]).toBe("1"); | ||
| }); | ||
|
|
||
| it("strips an ambient hosted-inference flag so the target stays hermetic", () => { | ||
| process.env[HOSTED_FLAG] = "1"; | ||
|
|
||
| const env = buildSnapshotCommandEnv(SANDBOX_NAME, INFERENCE); | ||
|
|
||
| expect(env[HOSTED_FLAG]).toBeUndefined(); | ||
| expect(Object.hasOwn(env, HOSTED_FLAG)).toBe(false); | ||
| }); | ||
|
|
||
| it("strips the hosted-inference flag even when no inference fixture is staged", () => { | ||
| process.env[HOSTED_FLAG] = "1"; | ||
|
|
||
| expect(buildSnapshotCommandEnv(SANDBOX_NAME)[HOSTED_FLAG]).toBeUndefined(); | ||
| }); | ||
|
|
||
| it("stages the compatible endpoint against the custom provider", () => { | ||
| const env = buildSnapshotCommandEnv(SANDBOX_NAME, INFERENCE); | ||
|
|
||
| expect(env).toMatchObject({ | ||
| COMPATIBLE_API_KEY: INFERENCE.apiKey, | ||
| NEMOCLAW_COMPAT_MODEL: INFERENCE.model, | ||
| NEMOCLAW_ENDPOINT_URL: INFERENCE.endpointUrl, | ||
| NEMOCLAW_MODEL: INFERENCE.model, | ||
| NEMOCLAW_PREFERRED_API: "openai-completions", | ||
| NEMOCLAW_PROVIDER: "custom", | ||
| NEMOCLAW_SANDBOX_NAME: SANDBOX_NAME, | ||
| }); | ||
| }); | ||
|
|
||
| it("leaves inference selection unset when no fixture is staged", () => { | ||
| const env = buildSnapshotCommandEnv(SANDBOX_NAME); | ||
|
|
||
| expect(env.COMPATIBLE_API_KEY).toBeUndefined(); | ||
| expect(env.NEMOCLAW_ENDPOINT_URL).toBeUndefined(); | ||
| expect(env.NEMOCLAW_PROVIDER).toBeUndefined(); | ||
| }); | ||
|
|
||
| it("never exposes a hosted NVIDIA inference credential to the child env", () => { | ||
| process.env[HOSTED_FLAG] = "1"; | ||
|
|
||
| const env = buildSnapshotCommandEnv(SANDBOX_NAME, INFERENCE); | ||
|
|
||
| expect(env.NVIDIA_INFERENCE_API_KEY).toBeUndefined(); | ||
| expect(env.NVIDIA_API_KEY).toBeUndefined(); | ||
| }); | ||
| }); | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.