Repository navigation
test(cli): replace source-shape checks with behavior tests #3653
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
Changes from 4 commits
Commits
Show all changes
14 commits
Select commit
Hold shift + click to select a range
557e5bb
test: baseline source-shape test budget
cv a8ae851
test: replace image cleanup source-shape checks
cv 3e68531
test: replace gateway cleanup source-shape checks
cv 8b9314c
test: extract gateway container liveness behavior
cv 4aec0bc
test: keep source-shape budget at zero
cv 014542c
test: remove remaining source-shape assertions
cv 94b8026
Merge branch 'main' into refactor/replace-source-shape-tests
cv 3cda066
test: expand source-shape scanner and remove caught cases
cv d171e5f
test: catch and remove shell source-shape checks
cv 567b0bb
test: catch unreachable and matcher source-shape assertions
cv 4a699dc
test: tighten source-shape scanner edge cases
cv 58b2b5f
Merge remote-tracking branch 'origin/main' into refactor/replace-sour…
cv d6f9b51
Merge branch 'main' into refactor/replace-source-shape-tests
cv d8a6e4c
fix(source-shape): address scanner review feedback
cv File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Some comments aren't visible on the classic Files Changed page.
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,3 +1,3 @@ | ||
| { | ||
| "maxSourceShapeCases": 0 | ||
| "maxSourceShapeCases": 20 | ||
| } |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,30 @@ | ||
| // SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved. | ||
| // SPDX-License-Identifier: Apache-2.0 | ||
|
|
||
| import { describe, expect, it, vi } from "vitest"; | ||
|
|
||
| import { resolveSandboxImageTagFromCreateOutput } from "./image-tag"; | ||
|
|
||
| describe("resolveSandboxImageTagFromCreateOutput", () => { | ||
| it("uses the sandbox image tag reported by OpenShell create output", () => { | ||
| const warn = vi.fn(); | ||
|
|
||
| expect( | ||
| resolveSandboxImageTagFromCreateOutput( | ||
| "Uploaded\nBuilt image openshell/sandbox-from:1776766054\nCreated sandbox alpha", | ||
| "1776766054999", | ||
| warn, | ||
| ), | ||
| ).toBe("openshell/sandbox-from:1776766054"); | ||
| expect(warn).not.toHaveBeenCalled(); | ||
| }); | ||
|
|
||
| it("falls back to the millisecond build id when OpenShell output omits the image tag", () => { | ||
| const warn = vi.fn(); | ||
|
|
||
| expect(resolveSandboxImageTagFromCreateOutput("Created sandbox alpha", "1776766054999", warn)).toBe( | ||
| "openshell/sandbox-from:1776766054999", | ||
| ); | ||
| expect(warn).toHaveBeenCalledWith(expect.stringContaining("could not parse image tag")); | ||
| }); | ||
| }); |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,20 @@ | ||
| // SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved. | ||
| // SPDX-License-Identifier: Apache-2.0 | ||
|
|
||
| const BUILT_SANDBOX_IMAGE_RE = /Built image (openshell\/sandbox-from:\d+)/; | ||
|
|
||
| export function resolveSandboxImageTagFromCreateOutput( | ||
| output: string, | ||
| buildId: string, | ||
| warn: (message: string) => void = console.warn, | ||
| ): string { | ||
| const builtImageMatch = output.match(BUILT_SANDBOX_IMAGE_RE); | ||
| if (builtImageMatch?.[1]) { | ||
| return builtImageMatch[1]; | ||
| } | ||
|
|
||
| warn( | ||
| " Warning: could not parse image tag from build output; imageTag may be stale. Run 'nemoclaw gc' if destroy fails.", | ||
| ); | ||
| return `openshell/sandbox-from:${buildId}`; | ||
| } |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,47 @@ | ||
| // SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved. | ||
| // SPDX-License-Identifier: Apache-2.0 | ||
|
|
||
| import { describe, expect, it, vi } from "vitest"; | ||
|
|
||
| import { verifyGatewayContainerRunning } from "./gateway-container-running"; | ||
|
|
||
| function dockerInspectResult(status: number, stdout = "", stderr = "") { | ||
| return { status, stdout, stderr }; | ||
| } | ||
|
|
||
| describe("verifyGatewayContainerRunning", () => { | ||
| it("returns running when Docker reports a running gateway container", () => { | ||
| const dockerInspect = vi.fn(() => dockerInspectResult(0, "true\n")); | ||
|
|
||
| expect(verifyGatewayContainerRunning("nemoclaw", { dockerInspect })).toBe("running"); | ||
| expect(dockerInspect).toHaveBeenCalledWith( | ||
| ["--type", "container", "--format", "{{.State.Running}}", "openshell-cluster-nemoclaw"], | ||
| { ignoreError: true, suppressOutput: true }, | ||
| ); | ||
| }); | ||
|
|
||
| it("returns missing for existing but stopped containers", () => { | ||
| const dockerInspect = vi.fn(() => dockerInspectResult(0, "false\n")); | ||
|
|
||
| expect(verifyGatewayContainerRunning("nemoclaw", { dockerInspect })).toBe("missing"); | ||
| }); | ||
|
|
||
| it("returns missing when Docker reports the gateway container is absent", () => { | ||
| expect( | ||
| verifyGatewayContainerRunning("nemoclaw", { | ||
| dockerInspect: vi.fn(() => dockerInspectResult(1, "", "Error: No such object")), | ||
| }), | ||
| ).toBe("missing"); | ||
| expect( | ||
| verifyGatewayContainerRunning("nemoclaw", { | ||
| dockerInspect: vi.fn(() => dockerInspectResult(1, "", "No such container")), | ||
| }), | ||
| ).toBe("missing"); | ||
| }); | ||
|
|
||
| it("returns unknown for daemon or inspection failures", () => { | ||
| const dockerInspect = vi.fn(() => dockerInspectResult(1, "", "Cannot connect to Docker daemon")); | ||
|
|
||
| expect(verifyGatewayContainerRunning("nemoclaw", { dockerInspect })).toBe("unknown"); | ||
| }); | ||
| }); |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,38 @@ | ||
| // SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved. | ||
| // SPDX-License-Identifier: Apache-2.0 | ||
|
|
||
| export type GatewayContainerState = "running" | "missing" | "unknown"; | ||
|
|
||
| type DockerInspect = ( | ||
| args: string[], | ||
| opts: { ignoreError: true; suppressOutput: true }, | ||
| ) => { status: number | null; stdout?: unknown; stderr?: unknown }; | ||
|
|
||
| export function verifyGatewayContainerRunning( | ||
| gatewayName: string, | ||
| deps: { dockerInspect?: DockerInspect } = {}, | ||
| ): GatewayContainerState { | ||
| const inspect = | ||
| deps.dockerInspect ?? | ||
| (require("../adapters/docker") as { dockerInspect: DockerInspect }).dockerInspect; | ||
| const containerName = `openshell-cluster-${gatewayName}`; | ||
| const result = inspect( | ||
| ["--type", "container", "--format", "{{.State.Running}}", containerName], | ||
| { ignoreError: true, suppressOutput: true }, | ||
| ); | ||
|
|
||
| if (result.status === 0 && String(result.stdout || "").trim() === "true") { | ||
| return "running"; | ||
| } | ||
|
|
||
| if (result.status === 0) { | ||
| return "missing"; | ||
| } | ||
|
|
||
| const stderr = (result.stderr || "").toString(); | ||
| if (stderr.includes("No such object") || stderr.includes("No such container")) { | ||
| return "missing"; | ||
| } | ||
|
|
||
| return "unknown"; | ||
| } |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,74 @@ | ||
| // SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved. | ||
| // SPDX-License-Identifier: Apache-2.0 | ||
|
|
||
| import { describe, expect, it, vi } from "vitest"; | ||
|
|
||
| import { destroyGatewayWithVolumeCleanup, type DestroyGatewayDeps } from "./gateway-destroy"; | ||
|
|
||
| function deps(overrides: Partial<DestroyGatewayDeps> = {}): DestroyGatewayDeps { | ||
| return { | ||
| clearRegistry: vi.fn(), | ||
| dockerRemoveVolumesByPrefix: vi.fn(), | ||
| gatewayName: "nemoclaw", | ||
| hasLifecycleCommands: vi.fn(() => true), | ||
| isDockerDriverGatewayEnabled: vi.fn(() => false), | ||
| removeDockerDriverGatewayRegistration: vi.fn(() => true), | ||
| runOpenshell: vi.fn(() => ({ status: 0 })), | ||
| stopDockerDriverGatewayProcess: vi.fn(), | ||
| ...overrides, | ||
| }; | ||
| } | ||
|
|
||
| describe("destroyGatewayWithVolumeCleanup", () => { | ||
| it("destroys lifecycle gateways and removes their OpenShell cluster volumes", () => { | ||
| const d = deps(); | ||
|
|
||
| expect(destroyGatewayWithVolumeCleanup(d)).toBe(true); | ||
|
|
||
| expect(d.runOpenshell).toHaveBeenCalledWith(["gateway", "destroy", "-g", "nemoclaw"], { | ||
| ignoreError: true, | ||
| }); | ||
| expect(d.clearRegistry).toHaveBeenCalledOnce(); | ||
| expect(d.dockerRemoveVolumesByPrefix).toHaveBeenCalledWith("openshell-cluster-nemoclaw", { | ||
| ignoreError: true, | ||
| }); | ||
| }); | ||
|
|
||
| it("stops Docker-driver gateways, unregisters them, and removes cluster volumes", () => { | ||
| const d = deps({ | ||
| hasLifecycleCommands: vi.fn(() => false), | ||
| isDockerDriverGatewayEnabled: vi.fn(() => true), | ||
| }); | ||
|
|
||
| expect(destroyGatewayWithVolumeCleanup(d)).toBe(true); | ||
|
|
||
| expect(d.stopDockerDriverGatewayProcess).toHaveBeenCalledOnce(); | ||
| expect(d.removeDockerDriverGatewayRegistration).toHaveBeenCalledOnce(); | ||
| expect(d.runOpenshell).not.toHaveBeenCalled(); | ||
| expect(d.clearRegistry).toHaveBeenCalledOnce(); | ||
| expect(d.dockerRemoveVolumesByPrefix).toHaveBeenCalledWith("openshell-cluster-nemoclaw", { | ||
| ignoreError: true, | ||
| }); | ||
| }); | ||
|
|
||
| it("does not clear registry or remove volumes when gateway removal fails", () => { | ||
| const d = deps({ runOpenshell: vi.fn(() => ({ status: 1 })) }); | ||
|
|
||
| expect(destroyGatewayWithVolumeCleanup(d)).toBe(false); | ||
|
|
||
| expect(d.clearRegistry).not.toHaveBeenCalled(); | ||
| expect(d.dockerRemoveVolumesByPrefix).not.toHaveBeenCalled(); | ||
| }); | ||
|
|
||
| it("preserves legacy gateway behavior without Docker volume cleanup", () => { | ||
| const d = deps({ hasLifecycleCommands: vi.fn(() => false) }); | ||
|
|
||
| expect(destroyGatewayWithVolumeCleanup(d)).toBe(true); | ||
|
|
||
| expect(d.runOpenshell).toHaveBeenCalledWith(["gateway", "remove", "nemoclaw"], { | ||
| ignoreError: true, | ||
| }); | ||
| expect(d.clearRegistry).toHaveBeenCalledOnce(); | ||
| expect(d.dockerRemoveVolumesByPrefix).not.toHaveBeenCalled(); | ||
| }); | ||
| }); |
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.