Repository navigation
fix(onboard): fail fast when Docker is unreachable during gateway start #4128
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
Changes from all commits
Commits
Show all changes
6 commits
Select commit
Hold shift + click to select a range
1d338f9
fix(onboard): short-circuit gateway start when Docker daemon is unrea…
chengjiew cccedd2
fix(onboard): keep gateway failure handling out of onboard entrypoint
chengjiew 6282137
fix(onboard): classify ANSI-stripped gateway start output
chengjiew 15d629c
Merge branch 'main' into fix/2347-onboard-docker-hang-v2
cv 34008e2
Merge branch 'main' into fix/2347-onboard-docker-hang-v2
cv 20fee19
test(onboard): cover docker-unreachable gateway start
cv File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,80 @@ | ||
| // SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved. | ||
| // SPDX-License-Identifier: Apache-2.0 | ||
|
|
||
| import { describe, expect, it, vi } from "vitest"; | ||
|
|
||
| import { classifyGatewayStartFailure } from "../validation"; | ||
| import { reportLegacyGatewayStartResultFailure } from "./gateway-start-failure"; | ||
|
|
||
| describe("classifyGatewayStartFailure", () => { | ||
| // Regression: NemoClaw #2347. When Colima is stopped on macOS, the | ||
| // openshell gateway-start stream prints "Failed to create Docker client. | ||
| // Socket not found: /var/run/docker.sock" before exiting non-zero. Onboard | ||
| // must short-circuit the retry loop with an actionable message instead of | ||
| // burning ~15 minutes on health polls against a dead socket. | ||
| it("detects colima-stopped signature on macOS (Socket not found)", () => { | ||
| const output = [ | ||
| " Error: Failed to create Docker client.", | ||
| " Socket not found: /var/run/docker.sock", | ||
| ].join("\n"); | ||
| expect(classifyGatewayStartFailure(output)).toEqual({ kind: "docker_unreachable" }); | ||
| }); | ||
|
|
||
| it("detects dockerd-stopped signature on Linux (Cannot connect to the Docker daemon)", () => { | ||
| const output = | ||
| "Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running?"; | ||
| expect(classifyGatewayStartFailure(output)).toEqual({ kind: "docker_unreachable" }); | ||
| }); | ||
|
|
||
| it("detects the standalone 'Failed to create Docker client' error marker", () => { | ||
| expect(classifyGatewayStartFailure("Failed to create Docker client")).toEqual({ | ||
| kind: "docker_unreachable", | ||
| }); | ||
| }); | ||
|
|
||
| it("does not match historical mentions of 'Failed to create Docker client'", () => { | ||
| const output = | ||
| "client created successfully; previous Failed to create Docker client issue fixed"; | ||
| expect(classifyGatewayStartFailure(output)).toEqual({ kind: "unknown" }); | ||
| }); | ||
|
|
||
| it("detects free-form 'docker daemon is not running' wording", () => { | ||
| expect(classifyGatewayStartFailure("the docker daemon is not running on this host")).toEqual({ | ||
| kind: "docker_unreachable", | ||
| }); | ||
| }); | ||
|
|
||
| it("returns unknown for healthy-but-slow output (should not short-circuit)", () => { | ||
| // Real output seen during a slow first-time k3s bootstrap — the retry | ||
| // loop must stay engaged for these, so we must not misclassify them. | ||
| const output = [ | ||
| "Applying HelmChart openshell", | ||
| "openshell-0 still starting", | ||
| "Observed pod startup duration 90s", | ||
| ].join("\n"); | ||
| expect(classifyGatewayStartFailure(output)).toEqual({ kind: "unknown" }); | ||
| }); | ||
|
|
||
| it("returns unknown for empty or missing output", () => { | ||
| expect(classifyGatewayStartFailure("")).toEqual({ kind: "unknown" }); | ||
| expect(classifyGatewayStartFailure()).toEqual({ kind: "unknown" }); | ||
| }); | ||
| }); | ||
|
|
||
| describe("reportLegacyGatewayStartResultFailure", () => { | ||
| it("classifies Docker-unreachable output after stripping ANSI sequences (#2347)", () => { | ||
| const log = vi.fn(); | ||
| const output = [ | ||
| "\x1b[31mError: Failed to create Docker client.\x1b[0m", | ||
| "\x1b[33mSocket not found: /var/run/docker.sock\x1b[0m", | ||
| ].join("\n"); | ||
|
|
||
| expect(reportLegacyGatewayStartResultFailure(output, log)).toEqual({ | ||
| kind: "docker_unreachable", | ||
| }); | ||
| expect(log).toHaveBeenCalledWith( | ||
| expect.stringContaining("Gateway start returned before healthy"), | ||
| ); | ||
| expect(log.mock.calls[0][0]).not.toContain("\x1b"); | ||
| }); | ||
| }); |
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
🧩 Analysis chain
🌐 Web query:
What is the current recommended commit SHA to pin actions/setup-node v6?💡 Result:
Current recommended commit SHA to pin actions/setup-node v6 (as of 2026-06-03) is: 48b55a011bda9f5d6aeb4c2d9c7362e8dae4041e. Rationale (how this was determined): GitHub’s actions/setup-node repository shows the latest v6 release as v6.4.0 [1][2][3]. The full commit SHA for that tagged release is shown in the v6.3.0...v6.4.0 comparison page, which lists the v6.4.0 head commit as 48b55a011bda9f5d6aeb4c2d9c7362e8dae4041e [4].
Citations:
Harden
docker-unreachable-gateway-start-e2e’s checkout/setup-node to match the sibling job.This job (lines 203-209) doesn’t set
persist-credentials: falseonactions/checkoutor pinactions/setup-node@v6; mirror the hardened pattern used bystrict-tool-call-probe-e2efor supply-chain hygiene.🔒 Proposed hardening
- name: Checkout uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2 + with: + persist-credentials: false - name: Setup Node - uses: actions/setup-node@v6 + uses: actions/setup-node@48b55a011bda9f5d6aeb4c2d9c7362e8dae4041e # v6 with: node-version: "22"🧰 Tools
🪛 zizmor (1.25.2)
[warning] 203-204: credential persistence through GitHub Actions artifacts (artipacked): does not set persist-credentials: false
(artipacked)
[error] 207-207: unpinned action reference (unpinned-uses): action is not pinned to a hash (required by blanket policy)
(unpinned-uses)
🤖 Prompt for AI Agents