Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
47 changes: 1 addition & 46 deletions .github/workflows/regression-e2e.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,7 @@ on:
jobs:
description: >-
Comma-separated regression job names to run (empty = all).
Valid: dashboard-remote-bind-e2e,gateway-health-honest-e2e,docker-unreachable-gateway-start-e2e,gateway-drift-preflight-e2e,openshell-version-pin-e2e,onboard-inference-smoke-e2e,strict-tool-call-probe-e2e,model-router-provider-routed-inference-e2e,openclaw-plugin-runtime-exdev-e2e,whatsapp-qr-compact-e2e
Valid: dashboard-remote-bind-e2e,gateway-health-honest-e2e,gateway-drift-preflight-e2e,openshell-version-pin-e2e,onboard-inference-smoke-e2e,strict-tool-call-probe-e2e,model-router-provider-routed-inference-e2e,openclaw-plugin-runtime-exdev-e2e,whatsapp-qr-compact-e2e
required: false
type: string
default: ""
Expand All @@ -46,7 +46,6 @@ jobs:
outputs:
dashboard: ${{ steps.select.outputs.dashboard }}
gateway: ${{ steps.select.outputs.gateway }}
docker_unreachable_gateway_start: ${{ steps.select.outputs.docker_unreachable_gateway_start }}
gateway_drift_preflight: ${{ steps.select.outputs.gateway_drift_preflight }}
openshell_version_pin: ${{ steps.select.outputs.openshell_version_pin }}
onboard_inference_smoke: ${{ steps.select.outputs.onboard_inference_smoke }}
Expand Down Expand Up @@ -81,12 +80,6 @@ jobs:
echo "gateway=false" >> "$GITHUB_OUTPUT"
fi

if [ -z "$normalized" ] || includes_job "docker-unreachable-gateway-start-e2e"; then
echo "docker_unreachable_gateway_start=true" >> "$GITHUB_OUTPUT"
else
echo "docker_unreachable_gateway_start=false" >> "$GITHUB_OUTPUT"
fi

if [ -z "$normalized" ] || includes_job "gateway-drift-preflight-e2e"; then
echo "gateway_drift_preflight=true" >> "$GITHUB_OUTPUT"
else
Expand Down Expand Up @@ -194,44 +187,6 @@ jobs:
/tmp/nemoclaw-e2e-gateway-health-honest-process.log
if-no-files-found: ignore

# ── Docker-unreachable gateway-start E2E ───────────────────
# Coverage guard for #2347's fallback path. Preflight catches Docker
# daemon outages before step [2/8] on the normal path; this focused
# regression simulates Docker dying during `openshell gateway start` and
# verifies onboard exits before health polling with Docker recovery advice.
docker-unreachable-gateway-start-e2e:
needs: select_regression_jobs
if: >-
github.repository == 'NVIDIA/NemoClaw' &&
needs.select_regression_jobs.outputs.docker_unreachable_gateway_start == 'true'
runs-on: ubuntu-latest
timeout-minutes: 15
steps:
- name: Checkout
uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6.0.3

- name: Setup Node
uses: actions/setup-node@v6
with:
node-version: "22"

- name: Run Docker-unreachable gateway-start E2E test
env:
NEMOCLAW_NON_INTERACTIVE: "1"
NEMOCLAW_ACCEPT_THIRD_PARTY_SOFTWARE: "1"
run: bash test/e2e/test-docker-unreachable-gateway-start.sh

- name: Upload Docker-unreachable gateway-start logs on failure
if: failure()
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
with:
name: docker-unreachable-gateway-start-logs
path: |
/tmp/nemoclaw-e2e-docker-unreachable-gateway-start.log
/tmp/nemoclaw-e2e-docker-unreachable-gateway-start-node.log
/tmp/nemoclaw-e2e-docker-unreachable-gateway-start-openshell.log
if-no-files-found: ignore

# ── OpenShell version-pin E2E ──────────────────────────────
# Coverage guard for #3474. If a host has sticky OpenShell 0.0.45 on PATH
# but this NemoClaw release supports only <=0.0.44, install-openshell.sh
Expand Down
2 changes: 1 addition & 1 deletion ci/test-file-size-budget.json
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@
"test/nemoclaw-start.test.ts": 5289,
"test/onboard-messaging.test.ts": 2097,
"test/onboard-selection.test.ts": 6922,
"test/onboard.test.ts": 4874,
"test/onboard.test.ts": 4783,
"test/policies.test.ts": 2763
}
}
62 changes: 8 additions & 54 deletions src/lib/onboard/gateway-start-failure-integration.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -5,31 +5,24 @@
// startGatewayWithOptions() takes when `openshell gateway start` reports
// the Docker daemon is not reachable. See src/lib/onboard.ts:2233.
//
// This file replaces test/e2e/test-docker-unreachable-gateway-start.sh,
// which was structurally a Node-process unit test of startGateway() with a
// PATH-shimmed openshell binary, not a sandbox-lifecycle e2e.
// This helper-level suite preserves coverage from the former
// test/e2e/test-docker-unreachable-gateway-start.sh, which was structurally
// a Node-process unit test of startGateway() with a PATH-shimmed openshell
// binary, not a sandbox-lifecycle e2e.
//
// Original regression: NemoClaw #2347.
// Owning migration issue: NemoClaw #4355.
//
// Coverage strategy: prove the contract through two layers:
// Coverage strategy: prove the helper-level contract through two layers:
//
// 1. Unit tests of the already-exported helpers (printDockerDaemonRecovery,
// handleFinalGatewayStartFailure with dockerUnreachable=true).
// 2. A composition test that runs the same helper sequence the call site
// uses (classify → handleFinal → exitProcess(1)).
//
// What this file deliberately DOES NOT cover (gap documented as it.todo
// at the bottom of the suite, with a follow-up issue tracking the
// refactor): direct executable proof that startGatewayWithOptions, on a
// docker-unreachable streamGatewayStart() result, (a) throws
// pRetry.AbortError instead of retrying, (b) never logs "Waiting for
// gateway health...", and (c) never calls openshell `status` or
// `gateway info` probes. Closing that gap requires either mocking the
// ~10 module-internal closures `startGatewayWithOptions` touches before
// reaching streamGatewayStart, or extracting the inner pRetry async body
// into an exported helper that takes streamGatewayStart as a DI
// parameter. Both changes are out of scope for the retirement PR.
// The caller-level process regression that drives startGateway() through a
// PATH-shimmed openshell binary lives in
// test/onboard-gateway-docker-unreachable.test.ts.

import { describe, expect, it, vi } from "vitest";
// `handleFinalGatewayStartFailure` is exposed via `module.exports = {...}` at
Expand Down Expand Up @@ -310,45 +303,6 @@ describe("startGatewayWithOptions docker-unreachable abort (#2347)", () => {
});
});

// ── Documented coverage gap: caller-level contracts that need a
// behavior-level seam (out of scope for this retirement PR) ─────────
//
// The legacy bash script directly executed `startGateway()` with a
// PATH-shimmed openshell binary and asserted runtime behavior of the
// call site. The unit + composition tests above cover the same ground
// for everything reachable through already-exported helpers, but three
// call-site contracts cannot be proven without driving the actual
// `startGatewayWithOptions` orchestrator past its ~200 lines of
// gateway-reuse / ssh-keygen / known_hosts / docker-driver-detect
// preamble. Doing that in a focused way requires either:
//
// (a) ~10 vi.mock() calls on module-internal closures inside
// onboard.ts (brittle), or
// (b) a small refactor extracting the inner pRetry async body of
// startGatewayWithOptions into an exported helper that takes
// streamGatewayStart as a DI parameter.
//
// Both are out of scope for this retirement PR. Follow-up issue
// [#5113](https://github.com/NVIDIA/NemoClaw/issues/5113) tracks
// landing option (b) and converting these `it.todo` placeholders into
// real assertions. Until then, the
// primary safety net for these contracts is code review on
// src/lib/onboard.ts:startGatewayWithOptions plus the existing
// classifyGatewayStartFailure / printDockerDaemonRecovery /
// handleFinalGatewayStartFailure unit tests above.

describe("call-site contracts (caller-level coverage gap)", () => {
it.todo(
"startGateway aborts via pRetry.AbortError without entering health-poll loop on docker-unreachable streamGatewayStart output",
);
it.todo(
"startGateway never invokes openshell `status` or `gateway info` after docker-unreachable streamGatewayStart output",
);
it.todo(
"startGateway forwards dockerUnreachable=true to handleFinalGatewayStartFailure (no doctor logs collection, no destroyGateway cleanup) when streamGatewayStart returns docker-unreachable signature",
);
});

// ── Sanity: classifyGatewayStartFailure recognises both signatures ─────
// (Already covered in gateway-start-failure.test.ts; this is a pinning
// assertion for the two strings the legacy script generated, kept here so
Expand Down
11 changes: 0 additions & 11 deletions test/e2e-scenario/migration/legacy-inventory.json
Original file line number Diff line number Diff line change
Expand Up @@ -266,17 +266,6 @@
"deletionReady": false,
"notes": "Initial completeness row; classify detailed coverage and deletion evidence in the owning migration issue before deleting."
},
{
"legacyScript": "test/e2e/test-docker-unreachable-gateway-start.sh",
"domain": "sandbox-lifecycle",
"ownerIssue": "#4355",
"status": "not-migrated",
"targetVitestScenarios": [],
"bridgeProbes": [],
"retiredReason": "",
"deletionReady": false,
"notes": "PR #5109 added Vitest helper coverage at src/lib/onboard/gateway-start-failure-integration.test.ts (two-layer strategy: unit tests of printDockerDaemonRecovery + handleFinalGatewayStartFailure, plus a composition test running the same classify -> handleFinal -> exitProcess(1) sequence the call site uses). Full retirement (delete this script + remove the docker-unreachable-gateway-start-e2e job in regression-e2e.yaml) was deferred per PR Review Advisor feedback because three caller-level contracts (no health-poll loop entry, no openshell status/gateway-info probes after gateway-start failure, dockerUnreachable forwarded to handleFinalGatewayStartFailure) are not directly proven without a behavior-level seam in startGatewayWithOptions. Follow-up issue #5113 tracks extracting attemptGatewayStart and converting the three it.todo placeholders into real assertions; deletion + retirement happen in a follow-up PR after that lands. Refs: #2347, #4355, #5113."
},
{
"legacyScript": "test/e2e/test-docs-validation.sh",
"domain": "support-diagnostics",
Expand Down
162 changes: 0 additions & 162 deletions test/e2e/test-docker-unreachable-gateway-start.sh

This file was deleted.

Loading
Loading