refactor(e2e): add a typed target evidence API by jyaunches · Pull Request #6359 · NVIDIA/NemoClaw

jyaunches · 2026-07-06T23:38:53Z

Summary

Add a typed target-evidence facade to the existing artifact sink and migrate live Vitest targets to it. Target metadata/results now retain the same artifact filenames and redaction boundary while receiving a consistent runner, result status, and contract field shape.

Related Issue

Closes #6353
Parent epic: #6346

Changes

Add artifacts.target.declare(...) and artifacts.target.complete(...) with typed extensible metadata/result inputs.
Emit target.json and target-result.json through the existing redacting ArtifactSink.
Supply runner: "vitest" centrally and default completed results without an explicit status to status: "passed".
Normalize legacy singular contract values to the canonical contracts array and reject conflicting or malformed contract fields.
Validate non-empty target IDs and explicit result statuses.
Migrate all 117 live target evidence writes and remove repeated runner fields.
Add focused support tests for emitted files, normalization, validation, redaction, and adoption.

Type of Change

Code change (feature, bug fix, or refactor)
Code change with doc updates
Doc only (prose changes, no code sample modifications)
Doc only (includes code sample changes)

Quality Gates

Tests added or updated for changed behavior
Existing tests cover changed behavior — justification:
Tests not applicable — justification:
Docs updated for user-facing behavior changes
Docs not applicable — justification: test-evidence refactor with no user-facing behavior
Sensitive paths changed (security, policy, credentials, preflight, onboarding, inference, runner, sandbox, or messaging)
Sensitive-path review completed or maintainer-approved waiver recorded — reviewer/approval link/justification:
Non-success, skipped, or missing CI check accepted by maintainer — check name, approval link, and follow-up issue:

Verification

PR description includes the DCO sign-off declaration and every commit appears as Verified in GitHub
Normal pre-commit, commit-msg, and pre-push hooks passed, or npm run check:diff passed when hooks were skipped or unavailable
Targeted behavior tests pass for the current change set, or tests are marked not applicable above — 28 focused artifact, fixture, and redaction assertions passed
Applicable broad gate passed — npm test for broad runtime/test-harness changes; npm run check for repo-wide validation/coverage changes — command/result:
Quality Gates section completed with required justifications or waivers
No secrets, API keys, or credentials committed
npm run docs builds without warnings (doc changes only)
Doc pages follow the style guide (doc changes only)
New doc pages include SPDX header and frontmatter (new pages only)

Additional local verification:

npm run build:cli
npm run typecheck
npm run typecheck:cli
npm run lint
npx vitest run --project e2e-support test/e2e/support/e2e-target-evidence.test.ts test/e2e/support/e2e-fixture-context.test.ts test/e2e/support/e2e-redaction-entry.test.ts

Signed-off-by: Julie Yaunches jyaunches@nvidia.com

Summary by CodeRabbit

New Features
- Added a typed E2E “target evidence” lifecycle with explicit declare/complete steps, producing consistent, normalized evidence outputs.
Bug Fixes
- Improved validation and normalization (non-empty identifiers, non-empty status when provided, enforce single contract vs contracts, consistent contract array formatting, and automatic result defaults).
- Added secret redaction in persisted evidence details.
Tests
- Migrated many live E2E scenarios to use the typed evidence API.
- Added test coverage for evidence normalization/redaction and a safeguard scan to prevent direct JSON target writes.

coderabbitai · 2026-07-06T23:39:11Z

Note

Reviews paused

It looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the reviews.auto_review.auto_pause_after_reviewed_commits setting.

Use the following commands to manage reviews:

@coderabbitai resume to resume automatic reviews.
@coderabbitai review to trigger a single review.

Use the checkboxes below for quick actions:

▶️ Resume reviews
🔍 Trigger review

📝 Walkthrough

Walkthrough

Adds a typed target evidence API in the E2E fixture layer, adds normalization/validation coverage, and migrates live E2E specs from direct target.json / target-result.json writes to artifacts.target.declare() and artifacts.target.complete().

Changes

Typed Target Evidence API

Layer / File(s)	Summary
Core typed evidence API and wiring `test/e2e/fixtures/artifacts.ts`	Adds target evidence types, normalization rules, the target evidence writer, and `ArtifactSink` wiring for target evidence output.
Support tests for target evidence `test/e2e/support/e2e-target-evidence.test.ts`	Adds tests for normalized target output, validation failures, redaction behavior, and a repository scan that rejects direct `target.json` and `target-result.json` writes in live specs.
Live test migration: agent-turn-latency through cron-preflight `test/e2e/live/agent-turn-latency.test.ts`, `test/e2e/live/bedrock-runtime-compatible-anthropic.test.ts`, `test/e2e/live/brave-search.test.ts`, `test/e2e/live/channels-add-remove.test.ts`, `test/e2e/live/channels-stop-start-helpers.ts`, `test/e2e/live/cloud-inference.test.ts`, `test/e2e/live/cloud-onboard.test.ts`, `test/e2e/live/common-egress-agent.test.ts`, `test/e2e/live/concurrent-gateway-ports.test.ts`, `test/e2e/live/credential-migration.test.ts`, `test/e2e/live/credential-sanitization.test.ts`, `test/e2e/live/cron-preflight-inference-local.test.ts`	Replaces manual `target.json` and `target-result.json` writes with `artifacts.target.declare()` and `artifacts.target.complete()` across the first live test group.
Live test migration: dashboard-remote-bind through hermes-slack `test/e2e/live/dashboard-remote-bind.test.ts`, `test/e2e/live/device-auth-health.test.ts`, `test/e2e/live/diagnostics.test.ts`, `test/e2e/live/docs-validation.test.ts`, `test/e2e/live/double-onboard.test.ts`, `test/e2e/live/full-e2e.test.ts`, `test/e2e/live/gateway-guard-recovery.test.ts`, `test/e2e/live/gateway-health-honest.test.ts`, `test/e2e/live/gpu-double-onboard.test.ts`, `test/e2e/live/gpu-e2e.test.ts`, `test/e2e/live/hermes-discord.test.ts`, `test/e2e/live/hermes-e2e.test.ts`, `test/e2e/live/hermes-gpu-startup.test.ts`, `test/e2e/live/hermes-inference-switch.test.ts`, `test/e2e/live/hermes-root-entrypoint-smoke.test.ts`, `test/e2e/live/hermes-sandbox-secret-boundary.test.ts`, `test/e2e/live/hermes-slack-e2e-helpers.ts`	Replaces manual `target.json` and `target-result.json` writes with `artifacts.target.declare()` and `artifacts.target.complete()` across the second live test group.
Live test migration: inference-routing through openshell-version-pin `test/e2e/live/inference-routing.test.ts`, `test/e2e/live/issue-2478-crash-loop-recovery.test.ts`, `test/e2e/live/issue-4434-tui-unreachable-inference.test.ts`, `test/e2e/live/issue-4462-scope-upgrade-approval.test.ts`, `test/e2e/live/jetson-nvmap-gpu.test.ts`, `test/e2e/live/kimi-inference-compat.test.ts`, `test/e2e/live/launchable-smoke.test.ts`, `test/e2e/live/messaging-compatible-endpoint.test.ts`, `test/e2e/live/model-router-provider-routed-inference.test.ts`, `test/e2e/live/network-policy.test.ts`, `test/e2e/live/ollama-auth-proxy.test.ts`, `test/e2e/live/onboard-negative-paths.test.ts`, `test/e2e/live/onboard-repair.test.ts`, `test/e2e/live/openclaw-discord-pairing.test.ts`, `test/e2e/live/openclaw-inference-switch.test.ts`, `test/e2e/live/openclaw-plugin-runtime-exdev.test.ts`, `test/e2e/live/openclaw-skill-cli.test.ts`, `test/e2e/live/openclaw-slack-pairing.test.ts`, `test/e2e/live/openclaw-tui-chat-correlation.test.ts`, `test/e2e/live/openshell-version-pin.test.ts`	Replaces manual `target.json` and `target-result.json` writes with `artifacts.target.declare()` and `artifacts.target.complete()` across inference-routing, issue, routing, and version-pin live tests.
Live test migration: registry-targets through upgrade-stale-sandbox `test/e2e/live/registry-targets.test.ts`, `test/e2e/live/runtime-overrides.test.ts`, `test/e2e/live/sandbox-operations.test.ts`, `test/e2e/live/sandbox-rlimits-connect.test.ts`, `test/e2e/live/sandbox-survival.test.ts`, `test/e2e/live/sessions-agents-cli.test.ts`, `test/e2e/live/shields-config.test.ts`, `test/e2e/live/skill-agent.test.ts`, `test/e2e/live/snapshot-commands.test.ts`, `test/e2e/live/spark-install.test.ts`, `test/e2e/live/state-backup-restore.test.ts`, `test/e2e/live/telegram-injection.test.ts`, `test/e2e/live/token-rotation.test.ts`, `test/e2e/live/ubuntu-repo-cli-smoke.test.ts`, `test/e2e/live/upgrade-stale-sandbox.test.ts`	Replaces manual `target.json` and `target-result.json` writes with `artifacts.target.declare()` and `artifacts.target.complete()` across registry and sandbox lifecycle live tests.

Estimated code review effort: 3 (Moderate) | ~25 minutes

Possibly related issues

Epic: Deduplicate and harden the Vitest E2E test infrastructure #6346: Directly related parent work for the typed target evidence API rollout.

Suggested labels: refactor, area: e2e

Suggested reviewers: ericksoa

🚥 Pre-merge checks | ✅ 5

✅ Passed checks (5 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title clearly states the main change: adding a typed target evidence API.
Linked Issues check	✅ Passed	The PR matches `#6353` by adding typed declare/complete APIs, centralizing runner, normalizing contract fields, adding tests, and migrating live writes without changing assertions.
Out of Scope Changes check	✅ Passed	The diff stays focused on target-evidence plumbing, live-spec migration, and tests, with no unrelated feature or workflow changes indicated.
Docstring Coverage	✅ Passed	No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.

✨ Finishing Touches

🧪 Generate unit tests (beta)

Create PR with unit tests

_{Comment @coderabbitai help to get the list of available commands.}

coderabbitai

Actionable comments posted: 1

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (1)

test/e2e/live/channels-stop-start-helpers.ts (1)
412-542: 🎯 Functional Correctness | 🟡 Minor | ⚡ Quick win

Emit target completion evidence here
declare() only writes target.json; this helper never calls artifacts.target.complete(...), so target-result.json is never produced for this target.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@test/e2e/live/channels-stop-start-helpers.ts` around lines 412 - 542, The
target is declared in runChannelsStopStartTarget but never marked complete, so
target-result.json is missing. Add a call to artifacts.target.complete(...) at
the end of runChannelsStopStartTarget after the final channel/start rebuild
assertions and before the function returns, using the same target metadata
already established by artifacts.target.declare.

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@test/e2e/fixtures/artifacts.ts`:
- Around line 24-60: Contract normalization in normalizeTargetEvidence is being
applied to both metadata and result payloads, which can accidentally rewrite or
reject unrelated extension fields on TargetResult. Update
normalizeTargetEvidence so the contract/contacts singular-vs-plural validation,
normalization, and delete logic only runs for kind === "metadata", while result
payloads keep their extension fields untouched except for the existing
status/default runner handling.

---

Outside diff comments:
In `@test/e2e/live/channels-stop-start-helpers.ts`:
- Around line 412-542: The target is declared in runChannelsStopStartTarget but
never marked complete, so target-result.json is missing. Add a call to
artifacts.target.complete(...) at the end of runChannelsStopStartTarget after
the final channel/start rebuild assertions and before the function returns,
using the same target metadata already established by artifacts.target.declare.

🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

Push a commit to this branch (recommended)
Create a new PR with the fixes

ℹ️ Review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: 5631f805-e73d-4b7e-8a85-6833e691c0ad

📥 Commits

Reviewing files that changed from the base of the PR and between ed6338e and 2fa8843.

📒 Files selected for processing (66)

test/e2e/fixtures/artifacts.ts
test/e2e/live/agent-turn-latency.test.ts
test/e2e/live/bedrock-runtime-compatible-anthropic.test.ts
test/e2e/live/brave-search.test.ts
test/e2e/live/channels-add-remove.test.ts
test/e2e/live/channels-stop-start-helpers.ts
test/e2e/live/cloud-inference.test.ts
test/e2e/live/cloud-onboard.test.ts
test/e2e/live/common-egress-agent.test.ts
test/e2e/live/concurrent-gateway-ports.test.ts
test/e2e/live/credential-migration.test.ts
test/e2e/live/credential-sanitization.test.ts
test/e2e/live/cron-preflight-inference-local.test.ts
test/e2e/live/dashboard-remote-bind.test.ts
test/e2e/live/device-auth-health.test.ts
test/e2e/live/diagnostics.test.ts
test/e2e/live/docs-validation.test.ts
test/e2e/live/double-onboard.test.ts
test/e2e/live/full-e2e.test.ts
test/e2e/live/gateway-guard-recovery.test.ts
test/e2e/live/gateway-health-honest.test.ts
test/e2e/live/gpu-double-onboard.test.ts
test/e2e/live/gpu-e2e.test.ts
test/e2e/live/hermes-discord.test.ts
test/e2e/live/hermes-e2e.test.ts
test/e2e/live/hermes-gpu-startup.test.ts
test/e2e/live/hermes-inference-switch.test.ts
test/e2e/live/hermes-root-entrypoint-smoke.test.ts
test/e2e/live/hermes-sandbox-secret-boundary.test.ts
test/e2e/live/hermes-slack-e2e-helpers.ts
test/e2e/live/inference-routing.test.ts
test/e2e/live/issue-2478-crash-loop-recovery.test.ts
test/e2e/live/issue-4434-tui-unreachable-inference.test.ts
test/e2e/live/issue-4462-scope-upgrade-approval.test.ts
test/e2e/live/jetson-nvmap-gpu.test.ts
test/e2e/live/kimi-inference-compat.test.ts
test/e2e/live/launchable-smoke.test.ts
test/e2e/live/messaging-compatible-endpoint.test.ts
test/e2e/live/model-router-provider-routed-inference.test.ts
test/e2e/live/network-policy.test.ts
test/e2e/live/ollama-auth-proxy.test.ts
test/e2e/live/onboard-negative-paths.test.ts
test/e2e/live/onboard-repair.test.ts
test/e2e/live/openclaw-discord-pairing.test.ts
test/e2e/live/openclaw-inference-switch.test.ts
test/e2e/live/openclaw-plugin-runtime-exdev.test.ts
test/e2e/live/openclaw-skill-cli.test.ts
test/e2e/live/openclaw-slack-pairing.test.ts
test/e2e/live/openclaw-tui-chat-correlation.test.ts
test/e2e/live/openshell-version-pin.test.ts
test/e2e/live/registry-targets.test.ts
test/e2e/live/runtime-overrides.test.ts
test/e2e/live/sandbox-operations.test.ts
test/e2e/live/sandbox-rlimits-connect.test.ts
test/e2e/live/sandbox-survival.test.ts
test/e2e/live/sessions-agents-cli.test.ts
test/e2e/live/shields-config.test.ts
test/e2e/live/skill-agent.test.ts
test/e2e/live/snapshot-commands.test.ts
test/e2e/live/spark-install.test.ts
test/e2e/live/state-backup-restore.test.ts
test/e2e/live/telegram-injection.test.ts
test/e2e/live/token-rotation.test.ts
test/e2e/live/ubuntu-repo-cli-smoke.test.ts
test/e2e/live/upgrade-stale-sandbox.test.ts
test/e2e/support/e2e-target-evidence.test.ts

copy-pr-bot · 2026-07-07T00:51:02Z

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

coderabbitai

🧹 Nitpick comments (1)

test/e2e/support/e2e-target-evidence.test.ts (1)
75-83: 📐 Maintainability & Code Quality | 🔵 Trivial | 💤 Low value

Migration-completeness check is inherently source-text based.

This regex scan for .writeJson("target...json") proves the superseded path is unreachable in test/e2e/live, which satisfies the migration-test requirement, though it relies on source-text matching rather than observable behavior. Since it directly implements the PR's stated acceptance criterion (a repo scan rejecting direct target evidence writes), this is acceptable as-is; just flagging the trade-off for awareness — a brittle regex could miss writes via aliased/renamed methods or computed filenames.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@test/e2e/support/e2e-target-evidence.test.ts` around lines 75 - 83, The
migration-completeness check in e2e-target-evidence.test.ts is brittle because
it relies on a regex over source text to detect direct target evidence writes.
Update the test around liveTypescriptFiles() to use a more robust source-level
check for writeJson calls on target/target-result filenames, or otherwise
explicitly document and constrain the accepted aliasing/computed-name cases so
the invariant remains enforceable even if the method is renamed or wrapped.
Source: Path instructions

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Nitpick comments:
In `@test/e2e/support/e2e-target-evidence.test.ts`:
- Around line 75-83: The migration-completeness check in
e2e-target-evidence.test.ts is brittle because it relies on a regex over source
text to detect direct target evidence writes. Update the test around
liveTypescriptFiles() to use a more robust source-level check for writeJson
calls on target/target-result filenames, or otherwise explicitly document and
constrain the accepted aliasing/computed-name cases so the invariant remains
enforceable even if the method is renamed or wrapped.

ℹ️ Review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: 689e691f-f86b-4352-8b87-95a2dc7dd2dc

📥 Commits

Reviewing files that changed from the base of the PR and between 2fa8843 and b655d1f.

📒 Files selected for processing (66)

test/e2e/fixtures/artifacts.ts
test/e2e/live/agent-turn-latency.test.ts
test/e2e/live/bedrock-runtime-compatible-anthropic.test.ts
test/e2e/live/brave-search.test.ts
test/e2e/live/channels-add-remove.test.ts
test/e2e/live/channels-stop-start-helpers.ts
test/e2e/live/cloud-inference.test.ts
test/e2e/live/cloud-onboard.test.ts
test/e2e/live/common-egress-agent.test.ts
test/e2e/live/concurrent-gateway-ports.test.ts
test/e2e/live/credential-migration.test.ts
test/e2e/live/credential-sanitization.test.ts
test/e2e/live/cron-preflight-inference-local.test.ts
test/e2e/live/dashboard-remote-bind.test.ts
test/e2e/live/device-auth-health.test.ts
test/e2e/live/diagnostics.test.ts
test/e2e/live/docs-validation.test.ts
test/e2e/live/double-onboard.test.ts
test/e2e/live/full-e2e.test.ts
test/e2e/live/gateway-guard-recovery.test.ts
test/e2e/live/gateway-health-honest.test.ts
test/e2e/live/gpu-double-onboard.test.ts
test/e2e/live/gpu-e2e.test.ts
test/e2e/live/hermes-discord.test.ts
test/e2e/live/hermes-e2e.test.ts
test/e2e/live/hermes-gpu-startup.test.ts
test/e2e/live/hermes-inference-switch.test.ts
test/e2e/live/hermes-root-entrypoint-smoke.test.ts
test/e2e/live/hermes-sandbox-secret-boundary.test.ts
test/e2e/live/hermes-slack-e2e-helpers.ts
test/e2e/live/inference-routing.test.ts
test/e2e/live/issue-2478-crash-loop-recovery.test.ts
test/e2e/live/issue-4434-tui-unreachable-inference.test.ts
test/e2e/live/issue-4462-scope-upgrade-approval.test.ts
test/e2e/live/jetson-nvmap-gpu.test.ts
test/e2e/live/kimi-inference-compat.test.ts
test/e2e/live/launchable-smoke.test.ts
test/e2e/live/messaging-compatible-endpoint.test.ts
test/e2e/live/model-router-provider-routed-inference.test.ts
test/e2e/live/network-policy.test.ts
test/e2e/live/ollama-auth-proxy.test.ts
test/e2e/live/onboard-negative-paths.test.ts
test/e2e/live/onboard-repair.test.ts
test/e2e/live/openclaw-discord-pairing.test.ts
test/e2e/live/openclaw-inference-switch.test.ts
test/e2e/live/openclaw-plugin-runtime-exdev.test.ts
test/e2e/live/openclaw-skill-cli.test.ts
test/e2e/live/openclaw-slack-pairing.test.ts
test/e2e/live/openclaw-tui-chat-correlation.test.ts
test/e2e/live/openshell-version-pin.test.ts
test/e2e/live/registry-targets.test.ts
test/e2e/live/runtime-overrides.test.ts
test/e2e/live/sandbox-operations.test.ts
test/e2e/live/sandbox-rlimits-connect.test.ts
test/e2e/live/sandbox-survival.test.ts
test/e2e/live/sessions-agents-cli.test.ts
test/e2e/live/shields-config.test.ts
test/e2e/live/skill-agent.test.ts
test/e2e/live/snapshot-commands.test.ts
test/e2e/live/spark-install.test.ts
test/e2e/live/state-backup-restore.test.ts
test/e2e/live/telegram-injection.test.ts
test/e2e/live/token-rotation.test.ts
test/e2e/live/ubuntu-repo-cli-smoke.test.ts
test/e2e/live/upgrade-stale-sandbox.test.ts
test/e2e/support/e2e-target-evidence.test.ts

✅ Files skipped from review due to trivial changes (3)

test/e2e/live/token-rotation.test.ts
test/e2e/live/hermes-inference-switch.test.ts
test/e2e/live/credential-migration.test.ts

🚧 Files skipped from review as they are similar to previous changes (62)

test/e2e/live/docs-validation.test.ts
test/e2e/live/ubuntu-repo-cli-smoke.test.ts
test/e2e/live/telegram-injection.test.ts
test/e2e/live/kimi-inference-compat.test.ts
test/e2e/live/spark-install.test.ts
test/e2e/live/gpu-e2e.test.ts
test/e2e/live/openshell-version-pin.test.ts
test/e2e/live/issue-2478-crash-loop-recovery.test.ts
test/e2e/live/hermes-root-entrypoint-smoke.test.ts
test/e2e/live/upgrade-stale-sandbox.test.ts
test/e2e/live/snapshot-commands.test.ts
test/e2e/live/gateway-guard-recovery.test.ts
test/e2e/live/cron-preflight-inference-local.test.ts
test/e2e/live/openclaw-slack-pairing.test.ts
test/e2e/live/concurrent-gateway-ports.test.ts
test/e2e/live/channels-add-remove.test.ts
test/e2e/live/sessions-agents-cli.test.ts
test/e2e/live/dashboard-remote-bind.test.ts
test/e2e/live/ollama-auth-proxy.test.ts
test/e2e/live/openclaw-skill-cli.test.ts
test/e2e/live/state-backup-restore.test.ts
test/e2e/live/hermes-gpu-startup.test.ts
test/e2e/live/jetson-nvmap-gpu.test.ts
test/e2e/live/onboard-negative-paths.test.ts
test/e2e/live/runtime-overrides.test.ts
test/e2e/live/hermes-e2e.test.ts
test/e2e/live/channels-stop-start-helpers.ts
test/e2e/live/device-auth-health.test.ts
test/e2e/live/brave-search.test.ts
test/e2e/live/issue-4462-scope-upgrade-approval.test.ts
test/e2e/live/double-onboard.test.ts
test/e2e/live/cloud-inference.test.ts
test/e2e/live/openclaw-inference-switch.test.ts
test/e2e/live/hermes-sandbox-secret-boundary.test.ts
test/e2e/live/launchable-smoke.test.ts
test/e2e/live/common-egress-agent.test.ts
test/e2e/live/diagnostics.test.ts
test/e2e/live/bedrock-runtime-compatible-anthropic.test.ts
test/e2e/fixtures/artifacts.ts
test/e2e/live/onboard-repair.test.ts
test/e2e/live/credential-sanitization.test.ts
test/e2e/live/full-e2e.test.ts
test/e2e/live/hermes-slack-e2e-helpers.ts
test/e2e/live/openclaw-discord-pairing.test.ts
test/e2e/live/openclaw-tui-chat-correlation.test.ts
test/e2e/live/sandbox-rlimits-connect.test.ts
test/e2e/live/agent-turn-latency.test.ts
test/e2e/live/registry-targets.test.ts
test/e2e/live/skill-agent.test.ts
test/e2e/live/gpu-double-onboard.test.ts
test/e2e/live/gateway-health-honest.test.ts
test/e2e/live/hermes-discord.test.ts
test/e2e/live/cloud-onboard.test.ts
test/e2e/live/sandbox-survival.test.ts
test/e2e/live/openclaw-plugin-runtime-exdev.test.ts
test/e2e/live/model-router-provider-routed-inference.test.ts
test/e2e/live/sandbox-operations.test.ts
test/e2e/live/network-policy.test.ts
test/e2e/live/issue-4434-tui-unreachable-inference.test.ts
test/e2e/live/inference-routing.test.ts
test/e2e/live/messaging-compatible-endpoint.test.ts
test/e2e/live/shields-config.test.ts

coderabbitai

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@test/e2e/support/e2e-target-evidence.test.ts`:
- Around line 77-85: The migration guard in liveTypescriptFiles() is too brittle
because the .writeJson("target.json") regex only matches same-line
single/double-quoted calls, so multi-line formatting or template literals can
bypass it. Update the check in e2e-target-evidence.test.ts to use a more robust
detection strategy in the it("keeps live target evidence behind the typed API")
test, so it still flags writeJson calls to target.json or target-result.json
regardless of line breaks or quote style.

🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

Push a commit to this branch (recommended)
Create a new PR with the fixes

ℹ️ Review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: 29baa5a4-35e3-4679-8af6-1be5128491d1

📥 Commits

Reviewing files that changed from the base of the PR and between b655d1f and eed6422.

📒 Files selected for processing (66)

test/e2e/fixtures/artifacts.ts
test/e2e/live/agent-turn-latency.test.ts
test/e2e/live/bedrock-runtime-compatible-anthropic.test.ts
test/e2e/live/brave-search.test.ts
test/e2e/live/channels-add-remove.test.ts
test/e2e/live/channels-stop-start-helpers.ts
test/e2e/live/cloud-inference.test.ts
test/e2e/live/cloud-onboard.test.ts
test/e2e/live/common-egress-agent.test.ts
test/e2e/live/concurrent-gateway-ports.test.ts
test/e2e/live/credential-migration.test.ts
test/e2e/live/credential-sanitization.test.ts
test/e2e/live/cron-preflight-inference-local.test.ts
test/e2e/live/dashboard-remote-bind.test.ts
test/e2e/live/device-auth-health.test.ts
test/e2e/live/diagnostics.test.ts
test/e2e/live/docs-validation.test.ts
test/e2e/live/double-onboard.test.ts
test/e2e/live/full-e2e.test.ts
test/e2e/live/gateway-guard-recovery.test.ts
test/e2e/live/gateway-health-honest.test.ts
test/e2e/live/gpu-double-onboard.test.ts
test/e2e/live/gpu-e2e.test.ts
test/e2e/live/hermes-discord.test.ts
test/e2e/live/hermes-e2e.test.ts
test/e2e/live/hermes-gpu-startup.test.ts
test/e2e/live/hermes-inference-switch.test.ts
test/e2e/live/hermes-root-entrypoint-smoke.test.ts
test/e2e/live/hermes-sandbox-secret-boundary.test.ts
test/e2e/live/hermes-slack-e2e-helpers.ts
test/e2e/live/inference-routing.test.ts
test/e2e/live/issue-2478-crash-loop-recovery.test.ts
test/e2e/live/issue-4434-tui-unreachable-inference.test.ts
test/e2e/live/issue-4462-scope-upgrade-approval.test.ts
test/e2e/live/jetson-nvmap-gpu.test.ts
test/e2e/live/kimi-inference-compat.test.ts
test/e2e/live/launchable-smoke.test.ts
test/e2e/live/messaging-compatible-endpoint.test.ts
test/e2e/live/model-router-provider-routed-inference.test.ts
test/e2e/live/network-policy.test.ts
test/e2e/live/ollama-auth-proxy.test.ts
test/e2e/live/onboard-negative-paths.test.ts
test/e2e/live/onboard-repair.test.ts
test/e2e/live/openclaw-discord-pairing.test.ts
test/e2e/live/openclaw-inference-switch.test.ts
test/e2e/live/openclaw-plugin-runtime-exdev.test.ts
test/e2e/live/openclaw-skill-cli.test.ts
test/e2e/live/openclaw-slack-pairing.test.ts
test/e2e/live/openclaw-tui-chat-correlation.test.ts
test/e2e/live/openshell-version-pin.test.ts
test/e2e/live/registry-targets.test.ts
test/e2e/live/runtime-overrides.test.ts
test/e2e/live/sandbox-operations.test.ts
test/e2e/live/sandbox-rlimits-connect.test.ts
test/e2e/live/sandbox-survival.test.ts
test/e2e/live/sessions-agents-cli.test.ts
test/e2e/live/shields-config.test.ts
test/e2e/live/skill-agent.test.ts
test/e2e/live/snapshot-commands.test.ts
test/e2e/live/spark-install.test.ts
test/e2e/live/state-backup-restore.test.ts
test/e2e/live/telegram-injection.test.ts
test/e2e/live/token-rotation.test.ts
test/e2e/live/ubuntu-repo-cli-smoke.test.ts
test/e2e/live/upgrade-stale-sandbox.test.ts
test/e2e/support/e2e-target-evidence.test.ts

✅ Files skipped from review due to trivial changes (2)

test/e2e/live/openclaw-slack-pairing.test.ts
test/e2e/live/hermes-root-entrypoint-smoke.test.ts

🚧 Files skipped from review as they are similar to previous changes (63)

test/e2e/live/kimi-inference-compat.test.ts
test/e2e/live/telegram-injection.test.ts
test/e2e/live/openclaw-discord-pairing.test.ts
test/e2e/live/channels-add-remove.test.ts
test/e2e/live/spark-install.test.ts
test/e2e/live/sandbox-rlimits-connect.test.ts
test/e2e/live/upgrade-stale-sandbox.test.ts
test/e2e/live/jetson-nvmap-gpu.test.ts
test/e2e/live/channels-stop-start-helpers.ts
test/e2e/live/registry-targets.test.ts
test/e2e/live/device-auth-health.test.ts
test/e2e/live/gpu-e2e.test.ts
test/e2e/live/hermes-inference-switch.test.ts
test/e2e/live/agent-turn-latency.test.ts
test/e2e/live/issue-2478-crash-loop-recovery.test.ts
test/e2e/live/credential-sanitization.test.ts
test/e2e/live/brave-search.test.ts
test/e2e/live/shields-config.test.ts
test/e2e/live/openshell-version-pin.test.ts
test/e2e/live/hermes-gpu-startup.test.ts
test/e2e/live/launchable-smoke.test.ts
test/e2e/live/docs-validation.test.ts
test/e2e/live/token-rotation.test.ts
test/e2e/live/gateway-health-honest.test.ts
test/e2e/live/full-e2e.test.ts
test/e2e/live/openclaw-plugin-runtime-exdev.test.ts
test/e2e/live/onboard-negative-paths.test.ts
test/e2e/live/credential-migration.test.ts
test/e2e/live/gateway-guard-recovery.test.ts
test/e2e/live/hermes-discord.test.ts
test/e2e/live/sandbox-survival.test.ts
test/e2e/live/dashboard-remote-bind.test.ts
test/e2e/live/diagnostics.test.ts
test/e2e/live/network-policy.test.ts
test/e2e/live/sessions-agents-cli.test.ts
test/e2e/live/double-onboard.test.ts
test/e2e/live/ollama-auth-proxy.test.ts
test/e2e/live/onboard-repair.test.ts
test/e2e/live/common-egress-agent.test.ts
test/e2e/live/openclaw-tui-chat-correlation.test.ts
test/e2e/live/runtime-overrides.test.ts
test/e2e/live/ubuntu-repo-cli-smoke.test.ts
test/e2e/live/issue-4434-tui-unreachable-inference.test.ts
test/e2e/live/hermes-slack-e2e-helpers.ts
test/e2e/live/inference-routing.test.ts
test/e2e/live/issue-4462-scope-upgrade-approval.test.ts
test/e2e/live/hermes-sandbox-secret-boundary.test.ts
test/e2e/live/skill-agent.test.ts
test/e2e/fixtures/artifacts.ts
test/e2e/live/cloud-onboard.test.ts
test/e2e/live/hermes-e2e.test.ts
test/e2e/live/openclaw-skill-cli.test.ts
test/e2e/live/snapshot-commands.test.ts
test/e2e/live/state-backup-restore.test.ts
test/e2e/live/cron-preflight-inference-local.test.ts
test/e2e/live/messaging-compatible-endpoint.test.ts
test/e2e/live/cloud-inference.test.ts
test/e2e/live/concurrent-gateway-ports.test.ts
test/e2e/live/bedrock-runtime-compatible-anthropic.test.ts
test/e2e/live/model-router-provider-routed-inference.test.ts
test/e2e/live/gpu-double-onboard.test.ts
test/e2e/live/sandbox-operations.test.ts
test/e2e/live/openclaw-inference-switch.test.ts

coderabbitai

🧹 Nitpick comments (1)

test/e2e/support/e2e-target-evidence.test.ts (1)
78-85: 🎯 Functional Correctness | 🔵 Trivial | ⚡ Quick win

Migration guard still has bypass paths via composed filenames or non-writeJson writers.

The regex requires the target.json/target-result.json literal to appear directly inside .writeJson(...) (allowing only leading whitespace). A composed path such as .writeJson(path.join(dir, "target.json"), data), or any raw fs.writeFileSync/fs.promises.writeFile call targeting the same filenames, would silently evade this check — undermining the guarantee that the manual write path is unreachable.

As per path instructions, "Migration tests must prove the superseded path is unreachable or removed, not merely prove that the new path also works," so this scan should catch writes to these filenames regardless of how the path argument is constructed or which write API is used.
🔧 Broaden detection to catch composed paths and alternate write APIs
-  const violations = liveTypescriptFiles()
-    .filter((file) =>
-      /\.writeJson\(\s*[`'"]target(?:-result)?\.json[`'"]/.test(fs.readFileSync(file, "utf8")),
-    )
-    .map((file) => path.basename(file));
+  const violations = liveTypescriptFiles()
+    .filter((file) => {
+      const source = fs.readFileSync(file, "utf8");
+      return (
+        /\.writeJson\(/.test(source) &&
+        /["'`]target(?:-result)?\.json["'`]/.test(source)
+      ) || /\bwriteFileSync?\(.*target(?:-result)?\.json/.test(source);
+    })
+    .map((file) => path.basename(file));
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@test/e2e/support/e2e-target-evidence.test.ts` around lines 78 - 85, The
migration guard in e2e-target-evidence.test.ts only matches literal target
filenames passed directly to .writeJson, so composed paths and other write APIs
can bypass it. Broaden the scan in the liveTypescriptFiles() check to detect any
reference to target.json or target-result.json regardless of how the path is
built, and include writes through writeJson, writeFileSync, and writeFile so the
superseded path cannot slip through. Use the existing violations assertion to
flag any file that targets those names through any writer.
Source: Path instructions

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Nitpick comments:
In `@test/e2e/support/e2e-target-evidence.test.ts`:
- Around line 78-85: The migration guard in e2e-target-evidence.test.ts only
matches literal target filenames passed directly to .writeJson, so composed
paths and other write APIs can bypass it. Broaden the scan in the
liveTypescriptFiles() check to detect any reference to target.json or
target-result.json regardless of how the path is built, and include writes
through writeJson, writeFileSync, and writeFile so the superseded path cannot
slip through. Use the existing violations assertion to flag any file that
targets those names through any writer.

ℹ️ Review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: 8f571191-90e8-4d6b-9824-d886e9c6e77c

📥 Commits

Reviewing files that changed from the base of the PR and between 1ac7254 and 570942b.

📒 Files selected for processing (66)

test/e2e/fixtures/artifacts.ts
test/e2e/live/agent-turn-latency.test.ts
test/e2e/live/bedrock-runtime-compatible-anthropic.test.ts
test/e2e/live/brave-search.test.ts
test/e2e/live/channels-add-remove.test.ts
test/e2e/live/channels-stop-start-helpers.ts
test/e2e/live/cloud-inference.test.ts
test/e2e/live/cloud-onboard.test.ts
test/e2e/live/common-egress-agent.test.ts
test/e2e/live/concurrent-gateway-ports.test.ts
test/e2e/live/credential-migration.test.ts
test/e2e/live/credential-sanitization.test.ts
test/e2e/live/cron-preflight-inference-local.test.ts
test/e2e/live/dashboard-remote-bind.test.ts
test/e2e/live/device-auth-health.test.ts
test/e2e/live/diagnostics.test.ts
test/e2e/live/docs-validation.test.ts
test/e2e/live/double-onboard.test.ts
test/e2e/live/full-e2e.test.ts
test/e2e/live/gateway-guard-recovery.test.ts
test/e2e/live/gateway-health-honest.test.ts
test/e2e/live/gpu-double-onboard.test.ts
test/e2e/live/gpu-e2e.test.ts
test/e2e/live/hermes-discord.test.ts
test/e2e/live/hermes-e2e.test.ts
test/e2e/live/hermes-gpu-startup.test.ts
test/e2e/live/hermes-inference-switch.test.ts
test/e2e/live/hermes-root-entrypoint-smoke.test.ts
test/e2e/live/hermes-sandbox-secret-boundary.test.ts
test/e2e/live/hermes-slack-e2e-helpers.ts
test/e2e/live/inference-routing.test.ts
test/e2e/live/issue-2478-crash-loop-recovery.test.ts
test/e2e/live/issue-4434-tui-unreachable-inference.test.ts
test/e2e/live/issue-4462-scope-upgrade-approval.test.ts
test/e2e/live/jetson-nvmap-gpu.test.ts
test/e2e/live/kimi-inference-compat.test.ts
test/e2e/live/launchable-smoke.test.ts
test/e2e/live/messaging-compatible-endpoint.test.ts
test/e2e/live/model-router-provider-routed-inference.test.ts
test/e2e/live/network-policy.test.ts
test/e2e/live/ollama-auth-proxy.test.ts
test/e2e/live/onboard-negative-paths.test.ts
test/e2e/live/onboard-repair.test.ts
test/e2e/live/openclaw-discord-pairing.test.ts
test/e2e/live/openclaw-inference-switch.test.ts
test/e2e/live/openclaw-plugin-runtime-exdev.test.ts
test/e2e/live/openclaw-skill-cli.test.ts
test/e2e/live/openclaw-slack-pairing.test.ts
test/e2e/live/openclaw-tui-chat-correlation.test.ts
test/e2e/live/openshell-version-pin.test.ts
test/e2e/live/registry-targets.test.ts
test/e2e/live/runtime-overrides.test.ts
test/e2e/live/sandbox-operations.test.ts
test/e2e/live/sandbox-rlimits-connect.test.ts
test/e2e/live/sandbox-survival.test.ts
test/e2e/live/sessions-agents-cli.test.ts
test/e2e/live/shields-config.test.ts
test/e2e/live/skill-agent.test.ts
test/e2e/live/snapshot-commands.test.ts
test/e2e/live/spark-install.test.ts
test/e2e/live/state-backup-restore.test.ts
test/e2e/live/telegram-injection.test.ts
test/e2e/live/token-rotation.test.ts
test/e2e/live/ubuntu-repo-cli-smoke.test.ts
test/e2e/live/upgrade-stale-sandbox.test.ts
test/e2e/support/e2e-target-evidence.test.ts

✅ Files skipped from review due to trivial changes (3)

test/e2e/live/upgrade-stale-sandbox.test.ts
test/e2e/live/openclaw-slack-pairing.test.ts
test/e2e/live/openshell-version-pin.test.ts

🚧 Files skipped from review as they are similar to previous changes (62)

test/e2e/live/brave-search.test.ts
test/e2e/live/ollama-auth-proxy.test.ts
test/e2e/live/openclaw-discord-pairing.test.ts
test/e2e/live/jetson-nvmap-gpu.test.ts
test/e2e/live/hermes-inference-switch.test.ts
test/e2e/live/model-router-provider-routed-inference.test.ts
test/e2e/live/issue-4434-tui-unreachable-inference.test.ts
test/e2e/live/cloud-onboard.test.ts
test/e2e/live/credential-sanitization.test.ts
test/e2e/live/full-e2e.test.ts
test/e2e/live/agent-turn-latency.test.ts
test/e2e/live/channels-stop-start-helpers.ts
test/e2e/live/launchable-smoke.test.ts
test/e2e/live/gpu-e2e.test.ts
test/e2e/live/sandbox-rlimits-connect.test.ts
test/e2e/live/hermes-gpu-startup.test.ts
test/e2e/live/state-backup-restore.test.ts
test/e2e/live/channels-add-remove.test.ts
test/e2e/live/sessions-agents-cli.test.ts
test/e2e/live/gpu-double-onboard.test.ts
test/e2e/live/shields-config.test.ts
test/e2e/live/network-policy.test.ts
test/e2e/live/double-onboard.test.ts
test/e2e/live/openclaw-tui-chat-correlation.test.ts
test/e2e/live/spark-install.test.ts
test/e2e/live/docs-validation.test.ts
test/e2e/live/hermes-e2e.test.ts
test/e2e/live/cron-preflight-inference-local.test.ts
test/e2e/live/hermes-sandbox-secret-boundary.test.ts
test/e2e/live/issue-2478-crash-loop-recovery.test.ts
test/e2e/live/hermes-root-entrypoint-smoke.test.ts
test/e2e/live/skill-agent.test.ts
test/e2e/live/token-rotation.test.ts
test/e2e/live/issue-4462-scope-upgrade-approval.test.ts
test/e2e/live/kimi-inference-compat.test.ts
test/e2e/live/snapshot-commands.test.ts
test/e2e/live/sandbox-survival.test.ts
test/e2e/live/gateway-health-honest.test.ts
test/e2e/live/onboard-repair.test.ts
test/e2e/live/registry-targets.test.ts
test/e2e/live/concurrent-gateway-ports.test.ts
test/e2e/live/device-auth-health.test.ts
test/e2e/live/cloud-inference.test.ts
test/e2e/live/dashboard-remote-bind.test.ts
test/e2e/live/openclaw-skill-cli.test.ts
test/e2e/live/credential-migration.test.ts
test/e2e/live/sandbox-operations.test.ts
test/e2e/live/hermes-discord.test.ts
test/e2e/live/runtime-overrides.test.ts
test/e2e/live/inference-routing.test.ts
test/e2e/live/ubuntu-repo-cli-smoke.test.ts
test/e2e/live/bedrock-runtime-compatible-anthropic.test.ts
test/e2e/live/hermes-slack-e2e-helpers.ts
test/e2e/live/openclaw-plugin-runtime-exdev.test.ts
test/e2e/fixtures/artifacts.ts
test/e2e/live/common-egress-agent.test.ts
test/e2e/live/onboard-negative-paths.test.ts
test/e2e/live/messaging-compatible-endpoint.test.ts
test/e2e/live/openclaw-inference-switch.test.ts
test/e2e/live/diagnostics.test.ts
test/e2e/live/gateway-guard-recovery.test.ts
test/e2e/live/telegram-injection.test.ts

github-actions · 2026-07-07T03:17:46Z

E2E Target Results — ✅ All default jobs passed

Run: 28838142470
Workflow ref: ci/pr-6359-e2e-9a3c595
Requested targets: (default — all supported)
Requested jobs: (default — all default-enabled free-standing jobs; explicit-only jobs openshell-gateway-auth-contract, mcp-bridge-dev, hermes-gpu-startup, sandbox-rlimits-connect, and jetson-nvmap-gpu are skipped unless selected)
Summary: 69 passed, 0 failed, 0 cancelled, 5 skipped

Job	Result
agent-turn-latency	✅ success
bedrock-runtime-compatible-anthropic	✅ success
brave-search	✅ success
channels-add-remove	✅ success
channels-stop-start	✅ success
cloud-inference	✅ success
cloud-onboard	✅ success
common-egress-agent	✅ success
concurrent-gateway-ports	✅ success
credential-migration	✅ success
credential-sanitization	✅ success
cron-preflight-inference-local	✅ success
device-auth-health	✅ success
diagnostics	✅ success
docs-validation	✅ success
double-onboard	✅ success
full-e2e	✅ success
gateway-drift-preflight	✅ success
gateway-guard-recovery	✅ success
gateway-health-honest	✅ success
generate-matrix	✅ success
gpu-double-onboard	✅ success
gpu-e2e	✅ success
hermes-dashboard	✅ success
hermes-discord	✅ success
hermes-e2e	✅ success
hermes-gpu-startup	⏭️ skipped
hermes-inference-switch	✅ success
hermes-slack	✅ success
inference-routing	✅ success
issue-2478-crash-loop-recovery	✅ success
issue-4434-tui-unreachable-inference	✅ success
issue-4462-scope-upgrade-approval	✅ success
jetson-nvmap-gpu	⏭️ skipped
kimi-inference-compat	✅ success
launchable-smoke	✅ success
live	✅ success
mcp-bridge	✅ success
mcp-bridge-dev	⏭️ skipped
messaging-compatible-endpoint	✅ success
messaging-providers	✅ success
model-router-provider-routed-inference	✅ success
network-policy	✅ success
ollama-auth-proxy	✅ success
onboard-negative-paths	✅ success
onboard-repair	✅ success
onboard-resume	✅ success
openclaw-discord-pairing	✅ success
openclaw-inference-switch	✅ success
openclaw-skill-cli	✅ success
openclaw-slack-pairing	✅ success
openclaw-tui-chat-correlation	✅ success
openshell-gateway-auth-contract	⏭️ skipped
openshell-gateway-upgrade	✅ success
openshell-version-pin	✅ success
overlayfs-autofix	✅ success
rebuild-hermes	✅ success
rebuild-hermes-stale-base	✅ success
rebuild-openclaw	✅ success
sandbox-operations	✅ success
sandbox-rebuild	✅ success
sandbox-rlimits-connect	⏭️ skipped
sandbox-survival	✅ success
security-posture	✅ success
sessions-agents-cli	✅ success
shields-config	✅ success
skill-agent	✅ success
snapshot-commands	✅ success
spark-install	✅ success
state-backup-restore	✅ success
telegram-injection	✅ success
token-rotation	✅ success
tunnel-lifecycle	✅ success
upgrade-stale-sandbox	✅ success

Explicit-only jobs skipped: openshell-gateway-auth-contract (default dispatch excludes the resource-heavy OpenShell auth-contract probe unless selected; validate with jobs=openshell-gateway-auth-contract or targets=openshell-gateway-auth-contract), mcp-bridge-dev (default dispatch excludes moving OpenShell dev artifacts unless explicitly selected; validate with jobs=mcp-bridge-dev or targets=mcp-bridge-dev), hermes-gpu-startup (default dispatch excludes this explicit-only job unless selected; validate with jobs=hermes-gpu-startup or targets=hermes-gpu-startup), sandbox-rlimits-connect (default dispatch excludes the destructive rlimit fork/connect probe unless selected; validate with jobs=sandbox-rlimits-connect or targets=sandbox-rlimits-connect), jetson-nvmap-gpu (default dispatch excludes Jetson until a stable Jetson runner is available; validate with jobs=jetson-nvmap-gpu or targets=jetson-nvmap-gpu).

Signed-off-by: Julie Yaunches <jyaunches@nvidia.com>

github-actions · 2026-07-07T03:54:58Z

E2E Target Results — ❌ Some jobs failed

Run: 28839471135
Workflow ref: ci/pr-6359-e2e-b36e465
Requested targets: (default — all supported)
Requested jobs: (default — all default-enabled free-standing jobs; explicit-only jobs openshell-gateway-auth-contract, mcp-bridge-dev, hermes-gpu-startup, sandbox-rlimits-connect, and jetson-nvmap-gpu are skipped unless selected)
Summary: 68 passed, 1 failed, 0 cancelled, 5 skipped

Job	Result
agent-turn-latency	✅ success
bedrock-runtime-compatible-anthropic	✅ success
brave-search	✅ success
channels-add-remove	✅ success
channels-stop-start	✅ success
cloud-inference	✅ success
cloud-onboard	✅ success
common-egress-agent	✅ success
concurrent-gateway-ports	✅ success
credential-migration	✅ success
credential-sanitization	✅ success
cron-preflight-inference-local	✅ success
device-auth-health	✅ success
diagnostics	✅ success
docs-validation	✅ success
double-onboard	✅ success
full-e2e	❌ failure
gateway-drift-preflight	✅ success
gateway-guard-recovery	✅ success
gateway-health-honest	✅ success
generate-matrix	✅ success
gpu-double-onboard	✅ success
gpu-e2e	✅ success
hermes-dashboard	✅ success
hermes-discord	✅ success
hermes-e2e	✅ success
hermes-gpu-startup	⏭️ skipped
hermes-inference-switch	✅ success
hermes-slack	✅ success
inference-routing	✅ success
issue-2478-crash-loop-recovery	✅ success
issue-4434-tui-unreachable-inference	✅ success
issue-4462-scope-upgrade-approval	✅ success
jetson-nvmap-gpu	⏭️ skipped
kimi-inference-compat	✅ success
launchable-smoke	✅ success
live	✅ success
mcp-bridge	✅ success
mcp-bridge-dev	⏭️ skipped
messaging-compatible-endpoint	✅ success
messaging-providers	✅ success
model-router-provider-routed-inference	✅ success
network-policy	✅ success
ollama-auth-proxy	✅ success
onboard-negative-paths	✅ success
onboard-repair	✅ success
onboard-resume	✅ success
openclaw-discord-pairing	✅ success
openclaw-inference-switch	✅ success
openclaw-skill-cli	✅ success
openclaw-slack-pairing	✅ success
openclaw-tui-chat-correlation	✅ success
openshell-gateway-auth-contract	⏭️ skipped
openshell-gateway-upgrade	✅ success
openshell-version-pin	✅ success
overlayfs-autofix	✅ success
rebuild-hermes	✅ success
rebuild-hermes-stale-base	✅ success
rebuild-openclaw	✅ success
sandbox-operations	✅ success
sandbox-rebuild	✅ success
sandbox-rlimits-connect	⏭️ skipped
sandbox-survival	✅ success
security-posture	✅ success
sessions-agents-cli	✅ success
shields-config	✅ success
skill-agent	✅ success
snapshot-commands	✅ success
spark-install	✅ success
state-backup-restore	✅ success
telegram-injection	✅ success
token-rotation	✅ success
tunnel-lifecycle	✅ success
upgrade-stale-sandbox	✅ success

Explicit-only jobs skipped: openshell-gateway-auth-contract (default dispatch excludes the resource-heavy OpenShell auth-contract probe unless selected; validate with jobs=openshell-gateway-auth-contract or targets=openshell-gateway-auth-contract), mcp-bridge-dev (default dispatch excludes moving OpenShell dev artifacts unless explicitly selected; validate with jobs=mcp-bridge-dev or targets=mcp-bridge-dev), hermes-gpu-startup (default dispatch excludes this explicit-only job unless selected; validate with jobs=hermes-gpu-startup or targets=hermes-gpu-startup), sandbox-rlimits-connect (default dispatch excludes the destructive rlimit fork/connect probe unless selected; validate with jobs=sandbox-rlimits-connect or targets=sandbox-rlimits-connect), jetson-nvmap-gpu (default dispatch excludes Jetson until a stable Jetson runner is available; validate with jobs=jetson-nvmap-gpu or targets=jetson-nvmap-gpu).

Failed jobs: full-e2e. Check run artifacts for logs.

Signed-off-by: Julie Yaunches <jyaunches@nvidia.com>

github-actions · 2026-07-07T04:37:35Z

E2E Target Results — ✅ All default jobs passed

Run: 28840988928
Workflow ref: ci/pr-6359-e2e-92f94b
Requested targets: (default — all supported)
Requested jobs: (default — all default-enabled free-standing jobs; explicit-only jobs openshell-gateway-auth-contract, mcp-bridge-dev, hermes-gpu-startup, sandbox-rlimits-connect, and jetson-nvmap-gpu are skipped unless selected)
Summary: 69 passed, 0 failed, 0 cancelled, 5 skipped

Job	Result
agent-turn-latency	✅ success
bedrock-runtime-compatible-anthropic	✅ success
brave-search	✅ success
channels-add-remove	✅ success
channels-stop-start	✅ success
cloud-inference	✅ success
cloud-onboard	✅ success
common-egress-agent	✅ success
concurrent-gateway-ports	✅ success
credential-migration	✅ success
credential-sanitization	✅ success
cron-preflight-inference-local	✅ success
device-auth-health	✅ success
diagnostics	✅ success
docs-validation	✅ success
double-onboard	✅ success
full-e2e	✅ success
gateway-drift-preflight	✅ success
gateway-guard-recovery	✅ success
gateway-health-honest	✅ success
generate-matrix	✅ success
gpu-double-onboard	✅ success
gpu-e2e	✅ success
hermes-dashboard	✅ success
hermes-discord	✅ success
hermes-e2e	✅ success
hermes-gpu-startup	⏭️ skipped
hermes-inference-switch	✅ success
hermes-slack	✅ success
inference-routing	✅ success
issue-2478-crash-loop-recovery	✅ success
issue-4434-tui-unreachable-inference	✅ success
issue-4462-scope-upgrade-approval	✅ success
jetson-nvmap-gpu	⏭️ skipped
kimi-inference-compat	✅ success
launchable-smoke	✅ success
live	✅ success
mcp-bridge	✅ success
mcp-bridge-dev	⏭️ skipped
messaging-compatible-endpoint	✅ success
messaging-providers	✅ success
model-router-provider-routed-inference	✅ success
network-policy	✅ success
ollama-auth-proxy	✅ success
onboard-negative-paths	✅ success
onboard-repair	✅ success
onboard-resume	✅ success
openclaw-discord-pairing	✅ success
openclaw-inference-switch	✅ success
openclaw-skill-cli	✅ success
openclaw-slack-pairing	✅ success
openclaw-tui-chat-correlation	✅ success
openshell-gateway-auth-contract	⏭️ skipped
openshell-gateway-upgrade	✅ success
openshell-version-pin	✅ success
overlayfs-autofix	✅ success
rebuild-hermes	✅ success
rebuild-hermes-stale-base	✅ success
rebuild-openclaw	✅ success
sandbox-operations	✅ success
sandbox-rebuild	✅ success
sandbox-rlimits-connect	⏭️ skipped
sandbox-survival	✅ success
security-posture	✅ success
sessions-agents-cli	✅ success
shields-config	✅ success
skill-agent	✅ success
snapshot-commands	✅ success
spark-install	✅ success
state-backup-restore	✅ success
telegram-injection	✅ success
token-rotation	✅ success
tunnel-lifecycle	✅ success
upgrade-stale-sandbox	✅ success

Explicit-only jobs skipped: openshell-gateway-auth-contract (default dispatch excludes the resource-heavy OpenShell auth-contract probe unless selected; validate with jobs=openshell-gateway-auth-contract or targets=openshell-gateway-auth-contract), mcp-bridge-dev (default dispatch excludes moving OpenShell dev artifacts unless explicitly selected; validate with jobs=mcp-bridge-dev or targets=mcp-bridge-dev), hermes-gpu-startup (default dispatch excludes this explicit-only job unless selected; validate with jobs=hermes-gpu-startup or targets=hermes-gpu-startup), sandbox-rlimits-connect (default dispatch excludes the destructive rlimit fork/connect probe unless selected; validate with jobs=sandbox-rlimits-connect or targets=sandbox-rlimits-connect), jetson-nvmap-gpu (default dispatch excludes Jetson until a stable Jetson runner is available; validate with jobs=jetson-nvmap-gpu or targets=jetson-nvmap-gpu).

cv

Hard contributor-compliance gate: commits 86a59c2, 02f8480, and b36e465 are not marked Verified by GitHub. Every PR commit must appear Verified, and maintainers cannot repair or sign contributor history. Please replace them with a clean verified history (or use a fresh branch/PR if the published branch cannot be safely rewritten), then rerun CI.

Maintainer waiver: commit-verification enforcement is waived for known team member jyaunches. Other CI, conflict, automated-review, security, and test gates still apply.

## Summary Add a typed target-evidence facade to the existing artifact sink and migrate live Vitest targets to it. Target metadata/results now retain the same artifact filenames and redaction boundary while receiving a consistent runner, result status, and contract field shape. ## Related Issue Closes NVIDIA#6353 Parent epic: NVIDIA#6346 ## Changes - Add `artifacts.target.declare(...)` and `artifacts.target.complete(...)` with typed extensible metadata/result inputs. - Emit `target.json` and `target-result.json` through the existing redacting `ArtifactSink`. - Supply `runner: "vitest"` centrally and default completed results without an explicit status to `status: "passed"`. - Normalize legacy singular `contract` values to the canonical `contracts` array and reject conflicting or malformed contract fields. - Validate non-empty target IDs and explicit result statuses. - Migrate all 117 live target evidence writes and remove repeated runner fields. - Add focused support tests for emitted files, normalization, validation, redaction, and adoption. ## Type of Change - [x] Code change (feature, bug fix, or refactor) - [ ] Code change with doc updates - [ ] Doc only (prose changes, no code sample modifications) - [ ] Doc only (includes code sample changes) ## Quality Gates - [x] Tests added or updated for changed behavior - [ ] Existing tests cover changed behavior — justification: - [ ] Tests not applicable — justification: - [ ] Docs updated for user-facing behavior changes - [x] Docs not applicable — justification: test-evidence refactor with no user-facing behavior - [x] Sensitive paths changed (security, policy, credentials, preflight, onboarding, inference, runner, sandbox, or messaging) - [ ] Sensitive-path review completed or maintainer-approved waiver recorded — reviewer/approval link/justification: - [ ] Non-success, skipped, or missing CI check accepted by maintainer — check name, approval link, and follow-up issue: ## Verification - [x] PR description includes the DCO sign-off declaration and every commit appears as `Verified` in GitHub - [x] Normal `pre-commit`, `commit-msg`, and `pre-push` hooks passed, or `npm run check:diff` passed when hooks were skipped or unavailable - [x] Targeted behavior tests pass for the current change set, or tests are marked not applicable above — 28 focused artifact, fixture, and redaction assertions passed - [ ] Applicable broad gate passed — `npm test` for broad runtime/test-harness changes; `npm run check` for repo-wide validation/coverage changes — command/result: - [x] Quality Gates section completed with required justifications or waivers - [x] No secrets, API keys, or credentials committed - [ ] `npm run docs` builds without warnings (doc changes only) - [ ] Doc pages follow the [style guide](https://github.com/NVIDIA/NemoClaw/blob/main/docs/CONTRIBUTING.md) (doc changes only) - [ ] New doc pages include SPDX header and frontmatter (new pages only) Additional local verification: - `npm run build:cli` - `npm run typecheck` - `npm run typecheck:cli` - `npm run lint` - `npx vitest run --project e2e-support test/e2e/support/e2e-target-evidence.test.ts test/e2e/support/e2e-fixture-context.test.ts test/e2e/support/e2e-redaction-entry.test.ts` --- Signed-off-by: Julie Yaunches <jyaunches@nvidia.com>  ## Summary by CodeRabbit * **New Features** * Added a typed E2E “target evidence” lifecycle with explicit **declare**/**complete** steps, producing consistent, normalized evidence outputs. * **Bug Fixes** * Improved validation and normalization (non-empty identifiers, non-empty status when provided, enforce single `contract` vs `contracts`, consistent contract array formatting, and automatic result defaults). * Added secret redaction in persisted evidence details. * **Tests** * Migrated many live E2E scenarios to use the typed evidence API. * Added test coverage for evidence normalization/redaction and a safeguard scan to prevent direct JSON target writes.  --------- Signed-off-by: Julie Yaunches <jyaunches@nvidia.com>

coderabbitai Bot reviewed Jul 6, 2026

View reviewed changes

Comment thread test/e2e/fixtures/artifacts.ts

jyaunches mentioned this pull request Jul 7, 2026

Epic: Deduplicate and harden the Vitest E2E test infrastructure #6346

Closed

15 tasks

jyaunches added the v0.0.76 label Jul 7, 2026

jyaunches force-pushed the refactor/e2e-target-evidence branch from 2fa8843 to b655d1f Compare July 7, 2026 00:51

coderabbitai Bot reviewed Jul 7, 2026

View reviewed changes

jyaunches force-pushed the refactor/e2e-target-evidence branch 2 times, most recently from 06e1e6a to eed6422 Compare July 7, 2026 01:25

coderabbitai Bot reviewed Jul 7, 2026

View reviewed changes

Comment thread test/e2e/support/e2e-target-evidence.test.ts

jyaunches force-pushed the refactor/e2e-target-evidence branch from 1ac7254 to 570942b Compare July 7, 2026 02:00

coderabbitai Bot reviewed Jul 7, 2026

View reviewed changes

jyaunches force-pushed the refactor/e2e-target-evidence branch from 570942b to 9a3c595 Compare July 7, 2026 02:17

jyaunches added 3 commits July 6, 2026 23:07

refactor(e2e): add typed target evidence

86a59c2

fix(e2e): preserve result evidence extension fields

02f8480

fix(e2e): harden target evidence migration guard

b36e465

jyaunches force-pushed the refactor/e2e-target-evidence branch from 9a3c595 to b36e465 Compare July 7, 2026 03:07

test(e2e): cover target evidence validation edges

b1d7147

Signed-off-by: Julie Yaunches <jyaunches@nvidia.com>

github-actions Bot mentioned this pull request Jul 7, 2026

fix(hermes): append resumed one-shot turns #6303

Merged

21 tasks

fix(e2e): canonicalize artifact sink roots

92f94b2

Signed-off-by: Julie Yaunches <jyaunches@nvidia.com>

cv previously requested changes Jul 7, 2026

View reviewed changes

cv approved these changes Jul 7, 2026

View reviewed changes

cv merged commit 9fe4cb3 into NVIDIA:main Jul 7, 2026
113 checks passed

cv mentioned this pull request Jul 7, 2026

refactor(e2e): centralize live-test gating and repository paths #6358

Merged

21 tasks

wscurran added area: ci CI workflows, checks, release automation, or GitHub Actions area: e2e End-to-end tests, nightly failures, or validation infrastructure labels Jul 7, 2026

wscurran added the refactor PR restructures code without intended behavior change label Jul 7, 2026

cv mentioned this pull request Jul 9, 2026

chore(security): track remaining CodeQL findings #6263

Open

coderabbitai Bot mentioned this pull request Aug 8, 2026

feat(qualification): enforce OpenShell v0.0.101 receipts #8635

Closed

23 tasks

Navigation Menu

Uh oh!

refactor(e2e): add a typed target evidence API - #6359

refactor(e2e): add a typed target evidence API#6359
cv merged 5 commits into
NVIDIA:mainfrom
jyaunches:refactor/e2e-target-evidence

jyaunches commented Jul 6, 2026 •

edited by coderabbitai Bot

Loading

Uh oh!

coderabbitai Bot commented Jul 6, 2026 •

edited

Loading

Reviews paused

Walkthrough

Changes

Uh oh!

coderabbitai Bot left a comment

Uh oh!

Uh oh!

copy-pr-bot Bot commented Jul 7, 2026

Uh oh!

coderabbitai Bot left a comment

Uh oh!

coderabbitai Bot left a comment

Uh oh!

Uh oh!

coderabbitai Bot left a comment

Uh oh!

github-actions Bot commented Jul 7, 2026

Uh oh!

github-actions Bot commented Jul 7, 2026

Uh oh!

github-actions Bot commented Jul 7, 2026

Uh oh!

cv left a comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Uh oh!

Conversation

jyaunches commented Jul 6, 2026 • edited by coderabbitai Bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Related Issue

Changes

Type of Change

Quality Gates

Verification

Summary by CodeRabbit

Uh oh!

coderabbitai Bot commented Jul 6, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Reviews paused

Walkthrough

Changes

Uh oh!

coderabbitai Bot left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

copy-pr-bot Bot commented Jul 7, 2026

Uh oh!

coderabbitai Bot left a comment

Choose a reason for hiding this comment

Uh oh!

coderabbitai Bot left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

coderabbitai Bot left a comment

Choose a reason for hiding this comment

Uh oh!

github-actions Bot commented Jul 7, 2026

E2E Target Results — ✅ All default jobs passed

Uh oh!

github-actions Bot commented Jul 7, 2026

E2E Target Results — ❌ Some jobs failed

Uh oh!

github-actions Bot commented Jul 7, 2026

E2E Target Results — ✅ All default jobs passed

Uh oh!

cv left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

jyaunches commented Jul 6, 2026 •

edited by coderabbitai Bot

Loading

coderabbitai Bot commented Jul 6, 2026 •

edited

Loading