Skip to content

fix(e2e): stabilize launchable smoke agent turn - #4039

Merged
jyaunches merged 1 commit into
NVIDIA:mainfrom
jyaunches:fix/launchable-smoke-thinking
May 22, 2026
Merged

fix(e2e): stabilize launchable smoke agent turn#4039
jyaunches merged 1 commit into
NVIDIA:mainfrom
jyaunches:fix/launchable-smoke-thinking

Conversation

@jyaunches

@jyaunches jyaunches commented May 22, 2026

Copy link
Copy Markdown
Contributor

Summary

  • run the launchable smoke openclaw agent probe with --thinking off
  • preserve agent stderr/exit code and include stdout/stderr snippets in the failure message

Why

The launchable smoke run already proves direct NVIDIA Endpoints and sandbox inference.local routing before the agent turn. The final agent-mediated probe can fail with an empty parsed reply when OpenClaw/model thinking defaults delay or alter first-turn output, and the old failure discarded stderr so the root cause was hidden.

Test plan

  • bash -n test/e2e/test-launchable-smoke.sh

Related: issue #2599

Summary by CodeRabbit

  • Tests
    • Improved smoke test diagnostics: capture agent exit codes, stdout and stderr into temporary files and include truncated agent reply/stdout/stderr in failure messages for clearer debugging.
    • Ensure temporary files created during test setup are always cleaned up.

Review Change Stack

@coderabbitai

coderabbitai Bot commented May 22, 2026

Copy link
Copy Markdown
Contributor

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: 2a0befa2-77ef-445d-9585-c774fa4c7da1

📥 Commits

Reviewing files that changed from the base of the PR and between afe41ec and da1293d.

📒 Files selected for processing (1)
  • test/e2e/test-launchable-smoke.sh

📝 Walkthrough

Walkthrough

Phase 6c of the smoke test now captures the openclaw agent's exit code and stderr to a temp file (always cleaned up), broadens JSON payload parsing, and reports agent rc plus truncated reply/stdout/stderr when the expected "6×7=42" check fails.

Changes

Phase 6c Agent Invocation Debugging

Layer / File(s) Summary
Agent invocation with stderr capture and error reporting
test/e2e/test-launchable-smoke.sh
Variables for agent_stderr, agent_rc, and a temp stderr file are added. The SSH invocation redirects agent stderr into the temp file, records a non-zero agent_rc when SSH/command fails, always loads stderr content, ensures the temp files (including ssh_config) are removed after the ssh-config step, broadens JSON payload parsing, and expands the failure assertion to include the agent exit code plus truncated agent reply, stdout, and stderr.

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~18 minutes

  • NVIDIA/NemoClaw#4038: Related changes to the same test/e2e/test-launchable-smoke.sh agent reply parsing and validation logic.

Suggested labels: Integration: OpenClaw, fix, E2E, status: rfr

  • cv

🐰 I hopped through logs both near and far,
Stashed stderr in temp files like a star.
I count the rc and trim the reply,
So when tests fail, we see the why —
A rabbit's debug carrot, wink and sigh.

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title 'fix(e2e): stabilize launchable smoke agent turn' directly reflects the main objective of the PR: stabilizing the launchable smoke OpenClaw agent probe through better diagnostics and error handling.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Comment @coderabbitai help to get the list of available commands and usage tips.

@github-actions

Copy link
Copy Markdown
Contributor

Selective E2E Results — ❌ Some jobs failed

Run: 26263494667
Target ref: 52fc710bfac5b8eff4306654fe1c27b9dc0cf8c9
Workflow ref: main
Requested jobs: launchable-smoke-e2e
Summary: 0 passed, 1 failed, 0 skipped

Job Result
launchable-smoke-e2e ❌ failure

Failed jobs: launchable-smoke-e2e. Check run artifacts for logs.

@github-actions

Copy link
Copy Markdown
Contributor

Selective E2E Results — ❌ Some jobs failed

Run: 26263536345
Target ref: fix/launchable-smoke-thinking
Requested jobs: launchable-smoke-e2e
Summary: 0 passed, 1 failed, 0 skipped

Job Result
launchable-smoke-e2e ❌ failure

Failed jobs: launchable-smoke-e2e. Check run artifacts for logs.

@jyaunches
jyaunches force-pushed the fix/launchable-smoke-thinking branch from afe41ec to da1293d Compare May 22, 2026 01:59
@jyaunches
jyaunches merged commit 971c526 into NVIDIA:main May 22, 2026
71 checks passed
@github-actions

Copy link
Copy Markdown
Contributor

Selective E2E Results — ✅ All requested jobs passed

Run: 26264292591
Target ref: ci/pr-4039-launchable-smoke-thinking
Requested jobs: launchable-smoke-e2e
Summary: 1 passed, 0 failed, 0 skipped

Job Result
launchable-smoke-e2e ✅ success

@wscurran wscurran added the bug-fix PR fixes a bug or regression label Jun 8, 2026
@jyaunches
jyaunches deleted the fix/launchable-smoke-thinking branch June 22, 2026 13:02
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug-fix PR fixes a bug or regression

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants