Skip to content

fix(e2e): fix messaging-providers and sandbox-survival nightly E2E regressions - #1859

Merged
ericksoa merged 3 commits into
mainfrom
fix/e2e-messaging-tier
Apr 14, 2026
Merged

fix(e2e): fix messaging-providers and sandbox-survival nightly E2E regressions#1859
ericksoa merged 3 commits into
mainfrom
fix/e2e-messaging-tier

Conversation

@ericksoa

@ericksoa ericksoa commented Apr 14, 2026

Copy link
Copy Markdown
Contributor

Summary

  • messaging-providers-e2e: The tier-based policy selector (feat(policy): tier-based policy selector with combined access-mode UI #1753) defaults non-interactive onboard to balanced, which excludes messaging presets. Adds NEMOCLAW_POLICY_TIER: "open" to the workflow env.
  • sandbox-survival-e2e: SSH after gateway restart can fail because the sandbox SSH agent isn't immediately reachable when the gateway reports healthy (more noticeable with OpenClaw 2026.4.2). Adds a 30s retry loop for SSH connectivity. Also removes cleanup_ssh() from the failure path — deleting the SSH config was causing all subsequent phases to fail with "Can't open user config file", masking whether data actually persisted.

Test plan

  • Nightly E2E messaging-providers job passes
  • Nightly E2E sandbox-survival job passes (or produces real diagnostics if data is actually lost)

🤖 Generated with Claude Code

Summary by CodeRabbit

  • Chores

    • Nightly end-to-end test environment configuration updated to set the test policy tier.
  • Tests

    • End-to-end test flow now retries SSH connectivity multiple times and preserves connection data on failure, improving test stability and enabling richer diagnostic logs.

The tier-based policy selector (#1753) defaults non-interactive onboard
to the "balanced" tier, which excludes messaging presets (telegram,
discord, slack). The messaging-providers E2E test needs the "open" tier
to allow egress to api.telegram.org and discord.com.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@coderabbitai

coderabbitai Bot commented Apr 14, 2026

Copy link
Copy Markdown
Contributor

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro Plus

Run ID: 136758f3-713e-470a-bbbb-a829f8729730

📥 Commits

Reviewing files that changed from the base of the PR and between 3efe7cb and 10c0b63.

📒 Files selected for processing (1)
  • test/e2e/test-sandbox-survival.sh

📝 Walkthrough

Walkthrough

Added an environment variable NEMOCLAW_POLICY_TIER: "open" to the nightly messaging-providers-e2e job and replaced a single SSH connectivity check after gateway restart with a retry loop that attempts SSH up to 6 times, preserving SSH config and collecting handshake logs on failure.

Changes

Cohort / File(s) Summary
Workflow Configuration
.github/workflows/nightly-e2e.yaml
Added environment variable NEMOCLAW_POLICY_TIER: "open" to the messaging-providers-e2e job.
E2E Test Script
test/e2e/test-sandbox-survival.sh
Replaced single-attempt SSH check with a retry loop (up to 6 attempts, 5s sleep). On failure, preserves SSH config and gathers handshake-related logs instead of calling cleanup_ssh.

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~20 minutes

Poem

🐰 I hopped through scripts with care,
Adding retries into the air,
A policy set, a gentle nudge,
Logs kept safe, no sudden trudge,
Tests and workflows — a joyful pair 🥕

🚥 Pre-merge checks | ✅ 3
✅ Passed checks (3 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title directly addresses the main changes: fixing E2E regressions in messaging-providers (via NEMOCLAW_POLICY_TIER) and sandbox-survival (via SSH retry logic).
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch fix/e2e-messaging-tier

Comment @coderabbitai help to get the list of available commands and usage tips.

@ericksoa ericksoa self-assigned this Apr 14, 2026
…failure

Two fixes for the sandbox-survival E2E:

1. Add retry loop (up to 30s) for SSH connectivity after gateway restart.
   The sandbox SSH agent may not be ready immediately after the gateway
   reports healthy, especially with OpenClaw 2026.4.2 which does more
   startup work.

2. Remove cleanup_ssh() from the Phase 8 failure path. Deleting the SSH
   config file caused all subsequent phases (9, 10) to fail with "Can't
   open user config file" — masking whether data actually persisted.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@ericksoa ericksoa changed the title fix(ci): set NEMOCLAW_POLICY_TIER=open for messaging-providers E2E fix(e2e): fix messaging-providers and sandbox-survival nightly E2E regressions Apr 14, 2026
@ericksoa
ericksoa enabled auto-merge (squash) April 14, 2026 03:18
@ericksoa
ericksoa merged commit 1eff9b6 into main Apr 14, 2026
8 checks passed
ColinM-sys pushed a commit to ColinM-sys/NemoClaw that referenced this pull request Apr 14, 2026
…gressions (NVIDIA#1859)

## Summary
- **messaging-providers-e2e**: The tier-based policy selector (NVIDIA#1753)
defaults non-interactive onboard to `balanced`, which excludes messaging
presets. Adds `NEMOCLAW_POLICY_TIER: "open"` to the workflow env.
- **sandbox-survival-e2e**: SSH after gateway restart can fail because
the sandbox SSH agent isn't immediately reachable when the gateway
reports healthy (more noticeable with OpenClaw 2026.4.2). Adds a 30s
retry loop for SSH connectivity. Also removes `cleanup_ssh()` from the
failure path — deleting the SSH config was causing all subsequent phases
to fail with "Can't open user config file", masking whether data
actually persisted.

## Test plan
- [ ] Nightly E2E messaging-providers job passes
- [ ] Nightly E2E sandbox-survival job passes (or produces real
diagnostics if data is actually lost)

🤖 Generated with [Claude Code](https://claude.com/claude-code)

<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->

## Summary by CodeRabbit

* **Chores**
* Nightly end-to-end test environment configuration updated to set the
test policy tier.

* **Tests**
* End-to-end test flow now retries SSH connectivity multiple times and
preserves connection data on failure, improving test stability and
enabling richer diagnostic logs.

<!-- end of auto-generated comment: release notes by coderabbit.ai -->

---------

Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Signed-off-by: ColinM-sys <cmcdonough@50words.com>
@wscurran wscurran added the bug-fix PR fixes a bug or regression label Jun 8, 2026
@cv
cv deleted the fix/e2e-messaging-tier branch June 28, 2026 00:21
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug-fix PR fixes a bug or regression

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants