Skip to content

fix(e2e): extend Docker Hub login retries - #8766

Merged
jyaunches merged 3 commits into
mainfrom
codex/fix-e2e-docker-auth-timeout
Aug 10, 2026
Merged

fix(e2e): extend Docker Hub login retries#8766
jyaunches merged 3 commits into
mainfrom
codex/fix-e2e-docker-auth-timeout

Conversation

@jyaunches

@jyaunches jyaunches commented Aug 10, 2026

Copy link
Copy Markdown
Contributor

Summary

The linked Slack E2E never reached the test because one runner exhausted the shared three-attempt Docker Hub authentication budget during a transient registry connectivity stall. This change gives the shared login path five bounded attempts while preserving credential isolation, immutable provenance, and fail-closed behavior.

Changes

  • Extend Docker Hub authentication from three to five bounded attempts in the shared E2E helper and the equivalent sandbox-image workflow path.
  • Pin the E2E workflow to the signed helper commit and update the enforced action, helper, and workflow provenance hashes.
  • Add regression coverage for recovery on attempts four and five, five-attempt exhaustion, exact sleep counts, and opaque-input changed-test routing.

Failure evidence: E2E run 31428738562, OpenClaw Slack pairing job 93595566444.

Type of Change

  • Code change (feature, bug fix, or refactor)
  • Code change with doc updates
  • Doc only (prose changes, no code sample modifications)
  • Doc only (includes code sample changes)

Quality Gates

  • Tests added or updated for changed behavior
  • Existing tests cover changed behavior — justification:
  • Tests not applicable — justification:
  • Docs updated for user-facing behavior changes
  • Docs not applicable — justification: This changes an internal CI authentication retry budget; it does not change a public API, CLI, configuration, onboarding flow, UI, or supported runtime behavior, and existing docs do not define the retry count.
  • Sensitive paths changed (security, policy, credentials, preflight, onboarding, inference, runner, sandbox, or messaging)
  • Sensitive-path review completed or maintainer-approved waiver recorded — reviewer/approval link/justification: Independent Codex security review passed all nine required categories with no findings; trusted-run gating, stdin-only token transfer, isolated Docker config, immutable pinning, bounded timeouts, cleanup, and fail-closed behavior remain intact.
  • Non-success, skipped, or missing CI check accepted by maintainer — check name, approval link, and follow-up issue:

Documentation Writer Review

  • Documentation writer subagent reviewed the completed changes
  • Result: no-docs-needed
  • Evidence: Internal CI Docker Hub authentication retry and regression-contract updates only; no public or supported user-facing behavior changed, and existing documentation does not define the retry budget.
  • Agent: Codex Desktop

Verification

  • PR description includes a Signed-off-by: line and every commit appears as Verified in GitHub
  • Normal pre-commit, commit-msg, and pre-push hooks passed, or npm run validate:pr passed after refreshing origin/main when hooks were skipped or unavailable
  • Targeted behavior tests pass for the current change set, or tests are marked not applicable above — 94/94 focused E2E-support boundary tests, 5/5 watch-trigger integration tests, 500/500 affected tests, and the final 12/12 Docker-auth rerun passed.
  • Applicable broad gate passed — npm test for broad runtime/test-harness changes; npm run check for repo-wide validation/coverage changes — command/result:
  • Quality Gates section completed with required justifications or waivers
  • No secrets, API keys, or credentials committed
  • npm run docs builds without warnings (doc changes only)
  • Doc pages follow the style guide (doc changes only)
  • New doc pages include SPDX header and frontmatter (new pages only)

Signed-off-by: Julie Yaunches jyaunches@nvidia.com

Summary by CodeRabbit

  • Bug Fixes

    • Improved Docker Hub authentication reliability by retrying failed logins up to five times with five-second delays.
    • Enhanced warning and error messages to clearly report retry settings.
  • Tests

    • Expanded end-to-end coverage for authentication retries and workflow validation.
    • Added monitoring so relevant workflow and authentication changes trigger the appropriate tests.

@jyaunches jyaunches added area: ci CI workflows, checks, release automation, or GitHub Actions area: e2e End-to-end tests, nightly failures, or validation infrastructure labels Aug 10, 2026
@jyaunches jyaunches self-assigned this Aug 10, 2026
@coderabbitai

coderabbitai Bot commented Aug 10, 2026

Copy link
Copy Markdown
Contributor

Review Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: d53ee2fc-e128-442b-906b-fb0b60c0dc22

📥 Commits

Reviewing files that changed from the base of the PR and between 3362694 and 9499c8c.

📒 Files selected for processing (1)
  • test/e2e/support/dockerhub-auth-workflow-boundary.test.ts

📝 Walkthrough

Walkthrough

Docker Hub authentication now retries five times with five-second delays. Workflows, provenance checks, boundary tests, and Vitest watch triggers use the updated contract. The CLI artifact workflow boundary hash is also updated.

Changes

Docker authentication retry

Layer / File(s) Summary
Retry behavior and workflow wiring
.github/scripts/docker-auth-setup.sh, .github/workflows/sandbox-images-and-e2e.yaml, .github/workflows/e2e.yaml
Authentication uses five attempts, five-second delays, and dynamic retry messages. The E2E workflow uses the updated action revision.
Boundary validation and provenance
test/e2e/support/dockerhub-auth-workflow-boundary.test.ts, tools/e2e/sandbox-images-workflow-boundary.mts, tools/e2e/workflow-boundary-policy.mts
Tests validate retry counts, delays, tokens, and failure messages. Boundary requirements and action provenance use the updated contract.
Watch-trigger coverage
test/helpers/vitest-watch-triggers.ts, test/vitest-watch-triggers.test.ts
Watch mappings and tests cover Docker authentication and sandbox-images workflow changes.

CLI workflow contract refresh

Layer / File(s) Summary
CLI artifact contract hash
tools/e2e/cli-artifact-workflow-boundary.mts
The expected CLI artifact workflow contract SHA-256 digest is updated.

Estimated code review effort: 3 (Moderate) | ~20 minutes

Sequence Diagram(s)

sequenceDiagram
  participant E2EWorkflow
  participant DockerAuthSetup
  participant DockerHub
  E2EWorkflow->>DockerAuthSetup: invoke authentication action
  DockerAuthSetup->>DockerHub: submit login token
  DockerHub-->>DockerAuthSetup: return authentication result
  DockerAuthSetup->>DockerAuthSetup: wait five seconds before retry
Loading
🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title clearly and concisely describes the main change: extending Docker Hub login retries in E2E workflows.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch codex/fix-e2e-docker-auth-timeout

Comment @coderabbitai help to get the list of available commands.

@github-code-quality

github-code-quality Bot commented Aug 10, 2026

Copy link
Copy Markdown
Contributor

Code Coverage Overview

Languages: TypeScript

TypeScript / code-coverage/plugin

The overall coverage in commit 9499c8c in the codex/fix-e2e-docker... branch remains at 96%, unchanged from commit 567a033 in the main branch.

TypeScript / code-coverage/cli

The overall coverage in commit 9499c8c in the codex/fix-e2e-docker... branch remains at 82%, unchanged from commit 567a033 in the main branch.

Show a code coverage summary of the most impacted files.
File main 567a033 codex/fix-e2e-docker... 9499c8c +/-
src/lib/messagi...annel-config.ts 99% 95% -4%
src/lib/sandbox...rce-identity.ts 86% 86% 0%
src/lib/tunnel/services.ts 80% 80% 0%
src/lib/adapter...shell/client.ts 88% 90% +2%
src/lib/core/bo...e-transcript.ts 93% 97% +4%

Updated August 10, 2026 22:41 UTC

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@test/e2e/support/dockerhub-auth-workflow-boundary.test.ts`:
- Around line 514-546: Add a separate runAuth recovery case with successAttempt:
5 in the test covering Docker Hub authentication retries. Assert status 0, five
Docker calls, five token writes, and four sleep entries, while preserving the
existing attempt-four recovery and exhaustion assertions.
🪄 Autofix

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: b9ee2bd5-60f4-414f-9358-ce7932deb874

📥 Commits

Reviewing files that changed from the base of the PR and between 567a033 and 3362694.

📒 Files selected for processing (9)
  • .github/scripts/docker-auth-setup.sh
  • .github/workflows/e2e.yaml
  • .github/workflows/sandbox-images-and-e2e.yaml
  • test/e2e/support/dockerhub-auth-workflow-boundary.test.ts
  • test/helpers/vitest-watch-triggers.ts
  • test/vitest-watch-triggers.test.ts
  • tools/e2e/cli-artifact-workflow-boundary.mts
  • tools/e2e/sandbox-images-workflow-boundary.mts
  • tools/e2e/workflow-boundary-policy.mts

Comment thread test/e2e/support/dockerhub-auth-workflow-boundary.test.ts
@github-actions

github-actions Bot commented Aug 10, 2026

Copy link
Copy Markdown
Contributor

PR Review Advisor — No blocking findings reported

Advisor assessment: No blocking advisor findings reported
Next action: No advisor follow-up needed.
Findings: 0 blockers · 0 warnings · 0 suggestions

Model lanes

  • GPT-5.6 Terra (primary): Completed · high confidence · 0 blockers · 0 warnings · 0 suggestions
  • Nemotron 3 Ultra (second opinion): Failed after a partial review · low confidence · 0 blockers · 0 warnings · 0 suggestions

Second-opinion terminology and E2E selections are advisory. Live E2E does not run automatically for pull requests.

E2E guidance

Advisory only. A maintainer can dispatch the default E2E suite against this exact revision.

Recommended E2E: cloud-inference, cloud-onboard, security-posture

Workflow run details

This automated review informs maintainers. Warnings and suggestions do not require a response. A maintainer decides whether to merge.

@jyaunches

Copy link
Copy Markdown
Contributor Author

Live E2E evidence: trusted main push run 31437795324 at 567a033 completed the OpenClaw Slack pairing job successfully: https://github.com/NVIDIA/NemoClaw/actions/runs/31437795324/job/93623045829. Docker Hub authentication, the live pairing test, artifact upload, and Docker-auth cleanup all passed. This trusted-main run still uses the pre-PR three-attempt helper, so it confirms the original auth failure was transient and that the previously skipped Slack test is green. The PR retry change itself is covered by the five-attempt boundary tests and can exercise authenticated live behavior only after merge, because manual PR E2E intentionally disables Docker Hub credentials.

@jyaunches
jyaunches merged commit b8232eb into main Aug 10, 2026
85 of 89 checks passed
@jyaunches
jyaunches deleted the codex/fix-e2e-docker-auth-timeout branch August 10, 2026 23:20
jyaunches added a commit that referenced this pull request Aug 11, 2026
<!-- markdownlint-disable MD041 -->
## Summary

This PR updates the August 10, 2026 v0.0.106 release entry with gateway
readiness fixes that merged after PR #8756.
PRs #8765, #8767, and #8768 remain outside this entry because they are
open and do not carry the `v0.0.106` release label.

## Changes

- Document acceptance of OpenShell v0.0.101 `Server:` endpoint output
and target-bound process tags when trusted listener evidence matches the
configured gateway.
- Document preservation of selected-gateway stale state so onboarding
can reconcile a registered gateway when a gateway-scoped OpenShell
status check cannot connect.
- Record evidence-backed exclusions for internal image, startup,
qualification, proxy-environment, CI, and test-harness changes in PRs
#8754, #8609, #8762, #8432, #8766, and #8581.
- Exclude PRs #8765, #8767, and #8768 because their changes are absent
from `main` and the PRs do not carry the `v0.0.106` release label.
The release entry must be updated after any of those PRs merges for
v0.0.106.

## Type of Change

- [ ] Code change (feature, bug fix, or refactor)
- [ ] Code change with doc updates
- [x] Doc only (prose changes, no code sample modifications)
- [ ] Doc only (includes code sample changes)

## Quality Gates

- [ ] Tests added or updated for changed behavior
- [x] Existing tests cover changed behavior — justification:
`test/changelog-docs.test.ts` validates dated changelog SPDX placement,
version headings, forbidden terms, and link form.
- [ ] Tests not applicable — justification:
- [x] Docs updated for user-facing behavior changes
- [ ] Docs not applicable — justification:
- [ ] Sensitive paths changed (security, policy, credentials, preflight,
onboarding, inference, runner, sandbox, or messaging)
- [ ] Sensitive-path review completed or maintainer-approved waiver
recorded — reviewer/approval link/justification:
- [ ] Non-success, skipped, or missing CI check accepted by maintainer —
check name, approval link, and follow-up issue:

## Documentation Writer Review

- [x] Documentation writer subagent reviewed the completed changes
- Result: `docs-updated`
- Evidence: `docs/changelog/2026-08-10.mdx`; an independent Codex
Desktop subagent reviewed the writing rules and documentation style,
terminology, structure, voice, code-sample presentation, links, source
and test accuracy, release meaning, product scope, and evidence-backed
exclusions at commit `190bf882c`.
- Agent: Codex Desktop
<!-- docs-review-head-sha: 190bf88 -->
<!-- docs-review-agents-blob-sha: c4923a3 -->

## DGX Station Hardware Evidence

- [ ] Tested on DGX Station
- Tested commit: Not applicable; `scripts/prepare-dgx-station-host.sh`
is unchanged.
- Station profile/scenario: Not applicable.
- Result: Not applicable.
- Supporting evidence: Not applicable.

## Verification

- [x] PR description includes a `Signed-off-by:` line and every commit
appears as `Verified` in GitHub
- [x] Normal `pre-commit`, `commit-msg`, and `pre-push` hooks passed, or
`npm run validate:pr` passed after refreshing `origin/main` when hooks
were skipped or unavailable
- [x] Targeted behavior tests pass for the current change set, or tests
are marked not applicable above — command/result or justification: `npx
vitest run test/changelog-docs.test.ts` passed 6 tests.
- [ ] Applicable broad gate passed — `npm test` for broad
runtime/test-harness changes; `npm run check` for repo-wide
validation/coverage changes — command/result: Not applicable to a
documentation-only release-entry update.
- [x] Quality Gates section completed with required justifications or
waivers
- [x] No secrets, API keys, or credentials committed
- [ ] `npm run docs` builds without warnings (doc changes only) —
result: passed with 0 errors and 2 existing warnings.
- [x] Doc pages follow the [style
guide](https://github.com/NVIDIA/NemoClaw/blob/main/docs/CONTRIBUTING.md)
(doc changes only)
- [ ] New doc pages include SPDX header and frontmatter (new pages only)
— no page was added.

---
Signed-off-by: Julie Yaunches <jyaunches@nvidia.com>


<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->

## Summary by CodeRabbit

* **Bug Fixes**
* Improved gateway readiness detection for OpenShell v0.0.101 endpoint
output.
* Process tags are now accepted only when they match trusted listener
information for the configured gateway.
* Preserved stale gateway status during connection failures to support
accurate onboarding reconciliation.

<!-- end of auto-generated comment: release notes by coderabbit.ai -->

Co-authored-by: Carlos Villela <cvillela@nvidia.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area: ci CI workflows, checks, release automation, or GitHub Actions area: e2e End-to-end tests, nightly failures, or validation infrastructure

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant