Skip to content

test: de-flake updater legacy-quarantine test - #332

Merged
KrasimirKralev merged 1 commit into
betafrom
fix/updater-test-flake
Aug 9, 2026
Merged

KrasimirKralev merged 1 commit into
betafrom
fix/updater-test-flake

Conversation

@KrasimirKralev

@KrasimirKralev KrasimirKralev commented Aug 9, 2026

Copy link
Copy Markdown
Contributor

Problem

updater.test.ts > "quarantines known legacy gateway blockers and completes when the gateway recovers" fails intermittently (~1 in 3 runs) with:

AssertionError: expected undefined to deeply equal ArrayContaining [ "-lc", StringContaining "installs.json" ]

Root cause

waitForGateway() polls isPortOpen in a while (Date.now() < deadline) loop, and these tests set GATEWAY_HEALTH_WAIT_MS / GATEWAY_RECOVERY_WAIT_MS / GATEWAY_WAIT_INTERVAL_MS all to 1. The test drove recovery with a fixed call sequence:

mockIsPortOpen
  .mockResolvedValueOnce(false)
  .mockResolvedValueOnce(false)
  .mockResolvedValueOnce(true);

Because the poll deadline is ~1ms, the number of isPortOpen calls per waitForGateway invocation is timing-dependent. When the first poll loop happens to iterate more than once, it consumes the true before quarantineLegacyOpenclawState() runs — so the gateway "recovers" early, the /bin/bash quarantine step never executes, and there is no call for the assertion to find.

Fix

Tie the probe to the actual quarantine side-effect instead of a fixed call count:

mockIsPortOpen.mockImplementation(async () =>
  mockExecFile.mock.calls.some(([cmd]) => cmd === "/bin/bash"),
);

Now the gateway reports "offline" until the quarantine bash step has run, guaranteeing quarantine executes before completion regardless of poll timing.

Validation

Ran the updater suite 10× consecutively on-device (Jetson) — 26/26 passed every time (previously ~1-in-3 failed).

Summary by CodeRabbit

  • Tests
    • Updated gateway recovery coverage to detect recovery based on legacy-state quarantine behavior rather than a fixed port-probe sequence.

…uarantine side-effect)

waitForGateway polls isPortOpen in a loop and the test sets the deadline/interval
to 1ms, so the fixed false/false/true mock sequence could be fully consumed by
the first poll — reporting gateway recovery BEFORE quarantineLegacyOpenclawState
ran, so no /bin/bash call existed to assert on (~1-in-3 failure). Drive the probe
off whether the quarantine bash step has actually run instead.
@KrasimirKralev
KrasimirKralev requested a review from a team as a code owner August 9, 2026 16:42
@coderabbitai

coderabbitai Bot commented Aug 9, 2026

Copy link
Copy Markdown

Review Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: ASSERTIVE

Plan: Pro Plus

Run ID: c44e665a-10b8-4d8f-bb4a-a192c7fa0bd2

📥 Commits

Reviewing files that changed from the base of the PR and between 4af016b and 942f9ce.

📒 Files selected for processing (1)
  • src/tests/unit/updater.test.ts

📝 Walkthrough

Walkthrough

The gateway recovery test now waits for the legacy-state quarantine to invoke /bin/bash before reporting the gateway port as open.

Changes

Gateway recovery test

Layer / File(s) Summary
Quarantine-driven recovery probe
src/tests/unit/updater.test.ts
mockIsPortOpen now returns true after a /bin/bash quarantine call instead of following a fixed false, false, true sequence.

Estimated code review effort: 2 (Simple) | ~10 minutes

Suggested reviewers: yalexx

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Description check ✅ Passed The description explains the intermittent failure, root cause, fix, and validation, but it does not use the repository template headings or checklist.
Title check ✅ Passed The title clearly identifies the test flake fix and matches the main change in the pull request.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch fix/updater-test-flake

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@github-actions

github-actions Bot commented Aug 9, 2026

Copy link
Copy Markdown

🦀 ClawReview

Poked my eyestalks out for this one. Quick tour:

This PR de-flakes an intermittently-failing Vitest test in the updater suite. The old mock drove waitForGateway with a fixed false/false/true call sequence, but because the poll deadline is ~1ms, the loop could consume all three values before the legacy-quarantine bash step ran — leaving nothing for the assertion to find. The fix replaces the rigid sequence with a dynamic mockImplementation that keeps the gateway "offline" until /bin/bash has actually been called, making the guarantee timing-independent.

At a glance

  • 🧪 Tests · touches updater test suite (legacy-quarantine path)
  • Base branch: beta · +0 source / +9 tests across 1 file
  • ✅ base beta matches the beta-first convention
  • ✅ conventional PR title

Good to know

  • ℹ️ No production code changes — only src/tests/unit/updater.test.ts (+9/-4).
  • ℹ️ Validated 10× consecutively on-device (Jetson): 26/26 passed, vs ~1-in-3 failure rate before the fix.

— ClawReview 🦀, scuttling off. General info only — see CodeRabbit for the detailed review. Conventions: docs.

@github-actions

github-actions Bot commented Aug 9, 2026

Copy link
Copy Markdown

CI Summary

✅ Tests

  • Result: passed
  • View run
  • Coverage: statements 70.72%, branches 61.02%, functions 66.72%, lines 72.61%

✅ E2E

✅ E2E Install

@KrasimirKralev
KrasimirKralev merged commit db88573 into beta Aug 9, 2026
8 checks passed
@KrasimirKralev
KrasimirKralev deleted the fix/updater-test-flake branch August 9, 2026 18:45
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant