Skip to content

refactor(opencode): default it.live tests to platform-aware timeout - #656

Merged
Astro-Han merged 1 commit into
devfrom
claude/fix-windows-advisory-flake
May 15, 2026
Merged

refactor(opencode): default it.live tests to platform-aware timeout#656
Astro-Han merged 1 commit into
devfrom
claude/fix-windows-advisory-flake

Conversation

@Astro-Han

@Astro-Han Astro-Han commented May 15, 2026

Copy link
Copy Markdown
Owner

Summary

Replace hand-tuned per-test timeouts in packages/opencode/test/session/prompt-effect.test.ts with a single Windows-aware default applied through an it.live wrapper. Removes 12 third-arg timeout literals (5 × 3_000, 5 × slowIOTimeout, 2 × shellQueueTimeout) plus the two duplicated const ... Timeout definitions. Explicit non-default timeouts (5_000, 10_000, 30_000) still override. The wrapper also covers .only and .skip so debug and skip variants do not fall back to bun's 3-second default.

Why

windows-advisory has been flaking at ~55% on dev push (27 failures / 50 runs in the latest sample), driven by three independent failure modes, only one of which is fixable in our test code. H1: bun's default 3-second it.live timeout is consistently tight on the Windows runner for Effect-fiber + SQLite + tmpdir-server tests in this file. Over the past 11 days four prior PRs (#543, a3b8e54, cf6d1cd, #579) each bumped one or two tests at a time to a slowIOTimeout constant without converging — today's failure on cancel records MessageAbortedError on interrupted process was the fifth instance of the same root cause. This change makes the slow-runner default the right thing once and ends the per-test bandaging pattern. H2 (bun 1.3.13 watcher.node segfault on Windows process exit) and H3 (transient actions/cache) are upstream / infra and intentionally left to fail "normally" per the design decision in d6fa1e6: the advisory workflow is already not in branch protection, and existing if: always() artifact and summary uploads preserve the diagnostic signal without masking the red run.

Related Issue

No issue. Surfaced from a direct investigation of dev-push CI noise.

Human Review Status

Pending. A human should make the final merge decision after reviewing the final diff and verification evidence.

Review Focus

The withDefaultLiveTimeout wrapper's type signature in prompt-effect.test.ts:276 intentionally uses structural any[] parameters to avoid widening the testEffect runtime layer's generic to unknown (an earlier ReturnType<typeof testEffect> attempt failed typecheck with a layer-incompatibility cascade). Confirm this trade-off is acceptable. The wrapper now also covers .only and .skip after a Gemini suggestion — there is currently no .only usage in the file, but applying defaults consistently avoids a future Windows-debug surprise. Every previously hand-tuned site (3_000 / slowIOTimeout / shellQueueTimeout) now relies on the 10-second Windows default — confirm none of those sites used a tight timeout as a deliberate budget assertion rather than "slowest we'll tolerate".

Risk Notes

If a future bug makes a live test actually hang, detection on Windows now takes up to 10 seconds rather than 3. Acceptable for an advisory signal. No production code paths touched. windows-advisory will still show occasional red runs from H2 / H3 by design.

How To Verify

typecheck (tsgo --noEmit): clean
bun test test/session/prompt-effect.test.ts --timeout 30000: 54 pass / 0 fail / 28.69s (macOS)
bun test test/github/ci-workflow.test.ts --timeout 30000: 9 pass / 0 fail / 5.04s

Windows-side validation will come from windows-advisory runs on the post-merge dev push: the cancel records ... test and the other four hard-coded 3_000 sites should stop timing out. The bun-watcher and cache failure modes are deferred and will continue to show as occasional red runs by design.

Screenshots or Recordings

N/A — no UI change.

Checklist

  • Human review status is stated above as pending, approved, or not required
  • I linked the related issue, or stated why there is no issue
  • This PR has type, primary area, and priority labels, or I requested maintainer labeling
  • I described the review focus and any meaningful risks
  • I listed the relevant verification steps and the key result for each
  • I did not introduce unrelated refactors, dependencies, generated files, or file changes beyond the stated scope
  • I manually checked visible UI or copy changes when needed, with screenshots or recordings
  • I considered macOS and Windows impact for platform, packaging, updater, signing, paths, shell, or permissions changes
  • I called out docs, release notes, dependencies, permissions, credentials, deletion behavior, generated content, or local file changes when relevant
  • I reviewed the final diff for unrelated changes and suspicious dependency changes
  • I am targeting dev, and my PR title and commit messages use Conventional Commits in English

@coderabbitai

coderabbitai Bot commented May 15, 2026

Copy link
Copy Markdown
Contributor

Warning

Rate limit exceeded

@Astro-Han has exceeded the limit for the number of commits that can be reviewed per hour. Please wait 3 minutes and 53 seconds before requesting another review.

You’ve run out of usage credits. Purchase more in the billing tab.

⌛ How to resolve this issue?

After the wait time has elapsed, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

We recommend that you space out your commits to avoid hitting the rate limit.

🚦 How do rate limits work?

CodeRabbit enforces hourly rate limits for each developer per organization.

Our paid plans have higher rate limits than the trial, open-source and free plans. In all cases, we re-allow further reviews after a brief timeout.

Please see our FAQ for further information.

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro Plus

Run ID: 4f754271-4fcc-4bcf-b977-fa29e0607a88

📥 Commits

Reviewing files that changed from the base of the PR and between 4d8a5c9 and 4d4792c.

📒 Files selected for processing (1)
  • packages/opencode/test/session/prompt-effect.test.ts
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch claude/fix-windows-advisory-flake

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

@Astro-Han Astro-Han added the ci Continuous integration / GitHub Actions label May 15, 2026
@github-actions github-actions Bot added the harness Model harness, prompts, tool descriptions, and session mechanics label May 15, 2026
@Astro-Han Astro-Han added the windows Windows-specific label May 15, 2026

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested priority: P3 (only low-risk paths changed (.github/workflows/windows-advisory.yml, packages/opencode/test/session/prompt-effect.test.ts)).

P1/P0 are reserved for maintainer confirmation. Please relabel manually if this is a release blocker, security issue, data-loss risk, or updater/runtime failure.

@Astro-Han Astro-Han added P2 Medium priority flaky-test Non-deterministic test failure tech-debt Supplemental cleanup, maintainability, architecture, test, or quality debt context labels May 15, 2026

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request introduces a withDefaultLiveTimeout utility to automatically apply platform-specific timeouts for test effects, addressing slow execution on Windows runners and removing redundant timeout arguments across the test suite. A review comment suggests improving the wrapper to also apply these default timeouts to the .only and .skip modifiers, ensuring consistent behavior during targeted test execution.

Comment thread packages/opencode/test/session/prompt-effect.test.ts
Wrap testEffect's live runner in prompt-effect.test.ts so every live test
picks up a Windows-aware default timeout. Previously each new live test
needed a hand-tuned third-arg timeout to avoid flaking on the slow Windows
runner, and the file accumulated two duplicate slowIOTimeout/shellQueueTimeout
constants plus 12 per-test overrides. Removing the overrides keeps the same
effective coverage while making the default the right thing.

Explicit third-arg timeouts (5_000 / 10_000 / 30_000) still override.

Verified locally: typecheck clean, 54/54 pass in 25s on macOS.
@Astro-Han
Astro-Han force-pushed the claude/fix-windows-advisory-flake branch from 6819a4f to 4d4792c Compare May 15, 2026 15:52
@github-actions github-actions Bot removed the ci Continuous integration / GitHub Actions label May 15, 2026
@Astro-Han Astro-Han changed the title ci(windows-advisory): fix flake at root cause refactor(opencode): default it.live tests to platform-aware timeout May 15, 2026
@Astro-Han
Astro-Han merged commit ccb53e6 into dev May 15, 2026
26 checks passed
@Astro-Han
Astro-Han deleted the claude/fix-windows-advisory-flake branch May 15, 2026 16:07
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

flaky-test Non-deterministic test failure harness Model harness, prompts, tool descriptions, and session mechanics P2 Medium priority tech-debt Supplemental cleanup, maintainability, architecture, test, or quality debt context windows Windows-specific

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant