Skip to content

test: size error-gc-test.test.js loops for debug builds - #38400

Open
robobun wants to merge 1 commit into
mainfrom
farm/daf0f5d6/error-gc-test-debug-counts
Open

robobun wants to merge 1 commit into
mainfrom
farm/daf0f5d6/error-gc-test-debug-counts

Conversation

@robobun

@robobun robobun commented Aug 14, 2026

Copy link
Copy Markdown
Collaborator

Problem

  • bun bd test test/js/bun/util/error-gc-test.test.js always fails on a debug build, on a clean checkout of main:
    (fail) error gc test [92037.49ms]
      ^ this test timed out after 5000ms.
    (fail) error gc test #2 [6986.80ms]
    (fail) error gc test #3 [7497.05ms]
    (fail) error gc test #4 [47917.83ms]
     0 pass, 4 fail, Ran 4 tests across 1 file. [157.23s]
    
    The tests are synchronous, so each one runs to completion and is then reported as timed out; the file takes about 2.5 minutes to fail.
  • Cause: the loop counts in test/js/bun/util/error-gc-test.test.js are sized for release builds. Test 1 inspects an error 1000 times and runs two Bun.gc(true) per outer iteration, 100 times over; tests Fix calling #private() functions in classes #2 and Copy source lines when generating error messages #3 create an error and call Bun.gc() 1000 times; test Support import assertions #4 runs a failing readFileSync, Bun.inspect and two Bun.gc(true) 1000 times. On a debug+ASAN build Bun.inspect(err) costs about 0.9ms (0.03ms on release) and Bun.gc(true) about 20ms (1ms on release), so the four tests take 17x to 38x longer than on a release build of the same machine (2.4s / 0.4s / 0.45s / 2.3s).
  • CI does not see this because scripts/runner.node.mjs passes its own --timeout (90s, tripled under ASAN). It breaks every local bun bd test run of the file, and nothing can be added to the file and pass under bun bd test.

Fix

  • Each test picks its count with isDebug from test/harness.ts: test 1 runs 5 errors x 100 inspects (was 100 x 1000), tests Fix calling #private() functions in classes #2 and Copy source lines when generating error messages #3 run 100 iterations (was 1000), test Support import assertions #4 runs 10 (was 1000). Release builds keep the existing counts, so CI (release and release ASAN lanes) runs exactly the work it ran before.
  • Why the smaller counts still cover what the file exists for: the tests guard against the stack trace formatter unbalancing the refcount of the strings it reads (source URL, function name, the ENOENT path; the original fix is c6f6db9 "Ref the strings", today Bun::toStringRef in src/jsc/bindings/ZigException.cpp). Debug builds assert on the refcount the first time it goes wrong (debug_assert!(old > 0) in WTFStringImplStruct::ref/deref, src/bun_alloc/lib.rs), so the repetition only buys something on release builds, where a string freed early is noticed only once its memory is reused or freed again. Checked by reintroducing the bug: with line 125 of ZigException.cpp changed to the non-ref'ing Bun::toString, the reduced file still crashes inside the first error of test 1 (details below).
  • Why these numbers: they keep each test under about a second on the debug+ASAN build here (10 and 25 iterations of test Support import assertions #4 measure 0.64s and 1.59s; 5 x 100 and 10 x 100 of test 1 measure 0.78s and 1.6s), which leaves room for slower machines under the 5s default. No per-test timeout is added (test/CLAUDE.md).
  • Verified:
    • bun bd test test/js/bun/util/error-gc-test.test.js (debug+ASAN): 4 pass, 797ms / 740ms / 758ms / 783ms. The same command on the unmodified file is the failure quoted above.
    • USE_SYSTEM_BUN=1 bun test test/js/bun/util/error-gc-test.test.js (release, unchanged counts): 4 pass, 2.46s / 0.48s / 0.53s / 2.63s, the same work as before the change.
    • Test-only change, so there is nothing under src/ to stash for a fail-before run; the before/after above is the test file itself.

Background

Reintroducing the bug against the reduced counts

src/jsc/bindings/ZigException.cpp:125 changed from Bun::toStringRef(functionName) to Bun::toString(functionName) (no ref, the pre-c6f6db95ff behavior), rebuilt with bun bd, then bun bd test test/js/bun/util/error-gc-test.test.js with this PR's counts. The run dies before test 1 reports a result:

panic: assertion failed: old > 0
[17239] pas panic: deallocation did fail at 0x725efe250960: Alloc bit not set in pas_segregated_page_deallocate_with_page
...
<bun_alloc::WTFStringImplStruct>::ref

The src/ change was reverted before the verification runs above.

Every test in the file does a synchronous GC per iteration at counts
sized for release builds, so under a debug build all four run past the
default 5s timeout (92s, 7.0s, 7.5s, 47.9s on debug+ASAN) and the file
always fails under `bun bd test`. Pick smaller counts when isDebug;
release builds keep the existing counts.
@coderabbitai

coderabbitai Bot commented Aug 14, 2026

Copy link
Copy Markdown
Contributor

Warning

Review limit reached

@robobun, you've reached your PR review limit, so we couldn't start this review.

Next review available in: 3 minutes

Enable usage-based reviews in Billing to review now. Otherwise, wait until the next included review is available.
You're only billed for reviews past your plan's rate limits ($0.25/file).

How can I continue?

After more reviews become available, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews.

How do review limits work?

CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability.

For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window.

Please refer docs for additional details.

Review details
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: ASSERTIVE

Plan: Pro

Run ID: 5ef93c24-50f4-4119-966f-c5985ddb9e86

📥 Commits

Reviewing files that changed from the base of the PR and between 01aa7cd and 56e5cfd.

📒 Files selected for processing (1)
  • test/js/bun/util/error-gc-test.test.js

Comment @coderabbitai help to get the list of available commands.

@robobun

robobun commented Aug 14, 2026 •

Copy link
Copy Markdown
Collaborator Author
Updated 6:05 AM PT - Aug 14th, 2026

❌ @robobun, your commit 56e5cfd has some failures in Build #95614 (All Failures)


🧪   To try this PR locally:

bunx bun-pr 38400

That installs a local version of the PR into your bun-38400 executable, so you can run:

bun-38400 --bun

@robobun

robobun commented Aug 14, 2026

Copy link
Copy Markdown
Collaborator Author

Status: fix pushed, waiting on CI.

Reproduced on main with a debug+ASAN build (bun bd test test/js/bun/util/error-gc-test.test.js): all 4 tests time out at the default 5s (92s / 7.0s / 7.5s / 47.9s, file fails after 157s). With this branch the same command passes with every test under 0.8s, and USE_SYSTEM_BUN=1 bun test (release, counts unchanged) still passes.

@claude claude Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM — test-only change that reduces loop counts under isDebug while leaving release/CI counts untouched.

What was reviewed

  • Confirmed isDebug is exported from test/harness.ts and that release-build iteration counts are byte-identical to before.
  • Checked this follows the same pattern as #38259 / #36929 / #37835 / #37488; the added comment explains why debug builds don't need the repetition (refcount debug_assert fires on first imbalance).
  • Verified the diff removes no assertions and adds no per-test timeout override.
Extended reasoning...

Overview

This PR touches a single test file, test/js/bun/util/error-gc-test.test.js, and only changes loop iteration counts to be conditional on isDebug from the test harness. Under release builds (what CI runs), every count is exactly what it was before (100×1000, 1000, 1000, 1000). Under debug builds, counts drop to 5×100, 100, 100, and 10 so the file completes within the default 5s per-test timeout instead of taking ~2.5 minutes and being reported as timed out. A four-line comment is added explaining why the reduced counts are safe on debug builds (the WTFStringImplStruct::ref/deref debug_assert catches an unbalanced refcount on the first occurrence, so repetition is only needed on release where the failure mode is a later reuse/double-free).

Security risks

None. Test-only change to iteration counts in a GC stress test; no source code, no user-facing behavior, no I/O paths, no auth/crypto.

Level of scrutiny

Low. This is a mechanical test-sizing change following an established repo pattern (four cited precedent PRs doing the same thing to other slow test files). Release-build behavior — and therefore CI coverage — is completely unchanged. The one review concern for this class of change ("does the reduced count still catch the bug it guards against?") is directly addressed in the PR description: the author reintroduced the original bug (Bun::toString instead of Bun::toStringRef in ZigException.cpp) and confirmed the reduced-count file still crashes on the first error under a debug build. That satisfies the REVIEW.md "when de-flaking, keep asserting the property the original assertion protected" rule.

Other factors

  • isDebug is confirmed present in test/harness.ts:29.
  • No assertions were removed or weakened; no setDefaultTimeout or per-test timeout was added (per test/CLAUDE.md guidance).
  • The added comment is load-bearing (explains the non-obvious reason lower counts are safe on debug), not narration.
  • No prior human or bot review comments to address; the bug-hunting system found nothing.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant