Skip to content

test(fetch-tcp-stress): drop duplicate variant, scale threshold under debug/ASAN - #36207

Open
robobun wants to merge 4 commits into
mainfrom
farm/53251a84/speed-up-fetch-tcp-stress
Open

robobun wants to merge 4 commits into
mainfrom
farm/53251a84/speed-up-fetch-tcp-stress

Conversation

@robobun

@robobun robobun commented Jul 28, 2026 •

Copy link
Copy Markdown
Collaborator

Speeds up test/js/web/fetch/fetch-tcp-stress.test.ts without weakening what it covers.

What changed

  • Dropped the duplicate test. "shutdown after timeout" and "gently close" had byte-identical bodies (socket.end() on the server, no-op on the client). That is one quarter of the file's wall time spent re-running the same work. The three remaining variants are collapsed into a table.
  • Stopped building 32k request-option objects per test. The batch loop only ever indexes objects[0..47], so allocating total entries (32768 on release) with per-entry string concatenation was dead weight.
  • Scaled the port-exhaustion threshold under debug/ASAN. Those builds iterate roughly 30x slower, so TIME_WAIT sockets drain faster than they accumulate and the 16k threshold only costs time. Without this the file times every test out at 30s under bun bd test. At 2k the fd-leak assertion and every close path still run. Release builds keep the full 16k (8k on macOS) threshold.
  • Tightened the assertions. The server now counts how many request bodies it matched and the test asserts that equals the number of fetch() calls issued, so a dropped connection fails loudly instead of the old console.log("Data is missing!"). Also asserts initialMaxFD was actually captured.

Why this is correct

The port-exhaustion aspect of the test is only meaningful at release speed, so the release threshold is unchanged. Under debug/ASAN the loop is slow enough that the ephemeral-port window can never fill, so only the fd-leak assertion and the close-path coverage do anything; 2k is well past the point where a one-fd-per-batch leak would exceed the +10 margin. No coverage is dropped: the removed test was an exact duplicate.

Running the three variants with test.concurrent was tried but gives no wall-time improvement on Windows aarch64 (each test went from ~7s to ~20s, same total) and makes the per-test getMaxFD() check racy, so the file stays sequential.

Timings

lane before after
windows 11 aarch64 (release) 28.3s 20.0s
linux x64 debug+ASAN (bun bd test) 122.7s, 4/4 time out at 30s 19.5s, 3/3 pass
linux x64 release 4.7s 3.9s

Stable across three consecutive runs on each.


no test proof · iteration 0 · Platform-specific test-only change; deferring to CI.

@coderabbitai

coderabbitai Bot commented Jul 28, 2026 •

Copy link
Copy Markdown
Contributor

Warning

Review limit reached

@robobun, you've reached your PR review limit, so we couldn't start this review.

Next review available in: 9 minutes

Enable usage-based reviews in Billing to review now. Otherwise, wait until the next included review is available.
You're only billed for reviews past your plan's rate limits ($0.25/file).

How can I continue?

After more reviews become available, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews.

How do review limits work?

CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability.

For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window.

Please refer docs for additional details.

Review details
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: ASSERTIVE

Plan: Pro

Run ID: cb23aff7-6a15-4200-932b-ae04f1f5f038

📥 Commits

Reviewing files that changed from the base of the PR and between e532ad9 and 2b62faa.

📒 Files selected for processing (1)
  • test/js/web/fetch/fetch-tcp-stress.test.ts

Comment @coderabbitai help to get the list of available commands.

@robobun

robobun commented Jul 28, 2026 •

Copy link
Copy Markdown
Collaborator Author
Updated 4:07 PM PT - Jul 28th, 2026

❌ @robobun, your commit 2b62faa has 1 failures in Build #84312 (All Failures):


🧪   To try this PR locally:

bunx bun-pr 36207

That installs a local version of the PR into your bun-36207 executable, so you can run:

bun-36207 --bun

@github-actions

Copy link
Copy Markdown
Contributor

This PR may be a duplicate of:

  1. make more tests faster #33704 - Also scales the fetch-tcp-stress PORT_EXHAUSTION_THRESHOLD to 2k under ASAN/debug builds, as part of a broader test speed-up effort

🤖 Generated with Claude Code

@robobun

robobun commented Jul 28, 2026

Copy link
Copy Markdown
Collaborator Author

Not a duplicate. #33704 touches many files and its only change to this one is the debug/ASAN threshold; this PR is scoped to fetch-tcp-stress.test.ts and additionally drops a byte-identical duplicate test, stops the 32k-entry over-allocation, and tightens the assertions. If #33704 lands first this rebases cleanly over it.

@claude claude Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM — test-only speedup with verifiably correct simplifications and strictly stronger assertions.

What was reviewed:

  • Confirmed "shutdown after timeout" and "gently close" had byte-identical bodies (socket.end() / no-op) — dropping one loses no coverage.
  • Confirmed the batch loop only ever indexes objects[0..47], so shrinking allocation from total to batch is dead-code removal.
  • The debug/ASAN threshold branch follows the repo convention (branch per build type, release lanes keep the full 16k/8k threshold); isASAN/isDebug both exist in harness.
  • Checked that the new serverReceived === issued / bodyMismatches === 0 assertions don't add fragility beyond the pre-existing body-parse logic — they turn a silent console.log into a real assertion.
Extended reasoning...

Overview

Single-file test change to test/js/web/fetch/fetch-tcp-stress.test.ts. Four modifications: (1) removes one of two byte-identical test variants and collapses the remaining three into a table, (2) shrinks the objects array from total (up to 32k) entries to batch (48) since only indices 0..47 are ever read, (3) branches PORT_EXHAUSTION_THRESHOLD on isASAN || isDebug to 2048 while keeping release at 16k/8k, (4) replaces a console.log("Data is missing!") with counted assertions on serverReceived/bodyMismatches/initialMaxFD.

Security risks

None. Test-only; no runtime code, no network egress beyond localhost, no auth/crypto surface.

Level of scrutiny

Low-to-moderate. The mechanical changes (duplicate removal, array shrink, table refactor) are directly verifiable from the diff and I confirmed each. The threshold branch is a judgment call, but it follows REVIEW.md's explicit guidance ("branch per-platform rather than dropping precision", "thresholds branch on isASAN/isDebug") and the PR description gives a sound argument: at debug/ASAN speed the ephemeral-port window drains faster than it fills, so only the fd-leak check does work — and 2048/48 ≈ 43 batches is still far past the point where a per-batch fd leak would blow the +10 margin. Release lanes are unchanged, so the port-exhaustion coverage that matters is preserved.

Other factors

The tightened assertions are the only place new flakiness could enter. I traced whether bodyMismatches++ could fire on TCP-fragmented requests: if the body marker were split across data calls, the pre-existing objects[i].body dereference would already throw on undefined.body, so the new assertion adds no fragility beyond what the test's structure already assumes. serverReceived === issued will now fail loudly if a connection is dropped where Promise.allSettled previously swallowed it — that's the stated intent and matches REVIEW.md's "assert the strongest invariant". The test.todoIf(isCI && isMacOS) gate on the historically-flaky lane is preserved. PR reports stable timings across three runs on three lanes (Windows aarch64 release, Linux debug+ASAN, Linux release).

Comment thread test/js/web/fetch/fetch-tcp-stress.test.ts Outdated
Comment thread test/js/web/fetch/fetch-tcp-stress.test.ts Outdated

@claude claude Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM — both earlier nits are addressed in the current diff (change-history comment gone, initialMaxFD dropped from the toEqual).

What was reviewed:

  • Confirmed the removed "shutdown after timeout" case was byte-identical to "gently close" in the old code; the three surviving variants match the originals exactly.
  • Verified objects[] is only ever indexed by 0..batch-1 (both in the fetch loop and via the parsed marker on the server side), so sizing it to batch is safe.
  • Checked the new serverReceived === issued / bodyMismatches === 0 assertions can't introduce flakiness the old test didn't already have — a mismatch previously left toClose non-zero and hung on pendingClose, so this converts a 30s timeout into a fast failure.
  • isASAN/isDebug threshold branching matches harness exports and repo convention; release lanes keep the full 16k/8k count.
Extended reasoning...

Overview

Test-only change to test/js/web/fetch/fetch-tcp-stress.test.ts: drops one of four test variants that was byte-identical to another, collapses the remaining three into a table-driven loop, sizes the objects[] array to batch (48) instead of total (up to 32768), branches PORT_EXHAUSTION_THRESHOLD on isASAN || isDebug, and replaces a console.log("Data is missing!") with counted assertions on serverReceived and bodyMismatches. No production code touched.

Security risks

None. Test file only; no auth, crypto, or user-input handling.

Level of scrutiny

Low-to-moderate. The two things that could go wrong with a stress-test cleanup are (a) silently weakening coverage and (b) introducing flakiness. On (a): release builds keep the exact same iteration count and the removed variant was verifiably a duplicate — I diffed the old "shutdown after timeout" and "gently close" bodies in the removed hunks and both are socket.end() / no-op. The debug/ASAN reduction is justified per REVIEW.md's "shrink the workload" guidance and the PR's reasoning (TIME_WAIT drains faster than accumulation at debug speed) is sound; 4096 requests across ~86 batches is ample for the getMaxFD() < initialMaxFD + 10 leak check. On (b): the new serverReceived === issued assertion codifies what await pendingClose.promise already required — if a body ever failed to match, the old test never wrote a response, toClose never reached 0, and it timed out at 30s. So the tightened assertion converts an existing timeout-failure into a fast failure with a clear message rather than adding a new failure mode.

Other factors

Both nits from my prior review pass are resolved in the current diff (the vacuous initialMaxFD: expect.any(Number) clause and the change-history comment). isASAN and isDebug are confirmed exports from test/harness.ts. The objects[] sizing change is provably correct: the fetch loop indexes objects[0..47] and the server-side index is parsed from a marker of 10 + i where i came from that same 0..47 range. test.todoIf(isCI && isMacOS) and the 30s per-test timeout are preserved unchanged.

@claude claude Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM — both prior nits addressed (change-history comment removed, vacuous expect.any(Number) clause dropped).

What was reviewed:

  • Confirmed the removed "shutdown after timeout" variant was byte-identical to "gently close" — no coverage lost.
  • Verified the batch loop and server data handler only ever index objects[0..47], so sizing the array to batch instead of total is safe.
  • Release-lane threshold is unchanged; only debug/ASAN is reduced, with the fd-leak assertion still meaningful at 2k iterations.
  • New serverReceived === issued assertion strictly tightens what was previously a console.log — no new flake vector vs. the pre-existing data-handler logic.
Extended reasoning...

Overview

Test-only change to test/js/web/fetch/fetch-tcp-stress.test.ts. Four things: (1) drops one of four test variants that was byte-identical to another, (2) sizes the objects request-options array to batch (48) instead of total (up to 32k) since only indices 0..47 are ever read, (3) scales PORT_EXHAUSTION_THRESHOLD to 2k under isASAN || isDebug while leaving release/macOS thresholds untouched, and (4) replaces a diagnostic console.log with counted serverReceived/bodyMismatches asserted against issued. The three surviving variants are collapsed into a table + loop.

Security risks

None. No production code touched; test uses a local Bun.listen({ port: 0 }) server over loopback.

Level of scrutiny

Low-to-moderate. It's test infrastructure, but REVIEW.md is explicit that deleting/weakening tests needs justification. Checked each axis:

  • Deleted variant: the pre-PR "shutdown after timeout" and "gently close" bodies were both { onServerWritten: s => s.end(), onFetchWritten: () => {} } — genuinely identical, so nothing is lost.
  • Threshold reduction: only applies under debug/ASAN, where per the PR's timings the file previously timed out 4/4 at 30s. Release lanes keep 16k/8k. The rationale (debug iterates ~30× slower so TIME_WAIT drains before the ephemeral range fills, leaving only the fd-leak check doing real work) is sound, and 2k × 48-per-batch iterations is well past the point where a per-batch fd leak would blow the +10 margin.
  • Array sizing: both consumers of objects[i] — the fetch loop (i < batch) and the server handler (index parsed from a two-digit marker 10..57) — stay within [0, batch).
  • Tightened assertion: the data handler's parse/match logic is unchanged from before; only the side effect on match changed from console.log to a counter. If TCP fragmentation over loopback were splitting these ~30-byte bodies, the old test would already have been hanging (socket never marked read), so this doesn't introduce a new flake mode — it makes an existing silent failure loud.

Other factors

I left two nits on the previous revision (vacuous expect.any(Number) clause; change-history comment). Both were addressed in 74b3ba9 and 8e879b7 and the threads are resolved. No outstanding reviewer comments. No CODEOWNER on this path. The bug-hunting pass on the current revision found nothing.

robobun added 4 commits July 28, 2026 21:00
… scale threshold under debug/ASAN

The "shutdown after timeout" and "gently close" test bodies were
byte-identical (both socket.end() then no-op), so one quarter of the
file's wall time was re-running the same test.

The request-options array was sized to `total` (32k entries on release)
but the batch loop only ever indexes 0..47, so 32k-48 of those objects
were dead weight built with per-entry string concat.

Under debug/ASAN the loop iterates ~30x slower so TIME_WAIT sockets
drain faster than they accumulate and the 16k threshold only costs time;
use 2k there so the fd-leak assertion and every close path still run
but the file completes instead of timing all four tests out at 30s.

Also assert the server actually received every issued request and that
no body mismatch was observed, instead of logging and carrying on, and
collapse the variants into a table.
The PR description already records why the fourth variant was removed;
per REVIEW.md comments carry durable content, not commit history.
initialMaxFD is always a Number; the >=0 assertion on the next line is
the real capture check.
@robobun
robobun force-pushed the farm/53251a84/speed-up-fetch-tcp-stress branch from 8e879b7 to 2b62faa Compare July 28, 2026 21:00

@claude claude Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM — both earlier nits (the vacuous expect.any(Number) clause and the change-history comment) are addressed.

What was reviewed:

  • Confirmed the dropped "shutdown after timeout" test was byte-identical to "gently close" — no coverage lost.
  • Traced objects[i] usage: only indices 0..47 are ever referenced (client loop and server-side BYTEMARKER parse), so sizing to batch is correct.
  • Checked the new serverReceived === issued assertion for over-count risk on chunked data events — the single-chunk assumption is pre-existing (old code would throw on objects[NaN].body), not introduced here.
  • Release thresholds and todoIf(isCI && isMacOS) are preserved; only debug/ASAN gets the 2k branch.
Extended reasoning...

Overview

Test-only change to test/js/web/fetch/fetch-tcp-stress.test.ts. Four things: (1) removes one of two byte-identical test cases, (2) reduces the objects allocation from total (~32k) to batch (48) since only those indices are ever read, (3) branches PORT_EXHAUSTION_THRESHOLD on isASAN || isDebug to 2k so the file doesn't time out under bun bd test, and (4) replaces a diagnostic console.log with counted assertions (bodyMismatches === 0, serverReceived === issued, initialMaxFD >= 0). The four test bodies are collapsed into a three-entry variants table.

Security risks

None. No runtime code touched; the test uses a local Bun.listen server on 127.0.0.1 port 0.

Level of scrutiny

Low-to-moderate. The main REVIEW.md concern for this class of change is silently weakening a test. I checked each axis: the deleted test is a verified exact duplicate (both s.end() server + no-op client); the release-lane iteration count is unchanged; the debug/ASAN reduction is the repo-endorsed pattern (branch per build type rather than raise the timeout) and 2k iterations × 48/batch is still >40 batches, well past where a per-batch fd leak would blow the initialMaxFD + 10 bound. The assertions are strictly stronger than before — the old code just printed to stdout on a body mismatch.

Other factors

I previously left two nits on this PR (a vacuous expect.any(Number) matcher clause and a change-history code comment); the author removed both in follow-up commits and the current diff reflects that. I looked at whether the new serverReceived counter could flake via multi-chunk data events — it relies on the request arriving in one TCP chunk, but the pre-existing code already depended on that (it dereferences objects[i].body where i is parsed from the chunk with no buffering, so a split chunk would already have thrown). The PR does not introduce that fragility, and the author reports stability across three runs per lane. issued correctly tracks batch per outer iteration, matching what the inner loop actually sends even when remaining < batch on the final pass.

@robobun

robobun commented Jul 28, 2026

Copy link
Copy Markdown
Collaborator Author

CI status: fetch-tcp-stress.test.ts is green on every lane in builds #83979 and #84312. The red lanes are unrelated to this diff (package-manager hoisting in bun-install-registry.test.ts on Windows aarch64, plus four tests that pass alone but fail inside the new --parallel batch from be5c92a). Ready for a maintainer.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant