Skip to content

test: make the h2 stream-reset flood tests pass on debug builds - #44256

Open
robobun wants to merge 4 commits into
mainfrom
robobun/722fe409/h2-flood-tests-debug-build
Open

robobun wants to merge 4 commits into
mainfrom
robobun/722fe409/h2-flood-tests-debug-build

Conversation

@robobun

@robobun robobun commented Sep 29, 2026 •

Copy link
Copy Markdown
Collaborator

Problem

  • On a debug build, the stream-reset floods tests in test/js/node/http2/h2-conformance.test.ts fail with this test timed out after 5000ms or timed out waiting for frame.
  • 1000 streams take a release build about 30 ms and a debug build 1 to 15 s. The default timeout is 5 s.
  • The bucket regains 33 tokens per second (RateLimit::drain, src/runtime/api/bun/h2/connection.rs:170). A flood of 1200 resets that takes more than 6 s never empties it.

Fix

  • flood() sends a PING after the pairs. It writes the pairs again only if the ack comes with no GOAWAY after more than 6 s. A faster build must send the GOAWAY.
  • Six tests must send more than 1000 streams. On debug builds only, they get a 60 s timeout and 20 s waits.
  • Two tests do not need a full bucket and send 100 streams.
  • Verified: bun bd test test/js/node/http2/h2-conformance.test.ts (84 pass). With one CPU, main fails 6 to 7 of 14 tests in 4 of 4 runs. This branch passes 3 of 3.

Background

  • The server counts stream resets in a token bucket: 1000 tokens, 33 more per second (node:http2: rate-limit stream resets per connection (CVE-2023-44487 rapid reset, CVE-2025-8671 MadeYouReset) #36230). An empty bucket sends GOAWAY(ENHANCE_YOUR_CALM).
  • A second bucket counts the resets the server sends itself. No option changes its size of 1000, so those tests cannot send less.
  • CI runs bun test --timeout with 90 s or more. Only local runs use the 5 s default.
  • Considered a longer timeout alone: a slow run still gets no GOAWAY. Considered one larger write: two writes leave the first write unchanged.
Notes

Reproduction (linux x64, debug build from bun bd, main at 9f70da0):

  • bun bd test test/js/node/http2/h2-conformance.test.ts -t "stream-reset floods": main fails 2 of 9 runs, with this test timed out after 5000ms in the first test (at 5204 ms and 7186 ms). In the other 7 runs that test takes 3.9 to 5.2 s. This branch (a888e0f) passes 3 of 3 runs, one of them with the first test at 7057 ms.
  • One CPU: taskset -c <cpu> ./build/debug/bun-debug test test/js/node/http2/h2-conformance.test.ts -t "stream-reset floods". Main: 7, 6, 6, 6 of 14 fail, with timed out waiting for frame and Unhandled error between tests in every run. This branch: 14 pass, 3 of 3 runs.
  • The first commit (da87031) passed 6 of 6 runs without pinning and 4 of 4 with one CPU. Its whole-file runs with one CPU gave 84 pass in 2 of 2 (main: 9 and 7 failures).

The arithmetic. The bucket is empty at reset 1001 + 33 * k, where k is the count of whole seconds since the connection started.

  • One write of 1200 pairs empties it for k <= 6. So a build that takes the pairs in less than 6 s must send the GOAWAY, and flood() fails if it does not. A probe that sends the same frames with one CPU: the first write took 13.6 s (RST_STREAM) and 10.5 s (WINDOW_UPDATE) and got no GOAWAY. The second write got it after 2.0 s and 5.3 s.
  • Two writes of 1200 pairs empty it for k1 + k2 <= 42, which is 21 s for each write. The wait is 20 s on a debug build.
  • server-sent resets stay charged after the server has sent its GOAWAY resets streams that must exist before the GOAWAY, so a second write is not possible. On a debug build it has 1700 streams (other builds: 1300, as before). 1001 + 33 * 21 = 1694, which covers a wait of 20 s. In the probe with 1300 streams and one CPU, the reset phase took 6.4 s and needed 1199 of them.

The upper bound. With streamResetBurst: 1500 in the first test (a bucket that is too large), the test fails on the release build and on the debug build with the server took the whole flood and sent no GOAWAY. The flood() of the first commit wrote a second time without the 6 s rule and passed that case on the release build.

The six tests with the debug timeout send a whole bucket: the RST_STREAM flood with default options, the three floods of server-sent resets, a header block refused before dispatch is not charged, and server-sent resets stay charged after the server has sent its GOAWAY. The last five use the bucket for server-sent resets (sent_reset_limit, connection.rs:390), which no option changes. The other eight tests keep the default timeout and their 10 s waits.

The two smaller tests.

  • a flood under the burst keeps the session serving requests: 100 pairs, not 500. The burst is the default of 1000 as before.
  • resets of streams opened after the server's GOAWAY are charged: streamResetBurst: 50 and 100 pairs, not the default burst and 1200 pairs. The flood still uses stream ids below the held stream.

Times per test, debug build (ms, -t "stream-reset floods". Main: 3 runs, and 4 runs with one CPU. Branch at a888e0f: 3 runs, and 3 runs with one CPU):

test main branch main, one CPU branch, one CPU
RST_STREAM flood, default options 4725 to 5169 4312 to 7057 fail, 4 of 4 15911 to 17205
streamResetBurst 294 to 312 184 to 410 2209 to 2780 1129 to 1844
flood under the burst 1255 to 1489 199 to 659 5172 to 5765 1266 to 2925
bucket refill 1672 to 1799 1729 to 1964 2689 to 3052 2456 to 2745
server-sent resets, WINDOW_UPDATE 0 1960 to 2347 2041 to 2609 fail, 4 of 4 10037 to 13246
server-sent resets, WINDOW_UPDATE past 2^31-1 1892 to 2389 1876 to 2397 fail, 4 of 4 5884 to 8606
server-sent resets, DATA after END_STREAM 1707 to 2190 2055 to 2325 fail, 4 of 4 6591 to 11441
RST_STREAM for a closed stream 157 to 209 153 to 209 694 to 871 465 to 913
late DATA 161 to 218 180 to 212 1099 to 1539 468 to 1017
stream error charged once 166 to 182 114 to 231 529 to 943 426 to 1407
header block refused 1526 to 1695 1215 to 1943 fail, 1 of 4 5540 to 7106
not charged after the GOAWAY 187 to 211 132 to 240 750 to 2748 390 to 507
opened after the GOAWAY 1585 to 1851 136 to 191 fail, 4 of 4 482 to 1512
server-sent resets after the GOAWAY 3308 to 3728 3460 to 4539 fail, 4 of 4 13645 to 14135

The whole file takes 58 s on main (1 run) and 52 to 81 s on this branch (3 runs, the slowest under a host load average of 564).

Release build (canary of main at 9f70da0): the block passes 3 of 3 runs in 2.7 to 3.2 s (main: 2.4 to 3.6 s). The file passes 3 of 3 runs.

A build with no reset limit (bun 1.4.3-canary at 367d939, before #36230): the same 8 tests fail with the old file and with the new file, and the same 6 pass. The five flood() tests now fail in 23 to 382 ms with the server took the whole flood and sent no GOAWAY. Before, each of them ran into the 5 s timeout, and its wait rejected later as Unhandled error between tests. The block takes 16.0 s (before: 40.9 s).

More load. One run of the first commit with a busy loop on the same CPU: the six tests with the debug timeout pass (9.7 to 32.7 s each). One test with the default timeout, streamResetBurst sets where the flood is detected, timed out at 5 s.

Why not one larger write. On the release build, one write of 4000 or more pairs (144 KB) gives the raw client ECONNRESET and no GOAWAY (15 of 15 runs). 3500 pairs or fewer give the GOAWAY (18 of 18 runs). I did not look for the cause, and I cannot run Windows or macOS to see where that limit is there. The first write stays at 1200 pairs (43 KB) on every build.

Seen, not changed here.

  • With one CPU, the stream release after a queued END_STREAM cases in the same file take 1.7 to 4.9 s and ran into the 5 s timeout in 2 of 4 runs of the whole file. test: deflake the h2 stream-release cases on debug builds #42357 is open for those cases.
  • Without pinning, 13 runs of the whole file at a888e0f gave 84 pass in 12. The other run had one failure in those cases: server streams answered with end("ok") behind another stream's stalled response are released, with Expected: <= 3 and Received: 6. The 14 flood tests passed in all 13 runs.
  • A test with the default timeout that runs into it leaves its 10 s wait pending. The wait rejects later as Unhandled error between tests. This is the same on main.

Not run locally. Windows and macOS.


[auto-merge] gate passed · iteration 1 · 1 files touched

passes on PR (with fix)
Test-only change.

Debug/ASAN (expected pass):
$ bun bd test 'test/js/node/http2/h2-conformance.test.ts'
$ BUN_DEBUG_QUIET_LOGS=1 bun scripts/build.ts --profile=debug --quiet test "test/js/node/http2/h2-conformance.test.ts"
bun test v1.4.3 (367d939d9)

test/js/node/http2/h2-conformance.test.ts:
(pass) connection preface & SETTINGS handshake (checklist §1) > server sends a SETTINGS frame first (§1.4) [664.70ms]
(pass) connection preface & SETTINGS handshake (checklist §1) > server ACKs the client's SETTINGS frame (§3.5) [204.37ms]
(pass) connection preface & SETTINGS handshake (checklist §1) > a SETTINGS frame with a non-zero stream id is a PROTOCOL_ERROR (§3.5) [147.12ms]
(pass) connection preface & SETTINGS handshake (checklist §1) > a SETTINGS frame whose length is not a multiple of 6 is a FRAME_SIZE_ERROR (§3.5) [78.84ms]
(pass) connection preface & SETTINGS handshake (checklist §1) > a SETTINGS ACK that carries a payload is a FRAME_SIZE_ERROR (§3.5) [64.77ms]
(pass) PING (checklist §3.7) > server replies to PING with a PING ACK echoing the payload [62.89ms]
(pass) PING (checklist §3.7) > a PING with length != 8 is a FRAME_SIZE_ERROR [67.88ms]
(pass) PING (checklist §3.7) > a PING on a non-zero stream id is a PROTOCOL_ERROR [54.01ms]
(pass) WINDOW_UPDATE (checklist §6) > a connection-level WINDOW_UPDATE with a 0 increment is a PROTOCOL_ERROR [57.67ms]
(pass) WINDOW_UPDATE (checklist §6) > a WINDOW_UPDATE with length != 4 is a FRAME_SIZE_ERROR [60.63ms]
(pass) frame structure (checklist §2,§3) > an unknown frame type is ignored, not an error (§2.4) [82.69ms]
(pass) frame structure (checklist §2,§3) > RST_STREAM on an idle stream is a PROTOCOL_ERROR (§4) [71.41ms]
(pass) stream-id rules (checklist §3) > HEADERS on stream 0 is a PROTOCOL_ERROR (§6.2) [55.16ms]
(pass) stream-id rules (checklist §3) > DATA on stream 0 is a PROTOCOL_ERROR (§6.1) [45.88ms]
(pass) fixed-length frames (checklist §3) > PRIORITY with length != 5 is a FRAME_SIZE_ERROR (§6.3) [46.76ms]
(pass) fixed-length frames (checklist §3) > RST_STREAM with length 
... (truncated)
Exit: 0
diff hotspot
test/js/node/http2/h2-conformance.test.ts | 159 +++++++++++++++++++-----------
 1 file changed, 101 insertions(+), 58 deletions(-)

gate history · 2 passed · 1 rejected · iteration 1

evidence per changed file
file                                       reads  edits  tests
test/js/node/http2/h2-conformance.test.ts      9     10     52

A debug build needs 1 to 15 s for the 1000 streams of one flood. The
tests ran with the 5 s default timeout. The bucket also regains 33
tokens per second, so a flood of 1200 resets that takes more than 6 s
never empties it, and no GOAWAY comes.

- flood() writes the pairs once more when the server answered the first
  write without a GOAWAY. A PING behind the pairs shows that.
- The tests that have to send a whole bucket of 1000 resets get a 60 s
  timeout on debug builds. Other builds keep the default.
- Two tests do not need a whole bucket. They send 100 streams now.
- The flood that must be older than the server's GOAWAY cannot be
  written once more. It has 1700 streams, enough for a wait of 20 s.
@coderabbitai

coderabbitai Bot commented Sep 29, 2026 •

Copy link
Copy Markdown
Contributor

Review in Change Stack →

Navigate logical layers of code changes, visualize relationships, and explore their blast radius.

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Repository: oven-sh/bun/.coderabbit.yaml

Review profile: ASSERTIVE

Plan: Essentials

Run ID: fa332f93-a9c7-4abf-a7f0-279ee0fffcdb

📥 Commits

Reviewing files that changed from the base of the PR and between bc9fae8 and 1649cc8.

📒 Files selected for processing (1)
  • test/js/node/http2/h2-conformance.test.ts

Included review availability: This review used your included allowance. Your plan provides up to 10 included reviews per hour; 6 remain after this review.


Walkthrough

The HTTP/2 conformance tests update reset-flood timing and batch sizes. They use debug-dependent waits and timeouts, revise reset and upload counts, and check reset accounting around malformed headers and GOAWAY.

Changes

HTTP/2 reset conformance

Layer / File(s) Summary
Reset-flood timing and batches
test/js/node/http2/h2-conformance.test.ts
The flood helper uses configurable waits and sends request/reset batches separated by PING barriers. Flood tests use debug-dependent timeouts and revised reset counts.
Reset accounting and GOAWAY cases
test/js/node/http2/h2-conformance.test.ts
The malformed-header test waits longer for stream 2401. GOAWAY tests use configurable waits and update client reset and server upload counts.

Suggested reviewers: jarred-sumner, cirospaciari

Priority: ⬇️ Low

Merge Risk: ⚪ Minimal · up to 1649c

The changes improve debug-build flood-test tolerance while preserving reset-accounting checks. No actionable merge-blocking risk remains after normal checks.

🚥 Pre-merge checks | ✅ 4
✅ Passed checks (4 passed)
Check name Status Explanation
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
Title check ✅ Passed The title clearly identifies the main change: updating HTTP/2 stream-reset flood tests so they pass on debug builds.
Description check ✅ Passed The description explains the problem, implementation, verification results, test limitations, and technical background. It does not use the template headings exactly, but it provides the required info…

Comment @coderabbitai help to get the list of available commands.

@robobun

robobun commented Sep 29, 2026 •

Copy link
Copy Markdown
Collaborator Author

Status: reproduced on linux x64 with the debug build from bun bd.

  • Pinned to one CPU (taskset -c <cpu> ./build/debug/bun-debug test test/js/node/http2/h2-conformance.test.ts -t "stream-reset floods"), main fails 6 to 7 of the 14 tests in 4 of 4 runs. The errors are this test timed out after 5000ms and timed out waiting for frame.
  • Without pinning, bun bd test of the same block fails 2 of 9 runs on main.
  • With this PR (a888e0f) the block passes 3 of 3 pinned runs and 3 of 3 runs without pinning. bun bd test test/js/node/http2/h2-conformance.test.ts passes (84 tests).

@robobun

robobun commented Sep 29, 2026

Copy link
Copy Markdown
Collaborator Author

The automated review above has no actionable items, and no review threads are open.

One correction to its walkthrough: this PR adds no test coverage. The refused header block test and the GOAWAY tests exist on main. The PR changes the flood helper, the timeouts on debug builds, and three stream counts.

@claude claude Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nothing blocking. The comments below are optional suggestions. There is no need to push a fix for them before merging.

Comment thread test/js/node/http2/h2-conformance.test.ts Outdated
Comment thread test/js/node/http2/h2-conformance.test.ts
Comment thread test/js/node/http2/h2-conformance.test.ts Outdated
Comment thread test/js/node/http2/h2-conformance.test.ts Outdated
- flood() writes the pairs a second time only when the first write took
  more than 6 s. A faster build must answer 1200 resets with the GOAWAY,
  as before. A bucket of 1500 passed the previous commit on a release
  build and fails now.
- The tests with the default timeout keep their 10 s waits. Only the
  six tests with the debug timeout wait 20 s on a debug build.
- The flood that must be older than the server's GOAWAY has 1700
  streams on debug builds only. Other builds keep 1300.
@robobun

robobun commented Sep 29, 2026

Copy link
Copy Markdown
Collaborator Author

The review has four comments. a888e0f changes the code for two of them, and the other two have an answer in their threads. All four threads are resolved.

  • Upper bound of the bucket: flood() writes a second time only when the first write took more than 6 s. A bucket of 1500 fails the first test on a release build again.
  • Late rejections: the tests with the default timeout keep their 10 s waits, as on main.
  • Larger flood: the last test has 1700 streams on debug builds only. Other builds keep 1300.
  • Not changed: the timeout on debug builds for the six tests that need a full bucket, and the wait that covers the uploads of the last test.

The regenerated summary above has no actionable items.

@claude claude Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code review found no issues

No high-confidence issues detected in this change.

1200 resets pass a bucket of 1000 only if it regained 200 tokens, which
takes 7 refills of 33. The comment said 199 tokens and 7 s at 33 per
second.

@claude claude Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code review found no issues

No high-confidence issues detected in this change.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant