Skip to content

test(net): measure mongodb-pattern RSS at steady state, not during JIT tier-up - #34752

Closed
robobun wants to merge 2 commits into
mainfrom
farm/9636f27c/net-mongodb-leak-warmup
Closed

robobun wants to merge 2 commits into
mainfrom
farm/9636f27c/net-mongodb-leak-warmup

Conversation

@robobun

@robobun robobun commented Jul 20, 2026 •

Copy link
Copy Markdown
Collaborator

What

net-mongodb-pattern-leak.test.ts started failing its 8 MB release RSS delta check on ~40% of PR builds after #33133 (WebKit upgrade, merged July 1). Sampling Buildkite annotations: 1/29 builds flaked in the week before #33133, 15/37 in the days after. Observed deltas 8.4-12.3 MB.

Why this is not a memory regression

Five-round instrumented runs on current main:

metric r1 r2 r3 r4 r5
heapSize 2.06 MB 2.06 MB 2.06 MB 2.06 MB 2.06 MB
Timeout / Promise / AsyncGenerator / Function delta 0 0 0 0 0
RSS (one run) 68.3 MB 69.1 MB 66.9 MB 63.6 MB 64.6 MB

heapSize and every tracked object count are flat; RSS in rounds 3-5 goes down. With BUN_JSC_useJIT=0 total RSS drops from ~65 MB to ~33 MB and the r2-r1 delta is <2 MB on every run. The extra RSS is JIT code pages: the newer JSC's tier-up thresholds put more DFG/FTL compilation into round 2, where the old build had finished it in round 1.

Over 30 local release runs the r2-r1 delta ranged -0.0..+11.0 MB (one would have failed the 8 MB bound), while r3-r2 ranged -2.2..+2.0 MB.

Change

  • Run one unmeasured warmup round before the two measured rounds, so the RSS comparison happens at steady state. The 8 MB release bound is unchanged.
  • Add a heapSize delta assertion (<1 MB), which is the precise signal for JS-heap retention.
  • Drop debug/ASAN ITER 2000 -> 1500 so total work (3x1500) stays near the previous 2x2000 under the existing 60 s timeout.

Verification

  • Release: 0/50 runs failed locally (was ~1/10 on the unmodified test under load).
  • Debug+ASAN: 3/3 passed, 28-45 s per case.
  • Simulated leak (listeners not removed in onData.close): Function count 15000 > 20, both cases fail as expected.

vs #33725

#33725 widens the release bound 8 MB -> 24 MB. That also stops the flake, but loosens the native-leak backstop. This keeps the bound where it was by measuring after JIT tier-up has settled, and adds the heapSize check the original test was missing. The node-net.test.ts await fix in #33725 is independent and still worth taking.


[stamp-90s] gate passed · iteration 0 · 1 files touched

passes on PR (with fix)
Test-only change.

Debug/ASAN (expected pass):
$ bun bd test 'test/js/node/net/net-mongodb-pattern-leak.test.ts'
$ BUN_DEBUG_QUIET_LOGS=1 bun scripts/build.ts --profile=debug --quiet test test/js/node/net/net-mongodb-pattern-leak.test.ts
info: syncing channel updates for nightly-2026-05-06-x86_64-unknown-linux-gnu
info: latest update on 2026-05-06 for version 1.97.0-nightly (e95e73209 2026-05-05)
info: component rust-src is up to date
info: checking for self-update (current version: 1.29.0)
bun test v1.4.0 (e038b51dd)

test/js/node/net/net-mongodb-pattern-leak.test.ts:
(pass) mongodb driver pattern over tcp does not leak > long-lived connection: framed command round-trips [31949.62ms]
(pass) mongodb driver pattern over tls does not leak > long-lived connection: framed command round-trips [34195.90ms]

 2 pass
 0 fail
 22 expect() calls
Ran 2 tests across 1 file. [68.61s]
Exit: 0
diff hotspot
test/js/node/net/net-mongodb-pattern-leak.test.ts | 18 ++++++++++++------
 1 file changed, 12 insertions(+), 6 deletions(-)

gate history · 1 passed · 0 rejected · iteration 0

evidence per changed file
file                                               reads  edits  tests
test/js/node/net/net-mongodb-pattern-leak.test.ts      4      6      0

…T tier-up

The WebKit upgrade in #33133 shifted JSC tier-up timing so DFG/FTL
compilation for this hot loop now often lands in round 2 instead of
finishing in round 1. That inflated the round-2 vs round-1 RSS delta
from <8 MB to 8-13 MB on ~40% of release CI runs since build 67387.

This is not a retention regression: heapSize is flat at ~2 MB across
five rounds, every tracked object count (Timeout, Promise,
AsyncGenerator, Function) is delta=0, and rounds 3-5 show RSS going
down. With BUN_JSC_useJIT=0 the round-2 delta is <2 MB and process RSS
drops from ~65 MB to ~33 MB, so the extra RSS is JIT code pages.

Run one unmeasured warmup round before the two measured rounds so the
8 MB release bound holds at steady state (0/50 local runs; r3-r2 ranged
-2.2..+2.0 MB over 30 probes vs -0.0..+11.0 MB for r2-r1). Also assert
heapSize itself is flat, and drop debug/ASAN ITER to 1500 so total work
(3x1500) stays close to the previous 2x2000.
@robobun

robobun commented Jul 20, 2026 •

Copy link
Copy Markdown
Collaborator Author
Updated 8:07 PM PT - Jul 19th, 2026

❌ @robobun, your commit e038b51 has 1 failures in Build #75932 (All Failures):


🧪   To try this PR locally:

bunx bun-pr 34752

That installs a local version of the PR into your bun-34752 executable, so you can run:

bun-34752 --bun

@coderabbitai

coderabbitai Bot commented Jul 20, 2026 •

Copy link
Copy Markdown
Contributor

Review Change Stack

Walkthrough

Changes

MongoDB pattern leak test

Layer / File(s) Summary
Heap measurement and leak assertions
test/js/node/net/net-mongodb-pattern-leak.test.ts
Snapshots now include heapSize; debug/ASAN iterations are reduced; an explicit warmup precedes measurements; and assertions verify heap-size stability alongside object-count and Transform listener checks.
🚥 Pre-merge checks | ✅ 4
✅ Passed checks (4 passed)
Check name Status Explanation
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
Title check ✅ Passed The title clearly matches the main change: measuring mongodb-pattern RSS after JIT tier-up settles.
Description check ✅ Passed The PR description covers what changed and how it was verified, which satisfies the template despite using different section headings.

Comment @coderabbitai help to get the list of available commands.

@robobun

robobun commented Jul 20, 2026 •

Copy link
Copy Markdown
Collaborator Author

Build 75932 (e038b51): net-mongodb-pattern-leak.test.ts green on every completed lane (linux glibc/musl x64/aarch64/baseline, ASAN, Windows x64/x64-baseline/aarch64). The only hard-red is test-net-connect-memleak.js on alpine x64-baseline, which fails on 15/15 recent PR builds and is being handled in #34176 / #34631; the rest of the annotation is retry-passed flakes (repl/spawn/cron/s3/http leak tests, none touched here). Same picture as 75914 on the previous commit. Ready for a maintainer.

heapStats().heapSize is vm.heap.size() (live-cell bytes after a sync
full GC), not subject to the bmalloc scavenger which only decommits OS
pages. Measured r2-r1 delta across 30 release + 3 debug/ASAN runs:
-38..+8 KB; the simulated listener leak at ITER=1500 produces ~56 MB.
256 KB sits 32x above the observed max and 220x below the leak.
@robobun

robobun commented Jul 20, 2026

Copy link
Copy Markdown
Collaborator Author

Self-review caught that the heapSize comment was wrong (the bmalloc scavenger decommits OS pages and affects RSS, not vm.heap.size()) and the 1 MB margin wasn't derived from data. Measured the actual r2-r1 heapSize delta: -38..+8 KB over 30 release + 3 debug/ASAN runs; the simulated listener leak at ITER=1500 produces ~56 MB. Tightened to 256 KB in e038b51 (32x the observed max, 220x below the leak).

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@test/js/node/net/net-mongodb-pattern-leak.test.ts`:
- Around line 202-203: Update the heap measurement reference in the test to use
heapStats().heapSize instead of vm.heap.size(), and remove the run-specific
measurement ranges and leak-size figures from the surrounding comment.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: ASSERTIVE

Plan: Pro

Run ID: 896e91b9-0f01-4c29-a837-73684aaa431a

📥 Commits

Reviewing files that changed from the base of the PR and between 0a17ce6 and e038b51.

📒 Files selected for processing (1)
  • test/js/node/net/net-mongodb-pattern-leak.test.ts

Comment thread test/js/node/net/net-mongodb-pattern-leak.test.ts

@claude claude Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM — de-flake that strengthens rather than weakens the leak test.

What was reviewed:

  • RSS bound (8 MB release / 32 MB debug) and all object-count / listener assertions are unchanged; only the measurement window moves past JIT tier-up.
  • New heapSize delta check: 256 KB bound is data-derived (32× observed max, ~220× below the simulated-leak signal) and branches correctly on the same snapshot after Bun.gc(true).
  • Debug/ASAN ITER 2000→1500 keeps total iterations (3×1500) at or above the previous 2×2000, so leak sensitivity isn't reduced; release ITER unchanged.
Extended reasoning...

Overview

Test-only change to test/js/node/net/net-mongodb-pattern-leak.test.ts (12 insertions, 6 deletions). Adds an unmeasured warmup round before the two measured rounds so the RSS delta is taken at steady state instead of during JIT tier-up, adds a heapStats().heapSize delta assertion, and reduces debug/ASAN per-round iterations to keep total wall-clock under the existing 60 s timeout now that there are three rounds.

Security risks

None. No production code is touched; the test uses a local in-process TCP/TLS echo server on port: 0 with rejectUnauthorized: false against the harness's self-signed cert, all pre-existing.

Level of scrutiny

Low-to-moderate. The main risk with de-flake PRs is silently weakening the safety net (REVIEW.md: "When de-flaking, keep asserting the property the original assertion protected"). This PR does the opposite: the 8 MB release RSS bound is kept, every existing object-count and listener assertion is preserved, and a strictly tighter heapSize check is added. The PR description shows the empirical basis (JIT-off run halves RSS and eliminates the r2−r1 delta; r3−r2 range −2.2..+2.0 MB over 30 runs; simulated listener leak still fails on Function count and heapSize). The competing approach in #33725 (widen 8→24 MB) is explicitly rejected in favor of preserving the bound.

Other factors

  • Debug/ASAN ITER drop 2000→1500 is compensated by the extra round (3×1500 = 4500 ≥ 2×2000 = 4000 total iterations), and per-round deltas are what's asserted, so per-round leak sensitivity is 1500/2000 = 75% of before on debug/ASAN — but the new 256 KB heapSize check is far more sensitive than the object-count thresholds it augments, so net coverage improves. Release ITER is unchanged at 5000.
  • CI: green on all completed lanes per the build-status comment; the one unrelated red (test-net-connect-memleak.js on alpine) is failing on main and tracked separately.
  • The only inline review comment (CodeRabbit, about the vm.heap.size() reference in a comment) was resolved and withdrawn; the code correctly uses heapStats().heapSize.
  • No prior claude[bot] reviews on this PR.

@github-actions

Copy link
Copy Markdown
Contributor

Found 3 issues this PR may fix:

  1. CI: test-net-connect-memleak.js fails on half of PR builds on linux-x64-musl since June 28 ~23:00 UTC #33044 - Same class of flaky CI memory leak test (test-net-connect-memleak.js on musl) caused by GC/runtime timing, not an actual leak
  2. serve-body-leak.test.ts absolute 512MB end-memory cap has no headroom under ASAN, flakes on the x64-asan lane #32179 - Same class of flaky leak test (serve-body-leak.test.ts) where ASAN overhead inflates RSS past the test's cap
  3. Bun appears to delay GC in low-pressure Fastify + mongodb workload, causing slow RSS/Timeout growth #30261 - Directly about the MongoDB workload RSS/GC behavior that net-mongodb-pattern-leak.test.ts guards against

If this is helpful, copy the block below into the PR description to auto-close these issues on merge.

Fixes #33044
Fixes #32179
Fixes #30261

🤖 Generated with Claude Code

@robobun

robobun commented Jul 20, 2026

Copy link
Copy Markdown
Collaborator Author

None of those three apply: this is a test-only change to how net-mongodb-pattern-leak.test.ts measures steady state, with no runtime behavior change. #33044 and #32179 are about different test files (test-net-connect-memleak.js and serve-body-leak.test.ts), and #30261 is about the actual MongoDB workload behavior that this test guards; this PR doesn't change that behavior, it only moves the RSS sample past JIT tier-up. Not adding any Fixes lines.

@github-actions

Copy link
Copy Markdown
Contributor

This PR may be a duplicate of:

  1. test: widen net-mongodb-pattern-leak RSS bound; await the named-pipe GC check #33725 - Also fixes the flaky net-mongodb-pattern-leak RSS test; widens the RSS bound to 24 MB instead of adding a warmup round

🤖 Generated with Claude Code

@robobun

robobun commented Sep 13, 2026

Copy link
Copy Markdown
Collaborator Author

Closing as part of a cleanup of stale pull requests. This PR has had no new commits since 2026-07-20, it conflicts with main, and its last CI run failed. This is not a judgment on the fix itself. If the problem still reproduces on a current build, reopen this PR after a rebase or open a new one against main.

@robobun robobun closed this Sep 13, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant