Repository navigation
test: speed up the 28632 MySQL leak test and cover wide prepared-statement rows #40633
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Closed
Closed
Changes from 1 commit
Commits
Show all changes
3 commits
Select commit
Hold shift + click to select a range
File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,48 @@ | ||
| // Child process for 28632.test.ts: re-executes one prepared statement against | ||
| // the wide table the test created and prints the RSS growth as its last line. | ||
| // argv: <mysql url> <column count> | ||
| import { SQL } from "bun"; | ||
|
|
||
| const [url, columnCount] = process.argv.slice(2); | ||
| const COLUMNS = Number(columnCount); | ||
| // The first batches let the JIT tiers, the statement cache and the heap reach | ||
| // a steady state, so the delta measures only what the later queries keep. | ||
| const WARMUP_QUERIES = 100; | ||
| const MEASURED_QUERIES = 300; | ||
| const BATCH = 50; | ||
|
|
||
| const rss: () => number = | ||
| process.platform === "darwin" && typeof Bun.unsafe.memoryFootprint === "function" | ||
| ? (Bun.unsafe.memoryFootprint as () => number) | ||
| : process.memoryUsage.rss; | ||
|
|
||
| await using sql = new SQL({ url, max: 1 }); | ||
|
|
||
| // `.values()` skips the per-row object build, which is not on the | ||
| // column-definition path under test and doubles the debug-build time at 500 | ||
| // columns. Every result is checked so a broken query path fails the test | ||
| // instead of measuring an idle process. | ||
| const query = () => sql`SELECT * FROM leak_test_28632 WHERE primary_id = ${"123"} LIMIT 1`.values(); | ||
| function check(rows: unknown[][]) { | ||
| if (rows.length !== 1 || rows[0].length !== COLUMNS + 1 || rows[0][0] !== "123") { | ||
| throw new Error("unexpected result: " + JSON.stringify(rows).slice(0, 200)); | ||
| } | ||
| } | ||
|
|
||
| // Prepare the statement with a lone query first. A failed prepare rejects here | ||
| // instead of stalling the queued copies behind it. | ||
| check(await query()); | ||
|
|
||
| // The queries in a batch queue on the single connection. Each batch ends with | ||
| // a full GC so RSS reflects native allocations rather than JS garbage. | ||
| async function run(count: number) { | ||
| for (let done = 0; done < count; done += BATCH) { | ||
| for (const rows of await Promise.all(Array.from({ length: BATCH }, query))) check(rows); | ||
| Bun.gc(true); | ||
| } | ||
| } | ||
|
|
||
| await run(WARMUP_QUERIES); | ||
| const before = rss(); | ||
| await run(MEASURED_QUERIES); | ||
| console.log(JSON.stringify({ deltaMiB: (rss() - before) / 1024 / 1024 })); |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,87 +1,60 @@ | ||
| // https://github.com/oven-sh/bun/issues/28632 | ||
| import { SQL } from "bun"; | ||
| import { beforeAll, expect, test } from "bun:test"; | ||
| import { describeWithContainer, isASAN, isDebug, isDockerEnabled, rss } from "harness"; | ||
|
|
||
| if (isDockerEnabled()) { | ||
| describeWithContainer( | ||
| "issue #28632: MySQL adapter should not leak memory on repeated queries", | ||
| { | ||
| image: "mysql_plain", | ||
| concurrent: true, | ||
| }, | ||
| container => { | ||
| let sql: SQL; | ||
|
|
||
| beforeAll(async () => { | ||
| await container.ready; | ||
| sql = new SQL({ | ||
| url: `mysql://root@${container.host}:${container.port}/bun_sql_test`, | ||
| max: 1, | ||
| }); | ||
| }); | ||
|
|
||
| test("prepared statement re-execution should not leak name_or_index", async () => { | ||
| // Create a wide table to amplify the per-column leak signal | ||
| await sql`DROP TABLE IF EXISTS leak_test_28632`; | ||
| await sql`CREATE TABLE leak_test_28632 ( | ||
| primary_id VARCHAR(255) PRIMARY KEY, | ||
| column_alpha_bravo TEXT, column_charlie_delta TEXT, column_echo_foxtrot TEXT, | ||
| column_golf_hotel TEXT, column_india_juliet TEXT, column_kilo_lima TEXT, | ||
| column_mike_november TEXT, column_oscar_papa TEXT, column_quebec_romeo TEXT, | ||
| column_sierra_tango TEXT, column_uniform_victor TEXT, column_whiskey_xray TEXT, | ||
| column_yankee_zulu TEXT, column_one_two_three TEXT, column_four_five_six TEXT, | ||
| column_seven_eight TEXT, column_nine_ten TEXT, column_eleven_twelve TEXT, | ||
| column_thirteen_fourtn TEXT, column_fifteen_sixtn TEXT, column_seventeen TEXT, | ||
| column_eighteen TEXT, column_nineteen TEXT, column_twenty_extra TEXT, | ||
| column_twentyone TEXT, column_twentytwo TEXT, column_twentythree TEXT, | ||
| column_twentyfour TEXT, column_twentyfive TEXT, column_twentysix TEXT, | ||
| column_twentyseven TEXT, column_twentyeight TEXT, column_twentynine TEXT, | ||
| column_thirty_extra TEXT, column_thirtyone TEXT, column_thirtytwo TEXT, | ||
| column_thirtythree TEXT, column_thirtyfour TEXT, column_thirtyfive TEXT, | ||
| column_thirtysix TEXT, column_thirtyseven TEXT, column_thirtyeight TEXT, | ||
| column_thirtynine TEXT, column_forty_extra TEXT, column_fortyone TEXT, | ||
| column_fortytwo TEXT, column_fortythree TEXT, column_fortyfour TEXT, | ||
| column_fortyfive TEXT | ||
| )`; | ||
| await sql`INSERT INTO leak_test_28632 (primary_id) VALUES ('123')`; | ||
|
|
||
| // Warm up to stabilize RSS | ||
| for (let i = 0; i < 500; i++) { | ||
| await sql`SELECT * FROM leak_test_28632 WHERE primary_id = ${"123"} LIMIT 1`; | ||
| } | ||
| Bun.gc(true); | ||
| await Bun.sleep(50); | ||
| const rssAfterWarmup = rss(); | ||
|
|
||
| // Run queries — each re-decodes 50 column definitions. Collect every 500 | ||
| // so the RSS delta reflects the name_or_index leak rather than the | ||
| // controller's opportunistic-GC cadence (which no longer fires at this | ||
| // allocation volume): without this, peak uncollected garbage between | ||
| // opportunistic passes commits extra MarkedBlock pages that Bun.gc(true) | ||
| // at the end does not synchronously decommit. | ||
| for (let i = 0; i < 5000; i++) { | ||
| await sql`SELECT * FROM leak_test_28632 WHERE primary_id = ${"123"} LIMIT 1`; | ||
| if (i % 500 === 499) Bun.gc(true); | ||
| } | ||
| Bun.gc(true); | ||
| await Bun.sleep(50); | ||
| const rssAfterQueries = rss(); | ||
|
|
||
| const growthMB = (rssAfterQueries - rssAfterWarmup) / 1024 / 1024; | ||
|
|
||
| // Without the fix, ~17MB growth (50 leaked name_or_index allocs × 5000 queries). | ||
| // With the fix, ~7MB (allocator noise + ASAN shadow memory). Under ASAN the | ||
| // Rust global allocator routes every alloc through the interceptor, so the | ||
| // per-query free/alloc churn (row cells, etc.) lands in ASAN's 256 MB | ||
| // quarantine and shows up as RSS even though nothing leaks — give it 3× | ||
| // headroom there. Debug builds enable ASAN by default on Linux/macOS, so | ||
| // treat them the same. The non-ASAN bound is what guards the actual | ||
| // regression. | ||
| expect(growthMB).toBeLessThan(isASAN || isDebug ? 36 : 12); | ||
|
|
||
| await sql`DROP TABLE IF EXISTS leak_test_28632`.catch(() => {}); | ||
| }); | ||
| }, | ||
| ); | ||
| } | ||
| import { afterAll, beforeAll, expect, test } from "bun:test"; | ||
| import { describeWithContainer, expectRssDeltaBelow } from "harness"; | ||
| import path from "node:path"; | ||
|
|
||
| // The server re-sends one column definition per column with every | ||
| // COM_STMT_EXECUTE response, and the adapter keeps an owned copy of each | ||
| // column name (`name_or_index`). The original bug leaked that copy on every | ||
| // re-decode, so the leak per query scales with the column count and the name | ||
| // length. MySQL caps identifiers at 64 bytes; 500 columns stay under MariaDB's | ||
| // table-definition size limit. | ||
| const COLUMNS = 500; | ||
| const columns = Array.from({ length: COLUMNS }, (_, i) => `col_${i}_`.padEnd(64, "x")); | ||
| const fixture = path.join(import.meta.dir, "28632.fixture.ts"); | ||
|
|
||
| describeWithContainer( | ||
| "issue #28632: MySQL adapter should not leak memory on repeated queries", | ||
| { | ||
| image: "mysql_plain", | ||
| concurrent: true, | ||
| }, | ||
| container => { | ||
| let url: string; | ||
|
|
||
| beforeAll(async () => { | ||
| await container.ready; | ||
| url = `mysql://root@${container.host}:${container.port}/bun_sql_test`; | ||
| await using sql = new SQL({ url, max: 1 }); | ||
| await sql.unsafe(`DROP TABLE IF EXISTS leak_test_28632`); | ||
| await sql.unsafe( | ||
| `CREATE TABLE leak_test_28632 (primary_id VARCHAR(255) PRIMARY KEY, ${columns.map(name => `${name} TINYINT`).join(", ")})`, | ||
| ); | ||
| await sql`INSERT INTO leak_test_28632 (primary_id) VALUES (${"123"})`; | ||
| }); | ||
|
|
||
| afterAll(async () => { | ||
| await using sql = new SQL({ url, max: 1 }); | ||
| await sql.unsafe(`DROP TABLE IF EXISTS leak_test_28632`); | ||
| }); | ||
|
|
||
| test("SELECT * returns the row with every column", async () => { | ||
| await using sql = new SQL({ url, max: 1 }); | ||
| const objects = await sql`SELECT * FROM leak_test_28632 WHERE primary_id = ${"123"} LIMIT 1`; | ||
| expect(objects).toEqual([{ primary_id: "123", ...Object.fromEntries(columns.map(name => [name, null])) }]); | ||
| const values = await sql`SELECT * FROM leak_test_28632 WHERE primary_id = ${"123"} LIMIT 1`.values(); | ||
| expect(values).toEqual([["123", ...new Array(COLUMNS).fill(null)]]); | ||
| }); | ||
|
|
||
| test("prepared statement re-execution should not leak name_or_index", async () => { | ||
| // Unfixed: one 64-byte block leaks per column per query, so the | ||
| // fixture's 300 measured queries over 500 columns leak 150,000 blocks: | ||
| // at least 9.6 MiB under mimalloc (release), and 15 to 17 MiB measured | ||
| // under ASAN, where each block also carries a header and redzone. | ||
| // Fixed: 0 to 1 MiB (release) and 0 to 3 MiB (debug/ASAN, quarantine | ||
| // off) after the fixture's 100-query warm-up. | ||
| await expectRssDeltaBelow([fixture, url, String(COLUMNS)], { release: 5, debug: 8 }); | ||
| }); | ||
| }, | ||
| ); | ||
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.