Skip to content

crash_handler: report Rust panics through crash_handler() and test the panic hook - #38927

Closed
robobun wants to merge 4 commits into
mainfrom
farm/de3eac1a/rust-panic-hook-tests
Closed

robobun wants to merge 4 commits into
mainfrom
farm/de3eac1a/rust-panic-hook-tests

Conversation

@robobun

@robobun robobun commented Aug 15, 2026 •

Copy link
Copy Markdown
Collaborator

Problem

  • Every Rust panic! in the binary (unwrap() on None/Err, failed assert!, unreachable!()) is reported by rust_panic_hook in src/crash_handler/lib.rs, which printed its own hand-copied crash report instead of calling crash_handler() like panic_impl, Bun__crashHandler, OOM and the signal handlers do.
  • The copy had drifted from crash_handler():
    • it printed panic: <msg> instead of panic(main thread): <msg>; scripts/runner.node.mjs:1657 extracts crash labels with /panic\(.*\): (.*)/, so a real Rust panic in CI lost its message while a panic_impl crash kept it;
    • it skipped the PANICKING count (so Global::exit() on another thread could cut the report short), HAS_PRINTED_MESSAGE, Output::source::stdio::restore(), the native-plugin / unsupported-uv-function notes, reset_segfault_handler() and the crash auto-reload;
    • a payload longer than its 1024-byte buffer was reported as an empty message (BoundedArray appends are all-or-nothing).
  • Nothing tested the hook: the crash_handler.panic() helper in bun:internal-for-testing calls panic_impl directly (src/runtime/api/crash_handler_jsc.rs), so run-crash-handler.test.ts and crash-report-command-char.test.ts only covered panic_impl. (Item 12 of the review of Rewrite Bun in Rust #30412.)

Fix

  • rust_panic_hook now takes the payload string (PanicHookInfo::payload_as_str, the same &'static str / String pair the old code downcast by hand), cuts it at MAX_MESSAGE_BYTES (1024) on a char boundary, and calls crash_handler(CrashReason::Panic(msg), TraceSeed::BeginAddr(return_address())). The duplicated report code is deleted.
  • CrashReason gets a lifetime (CrashReason<'a>). A reason is only formatted and passed down to crash_handler()/crash(), never stored (every use of the type is in lib.rs; the only constructors outside it are the unit/usize variants in crash_handler_jsc.rs), so the four sites that laundered a message to &'static with unsafe { detach_lifetime(..) } (panic_impl, the hook, Bun__crashHandler, cold_handle_error_return_trace) become plain constructors and the hook contains no unsafe. Same shape as the existing TraceSeed<'a>.
  • Why this is right:
    • this is exactly what panic_impl does with its message, so a Rust panic gets the panic_impl report, upload, multi-thread handling and SIGABRT by construction;
    • re-entrancy: std aborts a panic raised inside the hook itself, so the hook's own guard was only reachable for a panic raised while crash_handler() was already printing a signal/OOM report, and crash_handler()'s PANIC_STAGE branches already handle that case;
    • return_address() is read in the hook's own frame, so the trace is trimmed where the old code trimmed it (verified below: the trace starts at the hook's caller in std and reaches the panicking function);
    • 1024 bytes is the bound the old buffer already imposed; over it the message is now truncated instead of dropped, and every probed message shape still yields a trace string that decodes (table below).
  • Test helpers, named like their neighbours: crash_handler.rustPanic() is a real panic! (a literal, so a &'static str payload; rustPanic(message) panics with the given message, a String payload) and crash_handler.rustUnwrap() is a real Result::unwrap() on an Err (std formats it, also a String payload). panic() remains the panic_impl case. No-argument rustPanic() has the same name and message as the helper Restore BUN_DUMP_STATE_ON_CRASH: dump the DevServer graph when bun crashes #38617 adds for its own test, so the two PRs rebase onto each other in either order (Restore BUN_DUMP_STATE_ON_CRASH: dump the DevServer graph when bun crashes #38617 also adds a call to the old hook body, which becomes unnecessary once the hook delegates).
  • Tests in test/cli/run/run-crash-handler.test.ts:
    • panic, rustPanic and rustUnwrap held to the same assertions: report header, exact panic(main thread): <message> line, the oh no: Bun has crashed line, and the message inflated back out of the trace string (all platforms); SIGABRT (POSIX table, which now carries the expected message per row); upload of the report with the same decode (reporter tests); on debug Linux, a symbolized trace containing js_rust_panic / js_rust_unwrap and no capture machinery.
    • The cap: a 1024-byte message is reported whole, a longer one is cut after exactly 1024 bytes, and a two-byte char straddling the cut is dropped rather than split, checked on both the stderr line and the trace string.
    • fixture-crash.js exits 1 on an unknown approach; the reporter test asserts on the report before awaiting the upload so a missing report fails fast; the stale note in crash-report-command-char.test.ts saying run-crash-handler.test.ts is skip-listed is removed (test: stop quarantining whole files for one broken case #33952 un-skipped it).
  • Fail-before: on main the new cases fail because the helpers do not exist. They also fail against the old hook body with the helpers built in: 4 report-format cases fail on panic: ... lacking the thread tag (output below), and the over-cap cases would see an empty message.
  • Verified: bun bd test test/cli/run/run-crash-handler.test.ts test/cli/run/crash-report-command-char.test.ts is 32 pass, 9 skip (Windows/macOS/non-ASAN-only cases), 0 fail; cargo clippy on bun_crash_handler and bun_runtime, cargo fmt --check, cargo check of the crate for x86_64-pc-windows-msvc and aarch64-apple-darwin (the Windows classifier returns CrashReason<'static>), and test/internal/source-lints/ are clean.
  • Not changed here: encode_trace_string drops the message from the trace string when its compressed form does not fit the 1024-byte trace_str_buf (about 800+ bytes of high-entropy text). That affects panic_impl identically and is handed off separately.

Background

  • Crash report and trace string: crash_handler() prints a header (version, kernel, CPU, args), a panic(<thread>): <reason> line, then either a symbolized backtrace (debug builds) or a trace string: a URL under bun.report (or BUN_CRASH_REPORT_URL) encoding the version, platform, running command, frame addresses and the crash reason. For CrashReason::Panic the reason is the tag 0 followed by the zlib-compressed, base64 message, which bun.report inflates to show the message; the tests inflate it the same way. With reporting enabled the URL is also POSTed by report(); the tests observe that with a local Bun.serve.
  • panic_impl vs the std panic hook: panic_impl is the explicit entry point (the panic() test helper, CrashHandler__unsupportedUVFunction) and forwards to crash_handler(). Rust's panic! machinery instead calls whatever std::panic::set_hook installed. Bun builds with panic = "abort", so the hook is all that runs for a panic, and every .unwrap() / assert! in production goes through it.
  • Panic payloads: panic!("literal") and Option::unwrap() hand the hook a &'static str; panic! with arguments and Result::unwrap() (std formats "{msg}: {err:?}") hand it a String. payload_as_str() covers both; anything else only comes from panic_any. The payload lives in the std panic frames below the hook, which is why borrowing it for the (never returning) crash_handler() call is fine.
  • Trim anchor: the frame-pointer walk drops every frame above the return address passed as TraceSeed::BeginAddr, so that address has to be read in a frame that is still live when the walk runs (see the comment in panic_impl). Reading it in the hook makes the trace start at the hook's caller.
Report-format tests against the old hook body, helpers built in
error: expect(received).toContain(expected)
Expected to contain: "panic(main thread): invoked crashByRustPanic() handler\n"
Received: "...\n\n\npanic: invoked crashByRustPanic() handler\noh no: Bun has crashed. ..."
(fail) panics through panic_impl and through the std panic hook print the same report > rustPanic
(fail) panics through panic_impl and through the std panic hook print the same report > rustUnwrap
(fail) Rust panic trace includes the panic site > rustPanic
(fail) Rust panic trace includes the panic site > rustUnwrap
 22 pass, 4 fail

(Run before the cap tests and the trace-string decode were added.)

Symbolized trace of a real unwrap panic with the fix (debug build)
<bun_crash_handler::draft::rust_panic_hook as core::ops::function::Fn<(&std::panic::PanicHookInfo,)>>::call
<alloc::boxed::Box<dyn Fn(&PanicHookInfo)> as Fn<...>>::call
std::panicking::panic_with_hook
std::panicking::panic_handler::{closure#0}
std::sys::backtrace::__rust_end_short_backtrace
__rustc::rust_begin_unwind
core::panicking::panic_fmt
core::result::unwrap_failed
<core::result::Result<(), &str>>::unwrap
bun_runtime::api::crash_handler_jsc::js_bindings::js_rust_unwrap        src/runtime/api/crash_handler_jsc.rs
bun_runtime::api::crash_handler_jsc::js_bindings::__jsc_host_js_rust_unwrap::{closure#0}
bun_jsc::host_fn::to_js_host_call
bun_runtime::api::crash_handler_jsc::js_bindings::__jsc_host_js_rust_unwrap
llint_op_call_ignore_result
Long-message probe behind the 1024-byte cap
message                       stderr line   trace string   message decoded from trace string
repetitive text, 1024 B       1024 B        387 chars      1024 B
repetitive text, 5000 B       1024 B (cut)  387 chars      1024 B
hex, 1024 chars               1024 B        945 chars      1024 B
hex, 3000 chars               1024 B (cut)  944 chars      1024 B
"é" x 600 (1200 B)            1024 B (cut)  217 chars      1024 B, cut on a char boundary
base64-like, 700 chars        747 B         944 chars      747 B
base64-like, 800+ chars       full          133 chars      none: pre-existing encoder limit, handed off

The first three rows of this table are what the new cap tests pin (whole at 1024, cut after 1024, char boundary).

…e panic hook

rust_panic_hook printed its own copy of the crash report instead of calling
crash_handler() like panic_impl, Bun__crashHandler and the signal handlers
do. The copy had drifted: it printed "panic: <msg>" without the thread tag
that crash_handler() prints (and that scripts/runner.node.mjs matches to
label a crash), did not count the thread in PANICKING, did not restore
stdio, did not reset the fault handlers, and did not honor the crash
auto-reload. It also reported an empty message when the payload exceeded
its 1024-byte buffer. The hook now extracts the payload, caps it at 1024
bytes and calls crash_handler().

Nothing exercised the hook: the crash_handler.panic() test helper calls
panic_impl directly. Add rustPanic() (a real panic!, &'static str payload)
and rustUnwrap() (a real Result::unwrap() on an Err, String payload) and
run them through the same assertions as panic(): report header and
"panic(main thread): <message>" line, SIGABRT, upload of a trace string
whose panic message decodes to the message, and (debug Linux) a trace that
contains the panic site and not the crash handler's own frames.
@coderabbitai

coderabbitai Bot commented Aug 15, 2026 •

Copy link
Copy Markdown
Contributor

Review Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: ASSERTIVE

Plan: Pro

Run ID: 729b3db8-f457-4a8b-879a-b945471a8ef0

📥 Commits

Reviewing files that changed from the base of the PR and between 7d276b9 and 5a6acac.

📒 Files selected for processing (6)
  • src/crash_handler/lib.rs
  • src/js/internal-for-testing.ts
  • src/runtime/api/crash_handler_jsc.rs
  • test/cli/run/crash-report-command-char.test.ts
  • test/cli/run/fixture-crash.js
  • test/cli/run/run-crash-handler.test.ts

Walkthrough

Walkthrough

The crash handler now carries borrowed panic and error messages. The JavaScript test API can trigger Rust panics and unwrap failures. Crash-handler tests validate message decoding, truncation, stack traces, signals, Windows behavior, and report uploads.

Changes

Crash handling

Layer / File(s) Summary
Borrowed crash reasons and panic delegation
src/crash_handler/lib.rs
CrashReason and related crash-processing paths now accept borrowed message slices. The panic hook extracts, truncates, and delegates messages to crash_handler.
Rust panic test bindings
src/js/internal-for-testing.ts, src/runtime/api/crash_handler_jsc.rs
The testing API exposes rustPanic and rustUnwrap, which trigger Rust panic paths.
Crash report validation
test/cli/run/run-crash-handler.test.ts, test/cli/run/fixture-crash.js, test/cli/run/crash-report-command-char.test.ts
Tests use shared crash fixtures and validate Rust panic messages, UTF-8-safe truncation, stack traces, signals, Windows behavior, and report uploads.

Possibly related PRs

  • oven-sh/bun#38617: Both PRs modify Rust panic/crash handling and the rustPanic test binding.

Suggested reviewers: jarred-sumner

🚥 Pre-merge checks | ✅ 4
✅ Passed checks (4 passed)
Check name Status Explanation
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
Title check ✅ Passed The title clearly summarizes the main change: routing Rust panics through crash_handler() and testing the panic hook.
Description check ✅ Passed The description explains the problem, fix, testing scope, and verification results, although it uses different headings from the template.

Comment @coderabbitai help to get the list of available commands.

@robobun

robobun commented Aug 15, 2026 •

Copy link
Copy Markdown
Collaborator Author

Status: fixed, reviewed, ready for a maintainer.

  • Reproduced on main: rust_panic_hook prints panic: <msg> (no thread tag, none of crash_handler()'s bookkeeping) and nothing in test/ reaches it; the panic() helper calls panic_impl. Building the new rustPanic/rustUnwrap helpers against the old hook body fails the new report-format cases on the missing (main thread) tag.
  • Fix: the hook delegates to crash_handler(); CrashReason<'a> replaces the four detach_lifetime sites; the hook's 1024-byte cap is a named constant with boundary tests driven through rustPanic(message). Locally bun bd test test/cli/run/run-crash-handler.test.ts test/cli/run/crash-report-command-char.test.ts is 32 pass, 9 skip, 0 fail.
  • CI: build 98071 (the current head) has no failing jobs: 177 of 179 lanes passed, including the new tests on every platform that ran; the remaining two are the darwin 14 aarch64 test lanes, which have been waiting for an agent for hours, and the only red is in the flaky bucket (passed on retry). The two earlier builds each failed on one unrelated ASAN-lane test (test-http-chunk-problem.js use-after-free, require-cache.test.ts timeouts), both reported separately.
  • Related: Restore BUN_DUMP_STATE_ON_CRASH: dump the DevServer graph when bun crashes #38617 adds the same no-argument rustPanic helper (same name and message); the two rebase onto each other in either order.

@claude claude Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I reviewed this PR and didn't find any bugs. Because it changes the code path that every production Rust panic!/unwrap()/assert! goes through, a human look at the re-entrancy and lifetime reasoning would still be worthwhile.

What was reviewed:

  • rust_panic_hook now delegates to crash_handler() exactly as panic_impl and Bun__crashHandler already do; the detach_lifetime on the payload borrow mirrors those sites and crash_handler() is -> !.
  • Checked that crash_handler()'s own PANIC_STAGE staging (0 / 1|2 / 3 / _) covers the re-entrancy case the deleted hook-local guard handled; std aborts panic-in-hook so the only reachable re-entry is a panic during an in-progress signal/OOM report, which stage 1+ handles.
  • floor_char_boundary(1024) keeps the slice on a UTF-8 boundary; payload_as_str() covers both the &'static str and String payload shapes the old downcasts handled.
  • New tests hold panic/rustPanic/rustUnwrap to identical assertions (header, panic(main thread): line, SIGABRT, uploaded trace-string round-trip); fixture-crash.js now exits 1 on unknown approach so a missing helper fails instead of passing silently.
Extended reasoning...

Overview

This PR replaces the hand-copied crash-report body inside rust_panic_hook (the std::panic::set_hook handler) with a call to the shared crash_handler(), deleting ~130 lines of drifted duplication. It adds two bun:internal-for-testing helpers (rustPanic, rustUnwrap) that trigger a real panic! and a real Result::unwrap() on Err, and extends run-crash-handler.test.ts to hold all three panic entry points to the same report format, terminal signal, upload, and trace-string-encoded message. It also fixes fixture-crash.js to exit 1 on an unknown approach and updates a stale comment in crash-report-command-char.test.ts.

Security risks

None identified. The crash handler is diagnostic-only; the change routes panic reporting through an existing, more-complete path rather than adding new surface. The unsafe { detach_lifetime(...) } on the panic payload is the established idiom already used at three other CrashReason::Panic construction sites (panic_impl, Bun__crashHandler), and crash_handler() is -> ! so the borrow cannot outlive the payload's owning frames.

Level of scrutiny

High. rust_panic_hook is what runs for every .unwrap(), assert!, and unreachable!() in the production binary (panic = "abort", so the hook is the whole story). A regression here would affect every crash report Bun users see and upload. The change is a simplification that unifies with the already-tested panic_impl path, but the re-entrancy argument (dropping the hook-local PANIC_STAGE guard in favour of crash_handler()'s own staging, relying on std's abort-on-panic-in-hook) and the payload-lifetime soundness in the hook context are subtle enough that a maintainer should confirm them.

Other factors

  • The PR description is unusually thorough: it explains why each piece of the deleted code is subsumed (PANICKING count, HAS_PRINTED_MESSAGE, reset_segfault_handler(), auto-reload), documents the fail-before behaviour against both main and old-hook-with-new-helpers, and includes a probe table for the 1024-byte cap across message shapes.
  • Test coverage is comprehensive: both payload types, all-platforms report-format check, POSIX signal check, debug-Linux symbolized-trace check, and a trace-string decoder that verifies bun.report will show the right message.
  • No prior human reviews on the timeline; only a CodeRabbit rate-limit notice.
  • I verified crash_handler() at src/crash_handler/lib.rs:791 is -> ! and has full PANIC_STAGE staging, and that panic = "abort" is set for all profiles in Cargo.toml.

Deferring because the change, while clean and well-tested, rewires a critical production code path and the re-entrancy/lifetime reasoning deserves a maintainer's sign-off.

…sage cap

CrashReason is only ever formatted and passed down to crash_handler()/crash(),
never stored, so give it a lifetime instead of laundering every message to
&'static with detach_lifetime. This removes the four unsafe blocks that built
a Panic/ZigError reason, including the one in the panic hook, and lets
cold_handle_error_return_trace (which returns normally) borrow its name.

The cap the hook applies to the payload is now a named constant and is
covered: rustPanic(message) panics with the given message, and the tests
check a 1024-byte message is reported whole, a longer one is cut after
exactly 1024 bytes, and the cut never splits a multi-byte char, on both the
stderr line and the message decoded from the trace string. The report-format
tests also decode the trace string for all three panic approaches.
Comment thread src/crash_handler/lib.rs Outdated
Comment thread src/crash_handler/lib.rs Outdated
Comment thread src/crash_handler/lib.rs Outdated
Comment thread src/crash_handler/lib.rs Outdated
Comment thread src/crash_handler/lib.rs Outdated
Comment thread src/js/internal-for-testing.ts Outdated
Comment thread src/runtime/api/crash_handler_jsc.rs Outdated
Comment thread src/runtime/api/crash_handler_jsc.rs Outdated

@claude claude Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I reviewed this PR and didn't find any bugs. Because it rewires the process-wide Rust panic hook and changes CrashReason's lifetime across the crash-handling path — with re-entrancy reasoning that leans on std's panic-in-hook abort and crash_handler()'s own PANIC_STAGE guard — a human look is still worthwhile.

What was reviewed:

  • rust_panic_hook delegating to crash_handler() (-> !): with panic = "abort" no unwind follows the hook, and crash_handler()'s existing PANIC_STAGE match covers the re-entrant cases the deleted guard did.
  • CrashReason<'a>: all four detach_lifetime sites removed safely; the Windows classify_exception_windows path returns CrashReason<'static> so no borrowed data crosses the VEH.
  • floor_char_boundary on the 1024-byte cap keeps the truncated slice valid UTF-8; boundary tests cover exact, over, and split-char cases.
Extended reasoning...

Overview

The PR collapses rust_panic_hook's ~130-line hand-copied crash report into a 10-line delegate to crash_handler(), matching how panic_impl, Bun__crashHandler, OOM, and the signal handlers already enter it. To make the borrowed panic payload flow through without unsafe, CrashReason gains a lifetime parameter and four detach_lifetime sites are deleted. Two test-only host functions (rustPanic, rustUnwrap) are added so the std panic hook is reachable from tests, and run-crash-handler.test.ts gains ~100 lines of new coverage: report-format parity across all three panic entry points, the 1024-byte cap with a char-boundary split, symbolized-trace assertions, terminal-signal table entries, and crash-reporter upload decoding.

Security risks

None identified. The crash handler runs after the process is already terminating; the change removes unsafe rather than adding it. The new rustPanic(message) helper is gated behind bun:internal-for-testing and only reachable in debug builds or CI. No new external inputs reach the crash path.

Level of scrutiny

High. The crash handler is the last-resort diagnostic path for every unwrap(), assert!, and unreachable!() in the binary, and it runs while the process is in an undefined state. A regression here silently loses crash reports or hangs during termination. Two aspects in particular deserve a maintainer's eye: (1) removing the hook's own re-entrancy guard is argued correct because std aborts a panic-inside-hook and crash_handler()'s PANIC_STAGE handles panic-during-signal-report, but that reasoning is subtle enough to warrant confirmation; (2) CrashReason<'a> is a type change that ripples through crash_handler, crash, TraceString, and the Windows exception classifier — the author cross-checked x86_64-pc-windows-msvc and aarch64-apple-darwin builds, but a human should confirm nothing else in the tree constructs or stores a CrashReason.

Other factors

  • The comment-cop bot fired eight times on an earlier revision; commit da30a394 shortened the flagged comments and the current diff's comments are one-liners, so those look addressed.
  • PR #38617 adds an overlapping rustPanic helper; the description says the two rebase in either order, but a maintainer should decide the merge sequence.
  • The PR description is unusually thorough (fail-before output, symbolized traces, cross-target cargo check, a long-message probe table) and the bug-hunting pass found nothing, which raises confidence — but per the approval guidelines, critical-path changes should not be auto-approved regardless of test coverage.

@robobun

robobun commented Aug 15, 2026 •

Copy link
Copy Markdown
Collaborator Author
Updated 11:49 AM PT - Aug 15th, 2026

✅ @robobun, your commit 5a6acacba9ffa10b4ef8fb99a94485f7720cba2e passed in Build #98071! 🎉


🧪   To try this PR locally:

bunx bun-pr 38927

That installs a local version of the PR into your bun-38927 executable, so you can run:

bun-38927 --bun

@robobun

robobun commented Aug 28, 2026

Copy link
Copy Markdown
Collaborator Author

Superseded by #40708: the same change rebased onto the current main (this branch conflicts with main after the dead-code removal in #40610), plus a test that bun --watch restarts after a Rust panic. Closing this one.

@robobun robobun closed this Aug 28, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant