Skip to content
Open
Show file tree
Hide file tree
Changes from 3 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 0 additions & 1 deletion AGENTS.md

This file was deleted.

348 changes: 348 additions & 0 deletions AGENTS.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,348 @@
This is the Bun repository - an all-in-one JavaScript runtime & toolkit designed for speed, with a bundler, test runner, and Node.js-compatible package manager. It's written primarily in Rust with C++ for JavaScriptCore integration, powered by WebKit's JavaScriptCore engine.

## Building and Running Bun

### Build Commands

- **Build Bun**: `bun bd`
- Creates a debug build at `./build/debug/bun-debug`
- **CRITICAL**: do not set a timeout when running `bun bd`
- **Run tests with your debug build**: `bun bd test <test-file>`
- **CRITICAL**: Never use `bun test` directly - it won't include your changes
- **Run any command with debug build**: `bun bd <command>`
- **Run with JavaScript exception scope verification**: `BUN_JSC_validateExceptionChecks=1
BUN_JSC_dumpSimulatedThrows=1 bun bd <command>`

Tip: Bun is already installed and in $PATH. The `bd` subcommand is a package.json script.

**All build scripts support build-then-exec.** Any `bun run build*` command (and `bun bd`, and `bun scripts/build.ts` directly) accepts trailing args which are passed to the built executable after building. This is the recommended way to run your build — you never invoke `./build/debug/bun-debug` directly.

```sh
bun bd test foo.test.ts # debug build + quiet debug logs
bun run build test foo.test.ts # debug build
bun run build:release -p 'Bun.version' # release build
bun run build:local run script.ts # debug build with local WebKit
```

When exec args are present, build output is suppressed unless the build fails — you see only the binary's output. Build flags (e.g. `--asan=off`) go before the exec args; see `scripts/build.ts` header for the full arg routing rules.

**Comparing builds:** normally use the default `build/<profile>/` dir. If you need to preserve a build as a comparison point (rare — e.g. benchmarking before/after a change), `--build-dir` parks it somewhere the next build won't overwrite:

```sh
bun run build:release --build-dir=build/baseline
```

### Changes that don't require a build

Edits to **TypeScript type declarations** (`packages/bun-types/**/*.d.ts`) do not touch any compiled code, so `bun bd` is unnecessary. The types test just packs the `.d.ts` files and runs `tsc` against fixtures — it never executes your build. Run it directly with the system Bun:

```sh
bun test test/integration/bun-types/bun-types.test.ts
```

This is an explicit exception to the "never use `bun test` directly" rule. There are no native changes for a debug build to pick up, so don't wait on one.

## Testing

### Running Tests

- **Single test file**: `bun bd test test/js/bun/http/serve.test.ts`
- **Fuzzy match test file**: `bun bd test http/serve.test.ts`
- **With filter**: `bun bd test test/js/bun/http/serve.test.ts -t "should handle"`

### Test Organization

**Default: add your test to the existing test file for the code you're changing.** Do not create a new file. A fetch bug goes in `test/js/web/fetch/fetch.test.ts`, a `Bun.serve` bug goes in `test/js/bun/http/serve.test.ts`, and so on. Keeping tests next to related coverage is what makes them discoverable and prevents duplicated setup.

- `test/js/bun/` - Bun-specific API tests (http, crypto, ffi, shell, etc.)
- `test/js/node/` - Node.js compatibility tests
- `test/js/web/` - Web API tests (fetch, WebSocket, streams, etc.)
- `test/cli/` - CLI command tests (install, run, test, etc.)
- `test/bundler/` - Bundler and transpiler tests. Use `itBundled` helper.
- `test/integration/` - End-to-end integration tests
- `test/napi/` - N-API compatibility tests
- `test/v8/` - V8 C++ API compatibility tests

**Exception:** `test/regression/issue/${issueNumber}.test.ts` is reserved for bugs that have a GitHub issue number **and** are true regressions (worked in a previous release, then broke). An issue number alone is not enough — if the behavior was never correct, it's not a regression and the test belongs in the existing file for that module. The issue number must be **REAL**, not a placeholder.

### Writing Tests

Tests use Bun's Jest-compatible test runner with proper test fixtures.

- For **single-file tests**, prefer `-e` over `tempDir`.
- For **multi-file tests**, prefer `tempDir` and `Bun.spawn`.

```typescript
import { test, expect } from "bun:test";
import { bunEnv, bunExe, normalizeBunSnapshot, tempDir } from "harness";

test("(single-file test) my feature", async () => {
await using proc = Bun.spawn({
cmd: [bunExe(), "-e", "console.log('Hello, world!')"],
env: bunEnv,
});

const [stdout, stderr, exitCode] = await Promise.all([
proc.stdout.text(),
proc.stderr.text(),
proc.exited,
]);

expect(normalizeBunSnapshot(stdout)).toMatchInlineSnapshot(`"Hello, world!"`);
expect(exitCode).toBe(0);
});

test("(multi-file test) my feature", async () => {
// Create temp directory with test files
using dir = tempDir("test-prefix", {
"index.js": `import { foo } from "./foo.ts"; foo();`,
"foo.ts": `export function foo() { console.log("foo"); }`,
});

// Spawn Bun process
await using proc = Bun.spawn({
cmd: [bunExe(), "index.js"],
env: bunEnv,
cwd: String(dir),
stderr: "pipe",
});

const [stdout, stderr, exitCode] = await Promise.all([
proc.stdout.text(),
proc.stderr.text(),
proc.exited,
]);

// Prefer snapshot tests over expect(stdout).toBe("hello\n");
expect(normalizeBunSnapshot(stdout, dir)).toMatchInlineSnapshot(`"hello"`);

// Assert the exit code last. This gives you a more useful error message on test failure.
expect(exitCode).toBe(0);
});
```

- Always use `port: 0`. Do not hardcode ports. Do not use your own random port number function.
- Use `normalizeBunSnapshot` to normalize snapshot output of the test.
- NEVER write tests that check for no "panic" or "uncaught exception" or similar in the test output. These tests will never fail in CI.
- Use `tempDir` from `"harness"` to create a temporary directory. **Do not** use `tmpdirSync` or `fs.mkdtempSync` to create temporary directories.
- When spawning processes, tests should expect(stdout).toBe(...) BEFORE expect(exitCode).toBe(0). This gives you a more useful error message on test failure.
- **CRITICAL**: Do not write flaky tests. Do not use `setTimeout` in tests. Instead, `await` the condition to be met. You are not testing the TIME PASSING, you are testing the CONDITION.
- **CRITICAL**: Verify your test fails with `USE_SYSTEM_BUN=1 bun test <file>` and passes with `bun bd test <file>`. Your test is NOT VALID if it passes with `USE_SYSTEM_BUN=1`.

## Code Architecture

### Language Structure

- **Rust code** (`src/**/*.rs`): Core runtime, JavaScript bindings, bundler, package manager. This is what compiles and ships.
- **C++ code** (`src/jsc/bindings/*.cpp`): JavaScriptCore bindings, Web APIs
- **TypeScript** (`src/js/`): Built-in JavaScript modules with special syntax (see JavaScript Modules section)
- **Generated code**: Many `.rs` and `.cpp` files are auto-generated from `.classes.ts` and other sources. The build regenerates them automatically when their inputs change.

You will see `.zig` files alongside many `.rs` files (e.g. `fetch.zig` next to `fetch.rs`). These are the **original Zig implementation, kept only as a porting reference** — they are **not compiled** and **not shipped**. New code goes in `.rs`. When fixing a bug or porting a behavior, the `.zig` sibling is the source of truth for *intended semantics*: read it, then make the `.rs` match. Never add new behavior to a `.zig` file.

### Core Source Organization

The Rust side is a Cargo workspace of ~200 crates rooted at `Cargo.toml`. The key ones:

#### Foundation crates

- `src/bun_core/` - The `bun.*`-namespace foundation: strings/`String` (`string/`), formatting (`fmt.rs`), logging (`output.rs`), feature flags, env vars, allocator helpers
- `src/sys/` - Cross-platform syscall wrappers (`file.rs`, `dir.rs`, `fd.rs`, `Error.rs`, `tmp.rs`) — the `bun.sys` equivalent
- `src/collections/`, `src/threading/`, `src/paths/`, `src/semver/`, `src/sourcemap/` - shared utilities

#### Runtime Core (`src/`)

- `src/bun_bin/` - Cargo entrypoint; produces `libbun_rust.a`, linked into the final binary
- `src/runtime/cli/` - CLI argument parsing and command dispatch
- `src/js_parser/`, `src/js_printer/` - JavaScript/TypeScript parsing and printing (each is its own crate; the lexer is `src/js_parser/lexer.rs`)
- `src/transpiler/` - Wrapper around the parser/printer with sourcemap support
- `src/resolver/` - Module resolution system
- `src/ast/` - AST node types and arena allocation

#### JavaScript Runtime (`src/jsc/` + `src/runtime/`)

- `src/jsc/bindings/` - C++ JavaScriptCore bindings (generated classes from `.classes.ts` + manual bindings)
- `src/jsc/` - Rust-side JSC glue (`VirtualMachine.rs`, `web_worker.rs`, `event_loop.rs`, FFI imports)
- `src/runtime/api/` - Bun-specific JS-visible APIs (`BunObject.rs`, `JSBundler.rs`, `Glob`, `Archive`, …)
- `src/runtime/server/` - `Bun.serve` HTTP/WebSocket server
- `src/runtime/node/` - Node.js compatibility layer (fs, path, process, Buffer, …)
- `src/runtime/crypto/` - WebCrypto + `node:crypto` (`EVP.rs`, `HMAC.rs`, `CryptoHasher.rs`, …)
- `src/runtime/webcore/` - Web API implementations (`fetch.rs`, `streams.rs`, `Blob.rs`, `Response.rs`, `Request.rs`, …)
- `src/event_loop/` - Event loop and task management

#### Build Tools & Package Manager

- `src/bundler/` - JavaScript bundler (tree-shaking, CSS processing, HTML handling)
- `src/install/` - Package manager (`lockfile/`, `npm.rs` registry client, `lifecycle_script_runner.rs`)

#### Other Key Components

- `src/shell/` - Cross-platform shell implementation
- `src/css/` - CSS parser and processor
- `src/http/` - HTTP client + `websocket_client/` (WebSocket, deflate)
- `src/sql/` - SQL database integrations (Postgres, MySQL, SQLite)
- `src/bake/` - Server-side rendering / dev server framework

#### Vendored Dependencies (`vendor/`)

Third-party C/C++ libraries are vendored locally and can be read from disk (these are not git submodules):

- `vendor/boringssl/` - BoringSSL (TLS/crypto)
- `vendor/brotli/` - Brotli compression
- `vendor/cares/` - c-ares (async DNS)
- `vendor/hdrhistogram/` - HdrHistogram (latency tracking)
- `vendor/highway/` - Google Highway (SIMD)
- `vendor/libarchive/` - libarchive (tar/zip)
- `vendor/libdeflate/` - libdeflate (fast deflate)
- `vendor/libuv/` - libuv (Windows event loop)
- `vendor/lolhtml/` - lol-html (HTML rewriter)
- `vendor/lshpack/` - ls-hpack (HTTP/2 HPACK)
- `vendor/lsqpack/` - ls-qpack (HTTP/3 QPACK)
- `vendor/lsquic/` - lsquic (QUIC / HTTP/3)
- `vendor/mimalloc/` - mimalloc (memory allocator)
- `vendor/nodejs/` - Node.js headers (compatibility)
- `vendor/picohttpparser/` - PicoHTTPParser (HTTP parsing)
- `vendor/tinycc/` - TinyCC (FFI JIT compiler, fork: oven-sh/tinycc)
- `vendor/WebKit/` - WebKit/JavaScriptCore (JS engine)
- `vendor/zig/` - Zig toolchain (legacy; not used by the Rust build)
- `vendor/zlib/` - zlib-ng (compression, zlib-compat mode)
- `vendor/zstd/` - Zstandard (compression)

Build configuration for these is in `scripts/build/deps/*.ts`.

### JavaScript Class Implementation (C++)

When implementing JavaScript classes in C++:

1. Create three classes if there's a public constructor:
- `class Foo : public JSC::JSDestructibleObject` (if has C++ fields)
- `class FooPrototype : public JSC::JSNonFinalObject`
- `class FooConstructor : public JSC::InternalFunction`

2. Define properties using HashTableValue arrays
3. Add iso subspaces for classes with C++ fields
4. Cache structures in `ZigGlobalObject`

### Code Generation

Code generation happens automatically as part of the build process. The main scripts are:

- `src/codegen/generate-classes.ts` - Generates Rust & C++ bindings from `*.classes.ts` files
- `src/codegen/generate-jssink.ts` - Generates stream-related classes
- `src/codegen/bundle-modules.ts` - Bundles built-in modules like `node:fs`
- `src/codegen/bundle-functions.ts` - Bundles global functions like `ReadableStream`

In development, bundled JS modules can be reloaded without rebuilding native code by running `bun run build`.

## JavaScript Modules (`src/js/`)

Built-in JavaScript modules use special syntax and are organized as:

- `node/` - Node.js compatibility modules (`node:fs`, `node:path`, etc.)
- `bun/` - Bun-specific modules (`bun:ffi`, `bun:sqlite`, etc.)
- `thirdparty/` - NPM modules we replace (like `ws`)
- `internal/` - Internal modules not exposed to users
- `builtins/` - Core JavaScript builtins (streams, console, etc.)

## Code Review Self-Check

- Before writing code that makes a non-obvious choice, pre-emptively ask "why this and not the alternative?" If you can't answer, research until you can — don't write first and justify later.
- Don't take a bug report's suggested fix at face value; verify it's the right layer.
- If neighboring code does something differently than you're about to, find out _why_ before deviating — its choices are often load-bearing, not stylistic.

## Important Development Notes

1. **Never use `bun test` or `bun <file>` directly** - always use `bun bd test` or `bun bd <command>`. `bun bd` compiles & runs the debug build.
2. **All changes must be tested** - if you're not testing your changes, you're not done.
3. **Get your tests to pass**. If you didn't run the tests, your code does not work.
4. **Follow existing code style** - check neighboring files for patterns
5. **Create tests in the right folder** in `test/` and the test must end in `.test.ts` or `.test.tsx`
6. **Use absolute paths** - Always use absolute paths in file operations
7. **Avoid shell commands** - Don't use `find` or `grep` in tests; use Bun's Glob and built-in tools
8. **Memory management** - Prefer RAII (`Drop`) over manual cleanup. Watch the arena edge case: values allocated in an arena (`Arena<T>`/`bumpalo`) do **not** run `Drop` when the arena is reset — if a type owns a heap allocation or a refcount, it must be freed/deref'd explicitly before the arena resets, mirroring the original Zig `deinit()` order.
9. **Cross-platform** - Run `bun run rust:check-all` to compile across all targets (linux/macos/windows × x64/aarch64) when making platform-specific changes. `#[cfg(...)]`-gated code is not type-checked unless the matching target is built.
10. **Debug builds** - Use `BUN_DEBUG_QUIET_LOGS=1` to disable debug logging, or `BUN_DEBUG_<SCOPE>=1` to enable a specific `bun_core::output` scoped logger
11. **Be humble & honest** - NEVER overstate what you got done or what actually works in commits, PRs or in messages to the user.
12. **Branch names must start with `claude/`** - This is a requirement for the CI to work.

**ONLY** push up changes after running `bun bd test <file>` and ensuring your tests pass.

## Debugging CI Failures

Requires the BuildKite CLI (`brew install buildkite/buildkite/bk`) and a read-scoped token in `BUILDKITE_API_TOKEN`. The repo's `.bk.yaml` sets the org/pipeline so `-p bun` is not needed.

```bash
# Show rendered test-failure output for the current branch's latest build,
# tagged [new] vs [also on main]
bun run ci:errors
bun run ci:errors '#26173' # or a PR number / URL / branch / build number

# One-screen progress summary (job counts, failed jobs, failing tests so far)
bun run ci:status

# Save full logs for every failed job to ./tmp/ci-<build>/
bun run ci:logs

# Just the build number, for composing with raw `bk`
bun run ci:find
bk job log <job-uuid> -b $(bun run ci:find)

# Watch the current branch's build until it finishes
bun run ci:watch
```

For anything else, use `bk` directly — `bk build list`, `bk api`, `bk artifacts`, etc.

If output from these commands looks wrong — mis-parsed annotation HTML, confusing wording, a field BuildKite changed shape on — fix `scripts/find-build.ts` directly rather than working around it. It's a thin presenter over `bk`; keep it accurate.

## Reading PR Feedback

`gh pr view --comments` is fine for a quick look at the Conversation tab, but it has a footgun worth knowing about: it only returns issue-stream comments and silently omits review summaries and line-level review comments. If a reviewer leaves an inline comment on a specific file line, it will not show up — no error, no hint that anything is missing.

When you want the complete picture — especially when responding to a review or checking whether anyone requested changes — use `bun run pr:comments`. It fetches all three GitHub endpoints (`/issues/N/comments`, `/pulls/N/reviews`, `/pulls/N/comments`) and prints them in one chronological listing, each labelled with its actual type (issue comment, review verdict, line comment, reply, suggestion block).

```bash
bun run pr:comments # current branch's PR — XML, resolved threads hidden
bun run pr:comments 28838 # by PR number
bun run pr:comments '#28838' # also works
bun run pr:comments https://github.com/oven-sh/bun/pull/28838
bun run pr:comments --include-resolved # also show threads already marked resolved

# Machine-readable output for jq pipelines — one object per entry with
# { when, user, tag, state?, suggestion?, location?, body, url?, resolved?, outdated? }.
# Resolved threads and bot noise (robobun's CI status comment, CodeRabbit
# body-level summaries) are filtered out; --include-resolved restores the former.
bun run pr:comments --json | jq '.[] | select(.user == "Jarred-Sumner")'
```


<claude-mem-context>
# Memory Context

# [bun] recent context, 2026-05-15 11:49pm GMT+9

Legend: 🎯session 🔴bugfix 🟣feature 🔄refactor ✅change 🔵discovery ⚖️decision 🚨security_alert 🔐security_note
Format: ID TIME TYPE TITLE
Fetch details: get_observations([IDs]) | Search: mem-search skill

Stats: 14 obs (5,311t read) | 165,493t work | 97% savings

### May 15, 2026
269 11:34p 🔵 Bun FFI dlopen() regression from Rust rewrite blocks embedded native libraries
270 " 🔵 Issue #30717 FFI dlopen regression already fixed in merged PR #30720
271 11:35p 🔵 Bun FFI has multiple active threading and memory leak issues with JSCallback
272 " 🔵 Windows ARM64 FFI support blocked on TinyCC backend implementation
S137 Systematic investigation of Bun FFI critical issues, focusing on JSCallback threading and memory management problems (May 15 at 11:36 PM)
273 11:45p 🔵 FFI threadsafe JSCallback test hanging indefinitely
274 11:46p 🔴 Fixed hanging threadsafe JSCallback test in FFI test suite
275 11:47p ✅ Increased threadsafe JSCallback test stress level
276 " 🔵 FFI JSCallback crashes with segfault under high-volume cross-thread invocation
277 " 🔵 FFI callback implementation uses ThreadSafeRefCounted for lifecycle management
278 " 🔴 Fixed use-after-free in threadsafe FFI callbacks
279 11:48p ✅ Improved thread safety and cleanup in FFI callback test
280 " 🔄 Refined FFI callback protection to guard all member access
281 " ✅ Standardized move syntax in FFI callback lambda capture
282 11:49p ✅ Documented threading safety constraints in FFI callback implementation

Access 165k tokens of past work via get_observations([IDs]) or mem-search skill.
</claude-mem-context>
4 changes: 3 additions & 1 deletion scripts/build/rust.ts
Original file line number Diff line number Diff line change
Expand Up @@ -486,7 +486,9 @@ export function emitRust(n: Ninja, cfg: Config, inputs: RustBuildInputs): string
// `-fuse-ld=`. RUSTFLAGS only reach *target* crates when `--target` is given,
// and the `bun_bin` staticlib has no link step, so it's normally dead — but
// if a target cdylib ever appears it'd fail with "could not open '-fuse-ld=lld'".
if (!cfg.windows) rustflags.push(`-Clink-arg=-fuse-ld=lld`);
if (!cfg.windows && !cfg.darwin) {
rustflags.push(`-Clink-arg=-fuse-ld=lld`);
}
if (cfg.crossLangLto) {
// Cross-language LTO: emit LLVM bitcode (not machine code) into the .a
// so the final lld `-flto=full` link sees through Rust↔C++ call edges.
Expand Down
Loading
Loading