Skip to content

Everything is cmake - #13427

Merged
Electroid merged 266 commits into
mainfrom
everything-is-cmake
Sep 11, 2024
Merged

Everything is cmake#13427
Electroid merged 266 commits into
mainfrom
everything-is-cmake

Conversation

@Electroid

@Electroid Electroid commented Aug 20, 2024

Copy link
Copy Markdown
Contributor

What does this PR do?

This is a work-in-progress PR that will consolidate all of our build scripts to cmake. The goal is that you only need to run cmake to build Bun from scratch.

  • Documentation or TypeScript types (it's okay to leave the rest blank in this case)
  • Code changes

How did you verify your code works?

@robobun

robobun commented Aug 20, 2024 •
edited
Loading

Copy link
Copy Markdown
Collaborator

❌ @Electroid, your commit 3de5893 has 8 failures in #3243:

  • test/js/web/streams/streams.test.js - 1 failing on 🍎 13 aarch64
  • test/js/web/streams/streams.test.js - 1 failing on 🍎 14 x64
  • test/js/bun/spawn/spawn-streaming-stdin.test.ts - 1 failing on 🪟 x64
  • test/js/bun/spawn/spawn-streaming-stdin.test.ts - 1 failing on 🪟 x64-baseline
  • test/cli/watch/watch.test.ts - timeout on 🪟 x64-baseline
  • test/cli/watch/watch.test.ts - timeout on 🪟 x64
  • test/integration/expo-app/expo.test.ts - 1 failing on 🪟 x64
  • test/integration/expo-app/expo.test.ts - 1 failing on 🪟 x64-baseline
  • test/cli/install/registry/bun-install-registry.test.ts - 3 failing on 🪟 x64
  • test/cli/install/registry/bun-install-registry.test.ts - 3 failing on 🪟 x64-baseline
  • test/cli/hot/hot.test.ts - 2 failing on 🪟 x64
  • test/cli/hot/hot.test.ts - 2 failing on 🪟 x64-baseline
  • test/cli/hot/watch.test.ts - 1 failing on 🪟 x64-baseline
  • test/cli/hot/watch.test.ts - 1 failing on 🪟 x64
  • test/bundler/bun-build-api.test.ts - 1 failing on 🪟 x64-baseline
  • test/bundler/bun-build-api.test.ts - 1 failing on 🪟 x64
    		•

    @github-actions

    github-actions Bot commented Aug 20, 2024 •
    edited
    Loading

    Copy link
    Copy Markdown
    Contributor

    clang-tidy nits are fixed! Thank you.

    @Electroid
    Electroid force-pushed the everything-is-cmake branch 2 times, most recently from 25dd8c1 to 9e30441 Compare August 27, 2024 18:02
    @Electroid
    Electroid force-pushed the everything-is-cmake branch 2 times, most recently from f91e3aa to 94c0800 Compare September 6, 2024 23:48
    @Electroid
    Electroid marked this pull request as ready for review September 7, 2024 01:08
    Jarred-Sumner
    Jarred-Sumner reviewed Sep 10, 2024
    View reviewed changes
    Comment thread cmake/tools/SetupWebKit.cmake
    Jarred-Sumner
    Jarred-Sumner reviewed Sep 10, 2024
    View reviewed changes
    Comment thread cmake/analysis/RunIWYU.cmake
    @@ -0,0 +1,69 @@
    # IWYU = "Include What You Use"
    # https://include-what-you-use.org/

    @Jarred-Sumner Jarred-Sumner Sep 10, 2024

    Copy link
    Copy Markdown
    Collaborator

    Choose a reason for hiding this comment

    The reason will be displayed to describe this comment to others. Learn more.

    are we using this?

    Jarred-Sumner
    Jarred-Sumner reviewed Sep 10, 2024
    View reviewed changes
    Comment thread cmake/analysis/RunCppLint.cmake
    @@ -0,0 +1,24 @@
    include(Macros)

    @Jarred-Sumner Jarred-Sumner Sep 10, 2024

    Copy link
    Copy Markdown
    Collaborator

    Choose a reason for hiding this comment

    The reason will be displayed to describe this comment to others. Learn more.

    we aren't using cpp lint right? just clang-tidy

    cpplint looked deprecated when i last checked

    @Electroid
    Electroid merged commit 354df17 into main Sep 11, 2024
    @Electroid
    Electroid deleted the everything-is-cmake branch September 11, 2024 00:01
    dylan-conway added a commit that referenced this pull request Sep 11, 2024
    @dylan-conway
    Revert "Convert build scripts to CMake (#13427)"
    374bb15 
    This reverts commit 354df17.
    Electroid added a commit that referenced this pull request Sep 11, 2024
    @Electroid
    Reapply "Convert build scripts to CMake (#13427)"
    d39e422 
    This reverts commit 374bb15.
    @matubu matubu mentioned this pull request Oct 2, 2024
    Merged
    2 tasks
    @SunsetTechuila SunsetTechuila mentioned this pull request Jun 14, 2025
    Closed
    2 tasks
    @robobun robobun mentioned this pull request Aug 5, 2026
    Merged
    Jarred-Sumner pushed a commit that referenced this pull request Aug 5, 2026
    @robobun @autofix-ci
    Remove dead code from FFI sys crates, bun_core, built-in JS, codegen,…
    b58cd46 
    … and build scripts (#36937)

Removes 2,255 lines (net -2,057) of verified-dead code across 116 files:
unused `pub` items the `dead_code` lint cannot see (it treats every
`pub` item as an external API root), unreferenced FFI declarations,
orphaned files, stale commented-out C++ blocks, and dead build-script
helpers.

Method: started from `scripts/find-dead-exports.ts`, narrowed 8,259
candidates to 1,039 whose name appears in exactly one file across src/,
scripts/, test/, and regenerated `build/debug/codegen/` output, then
demoted each to private and let the workspace's `dead_code = deny` prove
which were genuinely unreferenced. Items the compiler proved live (macro
expansions, return-position escapes, cfg(windows)/cfg(darwin) usage,
codegen references) were restored; the rest were deleted. Areas with
open dead-code PRs (#36237, #35775, #36115, #35437, #35880) were
excluded, and the one overlapping deletion found during final diffing
(`kGetNativeReadableProto`, already in #35775) was dropped from this PR.

### Removed

**FFI declaration crates** (unused imports of vendored C functions;
declarations only, no link-time effect):
- `src/mimalloc_sys/mimalloc.rs`: ~100 declarations (heap-local variants
`mi_heap_*`, stats/options surface `mi_stats_*` `mi_option_*`, posix
shims `mi_posix_memalign`/`mi_valloc`/`mi_pvalloc`, C++ `mi_new_*`
family)
- `src/windows_sys/externs.rs`: never-constructed `WinsockError` + ~100
`WSA*` error consts, `GetBinaryTypeW`, `CreateJobObjectW`, `M128A` kept
(used by `CONTEXT` on windows)
- `src/zlib_sys/win32.rs`: 44 declarations (`gz*` file API,
`deflateTune`, `inflateUndermine`, `zlibCompileFlags`, ...)
- `src/cares_sys/c_ares.rs`: 21 (option/server-config surface
`ares_set_servers*`, `ares_mkquery`, `ares_parse_txt_reply_ext`, ...)
- `src/lsquic_sys/lib.rs`: 11 (handshake/conn status consts, stream-ctx
helpers)
- `src/brotli_sys`, `src/boringssl_sys`, `src/libdeflate_sys`: 17
combined (`BrotliEncoder*` estimators, `TLS_with_buffers_method`,
`libdeflate_*_decompress`, ...)

**Rust runtime/support crates**:
- `src/bun_core/env.rs`: `BuildTarget::Wasi` variant + `IS_WASI` (never
constructed; `IS_BROWSER` simplifies to `IS_WASM`)
- `src/jsc/HTTPServerAgent.rs`: 5 unused Rust-side imports of
`Bun__HTTPServerAgent__notify*` + 2 type aliases
- `src/runtime/test_runner/mod.rs`: `JSGlobalObjectTestExt::throw2`
(duplicated `throw_error`)
- `src/runtime/server/NodeHTTPResponse.rs`: unused `pause_socket`
sibling cleanup; `pause_socket_reads` kept (referenced by generated
bindings)
- ~230 surviving demotions of file-local `pub` items to private across
60 crates, which moves them permanently under `dead_code` analysis
- assorted single items: dead re-export lines in
`sql_jsc`/`sourcemap_jsc`/`bundler_jsc`/`runtime/api.rs`,
`EventLoopGuard`-adjacent aliases, unused imports

**Built-in JS / codegen / build scripts**:
- `src/node-fallbacks/timers.promises.js` (238 lines): never registered
in `src/resolver/node_fallbacks.rs`'s 23-module registry, so it was
built and compressed on every build but could never be served
- `src/js/internal/crypto/x509.ts` + its row in
`ProcessBindingNatives.cpp`: `process.binding("crypto/x509")` is
implemented natively in `BunProcess.cpp`
- `src/js/internal/validators.ts`: `validateUndefined`,
`validateSignalName`/`validatePlainFunction` export entries
- `src/js/internal/{shared,tls,streams/utils}.ts`: dead export-object
entries (definitions stay where used in-file); `primordials.js`: 5 dead
scalar-constructor keys (typed-array keys kept: `util.inspect` reaches
them via computed `primordials[tag]` access)
- `src/codegen`: `camelCase`, `pascalCase`,
`warnOnIdentifiersNotPresentAtRuntime`, `DOMJITReturnType`, `ownRow`,
`cppPointer`
- `scripts/build`: `explainFlags` (no `--explain-flags` exists),
`assertDefined`, `depSourceStamp`; root `package.json` `bump` script
(its target was deleted in #13427)

**Orphaned files**: `src/fixtures_example.com.html`, `src/zlib.test.txt`
+ `src/zlib.test.gz` (2021 inline-test fixtures), `src/fallback.html`
(only `fallback-backend.html` is embedded), `src/logo.svg`,
`src/favicon.png`

**Stale commented-out C++ blocks** (~243 lines, all >6 months old via
git blame): minicoro scaffolding in `coroutine.cpp` (2022), pasted Node
JS source in `JSX509CertificatePrototype.cpp`, commented
`BINDING_INTEGRITY` vtable checks, `InspectorInstrumentation` calls in
`WebSocket.cpp`, suspended-event-loop paths in
`JSDOMPromiseDeferred.cpp`, commented-out function bodies in
`Performance.cpp`/`Event.cpp`/`DOMWrapperWorld.cpp`/`ErrorEvent.cpp` and
8 more webcore files

### Verification

```
cargo check --workspace            # green (dead_code/unreachable_pub/unused_* all deny)
bun run rust:check-all             # 10/10 targets (linux/macos/windows/freebsd/android x arches)
bun bd                             # full debug build with freshly regenerated codegen
bun test test/internal/source-lints/   # 65 pass (includes the new pin test below)
bun bd test test/js/node/fs/fs.test.ts            # 445 pass
bun bd test test/js/node/http/node-http.test.ts   # 144 pass, 1 env-dependent proxy failure also fails on released bun
bun bd test test/js/bun/glob/match.test.ts        # 29 pass
bun bd test test/js/bun/resolve/resolve.test.ts   # 50 pass
```

`test/internal/source-lints/dead-symbols-pub-exports-sweep.test.ts` pins
representative removed symbols and the deleted files against
reintroduction.

The compiler-driven loop caught and restored every false positive the
textual scan missed: methods on types that escape only via return
position (`StdinReader::take_byte`, `HeaderSet::pairs`, bitset
iterators), macro-referenced items (`comptime_string_map!` statics,
`$crate::` paths), platform-gated items (`EmptyCopyFileState` on
darwin/freebsd, `M128A`/`WriteKind` on windows), and generated-binding
references (`get_insecure_http_parser`, `pause_socket_reads`) which only
appear after codegen reruns.

### Followups (not removed)

- `patches/ncrypto.patch` (919 lines) is referenced by nothing in
`scripts/build/deps/*.ts` while every other patch file is;
`src/jsc/bindings/ncrypto.{h,cpp}` already exist in patched form.
Possibly kept as upstream-sync documentation, so left alone.
- The `#if ENABLE(BINDING_INTEGRITY)` extern vtable scaffolding in
`JSCustomEvent.cpp`/`JSPerformanceServerTiming.cpp` lost its only
(commented) consumer but matches upstream WebKit codegen shape; left in
place.
- `.github/workflows/release.yml:2` references
`.buildkite/scripts/release.sh`, which no longer exists (comment only).
- `src/react_compiler` and `src/ini` were deliberately excluded: the
former carries explicit not-yet-wired port markers, the latter's
candidates proved live on inspection.

<!-- robobun:evidence:begin -->

---

**[decide:dep]** gate passed · iteration 3 · 126 files touched

<details><summary>fails on main (without fix)</summary>

```console
ASAN without fix: 2 FAILED
$ BUN_DEBUG_QUIET_LOGS=1 bun scripts/build.ts --profile=debug --quiet test "--reporter=junit" "--reporter-outfile=/tmp/mechgate.xml" test/internal/source-lints/dead-symbols-pub-exports-sweep.test.ts
bun test v1.4.0 (a9da323)

test/internal/source-lints/dead-symbols-pub-exports-sweep.test.ts:
78 |     ["src/libdeflate_sys/libdeflate.rs", /\bfn libdeflate_gzip_decompress\b/],
79 |     ["src/brotli_sys/brotli_c.rs", /\bfn BrotliEncoderEstimatePeakMemoryUsage\b/],
80 |     ["src/boringssl_sys/boringssl.rs", /\bfn TLS_with_buffers_method\b/],
81 |   ];
82 |   const resurrected = checks.filter(([file, re]) => re.test(src(file))).map(([file, re]) => `${file}: ${re.source}`);
83 |   expect(resurrected).toEqual([]);
                           ^
error: expect(received).toEqual(expected)

- []
+ [
+   "src/windows_sys/externs.rs: \bfn GetBinaryTypeW\b",
+   "src/windows_sys/externs.rs: \bfn CreateJobObjectW\b",
+   "src/windows_sys/externs.rs: \bstruct WinsockError\b",
+   "src/windows_sys/externs.rs: \bWSA_QOS_ESHAPERATEOBJ\b",
+   "src/mimalloc_sys/mimalloc.rs: \bfn mi_stats_print\b",
+   "src/mimalloc_sys/mimalloc.rs: \bfn mi_reserve_huge_os_pages_interleave\b",
+   "src/
... (truncated)

release without fix: 2 FAILED
bun test v1.4.0-canary.1 (57ae5f0)

test/internal/source-lints/dead-symbols-pub-exports-sweep.test.ts:
78 |     ["src/libdeflate_sys/libdeflate.rs", /\bfn libdeflate_gzip_decompress\b/],
79 |     ["src/brotli_sys/brotli_c.rs", /\bfn BrotliEncoderEstimatePeakMemoryUsage\b/],
80 |     ["src/boringssl_sys/boringssl.rs", /\bfn TLS_with_buffers_method\b/],
81 |   ];
82 |   const resurrected = checks.filter(([file, re]) => re.test(src(file))).map(([file, re]) => `${file}: ${re.source}`);
83 |   expect(resurrected).toEqual([]);
                           ^
error: expect(received).toEqual(expected)

- []
+ [
+   "src/windows_sys/externs.rs: \bfn GetBinaryTypeW\b",
+   "src/windows_sys/externs.rs: \bfn CreateJobObjectW\b",
+   "src/windows_sys/externs.rs: \bstruct WinsockError\b",
+   "src/windows_sys/externs.rs: \bWSA_QOS_ESHAPERATEOBJ\b",
+   "src/mimalloc_sys/mimalloc.rs: \bfn mi_stats_print\b",
+   "src/mimalloc_sys/mimalloc.rs: \bfn mi_reserve_huge_os_pages_interleave\b",
+   "src/mimalloc_sys/mimalloc.rs: \bfn mi_heap_recalloc_aligned_at\b",
+   "src/mimalloc_sys/mimalloc.rs: \bfn mi_wdupenv_s\b",
+   "src/zlib_sys/win32.rs: \bfn gzprintf\b",
+   "src/zlib_sys/win32.
... (truncated)
```

</details>

<details><summary>passes on PR (with fix)</summary>

```console
ASAN with fix: all passed
$ BUN_DEBUG_QUIET_LOGS=1 bun scripts/build.ts --profile=debug --quiet test "--reporter=junit" "--reporter-outfile=/tmp/mechgate.xml" test/internal/source-lints/dead-symbols-pub-exports-sweep.test.ts
bun test v1.4.0 (a9da323)

test/internal/source-lints/dead-symbols-pub-exports-sweep.test.ts:
(pass) dead FFI declarations (sys crates) do not reappear [37.66ms]
(pass) dead Rust symbols (bun_core, jsc, test_runner) do not reappear [9.84ms]
(pass) orphaned files stay deleted [566.74ms]
(pass) dead JS/codegen helpers do not reappear [56.45ms]
(pass) stale commented-out C++ blocks stay deleted [22.77ms]

 5 pass
 0 fail
 5 expect() calls
Ran 5 tests across 1 file. [2.71s]
__F:0:S:0

release with fix: all passed
$ bun scripts/build.ts --profile=release
[configured] bun-profile → bun (stripped) in 658ms (unchanged)
ninja: Entering directory `/workspace/bun/build/release'
[1/133] gen ErrorCode+*.h
[2/133] gen generated_host_exports.rs
generated_host_exports.rs: 94 exports (host=3, lazy=10, generic=81, rust=0); 239 extern-C blocks audited
[3/133] gen ZigGeneratedClasses.{cpp,h,rs}
Found 2 classes from /workspace/bun/src/jsc/resolve_message.classes.ts
  - ResolveMessage (13 fields)
  - BuildMessage (10 fields)
Found 1 classes from /workspace/bun/src/runtime/api/Archive.classes.ts
  - Archive (4 fields, 1 class fields)
Found 2 classes from /workspace/bun/src/runtime/api/BunObject.classes.ts
  - ResourceUsage (8 fields)
  - Subprocess (20 fields)
Found 1 classes from /workspace/bun/src/runtime/api/cron.classes.ts
  - CronJob (5 fields)
Found 3 classes from /workspace/bun/src/runtime/api/filesystem_router.classes.ts
  - FileSystemRouter (5 fields)
  - FrameworkFileSystemRouter (2 fields)
  - MatchedRoute (8 fields)
Found 1 classes from /workspace/bun/src/runtime/api/Glob.classes.ts
  - Glob (5 fields)
Found 1 classes from /workspace/bun/src/runtime/api/h2.classes.ts
  - H2FrameP
... (truncated)
```

</details>

<details><summary>diff hotspot</summary>

```
package.json                                       |   1 -
 scripts/build/error.ts                             |  13 -
 scripts/build/flags.ts                             |  39 +-
 scripts/build/source.ts                            |   8 -
 src/ast/lib.rs                                     |   2 +-
 src/ast/nodes.rs                                   |   1 -
 src/base64/lib.rs                                  |   2 +-
 src/boringssl/lib.rs                               |   8 +-
 src/boringssl_sys/boringssl.rs                     |  12 +-
 src/brotli/lib.rs                                  |   4 +-
 src/brotli_sys/brotli_c.rs                         |  47 +-
 src/bun_alloc/lib.rs                               |  10 +-
 src/bun_core/Global.rs                             |   2 +-
 src/bun_core/env.rs                                |   6 +-
 src/bun_core/fmt.rs                                |  14 +-
 src/bundler/HTMLScanner.rs                         |   2 +-
 src/bundler_jsc/PluginRunner.rs                    |   4 -
 src/cares_sys/c_ares.rs                            | 126 +---
 src/codegen/generate-classes.ts                    |  33 --
 src/codegen/generate-js2native.ts                  |   4 -
 src/codegen/helpers.ts                             |  10 -
 src/codegen/replacements.ts                        |  10 -
 src/collections/array_hash_map.rs                  |   4 +-
 src/crash_handler/lib.rs                           |   8 +-
 src/event_loop/MiniEventLoop.rs                    |   4 +-
 src/fallback.html                                  |  28 -
 src/favicon.png                                    | Bin 7804 -> 0 bytes
 src/fixtures_example.com.html                      |  50 --
 src/glob/GlobWalker.rs                             |   4 +-
 src/http/lib.rs                                    |   6 +-
 src/install/lib.rs                                 |   8 +-
 src/install/lockfile.rs                            |   4 +-
 src/install/resolvers/fold
... (truncated)
```

</details>

**gate history** · 3 passed · 1 rejected · iteration 3

<details><summary>evidence per changed file</summary>

```
file                            reads  edits  tests
package.json                        0      0      0
scripts/build/error.ts              0      0      0
scripts/build/flags.ts              0      0      0
scripts/build/source.ts             0      0      0
src/ast/lib.rs                      0      0      0
src/ast/nodes.rs                    1      1      0
src/base64/lib.rs                   4      4      0
src/boringssl/lib.rs                1      1      0
src/boringssl_sys/boringssl.rs      1      2      0
src/brotli/lib.rs                   0      0      0
src/brotli_sys/brotli_c.rs          1      2      0
src/bun_alloc/lib.rs                1      1      0
src/bun_core/Global.rs              0      0      0
src/bun_core/env.rs                 1      1      0
src/bun_core/fmt.rs                 1      2      0
src/bundler/HTMLScanner.rs          0      0      0
(+ 110 more files)
```

</details>

<!-- robobun:evidence:end -->

---------

Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
    @robobun robobun mentioned this pull request Sep 23, 2026
    Closed
    Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

    Reviewers

    @Jarred-Sumner Jarred-Sumner Jarred-Sumner left review comments

    Assignees

    No one assigned

    Labels

    None yet

    Projects

    None yet

    Milestone

    No milestone

    Development

    Successfully merging this pull request may close these issues.

    3 participants

    @Electroid @robobun @Jarred-Sumner