From 270a972173b5ca065f918aa45a7cc295dbf54656 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?=E8=AE=B8=E6=9D=B0=E5=8F=8B=20Jieyou=20Xu=20=28Joe=29?= Date: Wed, 13 Mar 2024 16:55:57 +0000 Subject: [PATCH 1/4] Extract Bootstrap into its own section Add brief explanation for `Step` and `Builder::ensure` as core Bootstrap internal concepts. --- src/SUMMARY.md | 9 +++- src/building/bootstrapping/common-commands.md | 15 ++++++ .../bootstrapping/how-bootstrap-does-it.md | 48 +++++++++++++++++++ src/building/bootstrapping/intro.md | 23 +++++++++ .../what-bootstrapping-does.md} | 15 +----- 5 files changed, 94 insertions(+), 16 deletions(-) create mode 100644 src/building/bootstrapping/common-commands.md create mode 100644 src/building/bootstrapping/how-bootstrap-does-it.md create mode 100644 src/building/bootstrapping/intro.md rename src/building/{bootstrapping.md => bootstrapping/what-bootstrapping-does.md} (96%) diff --git a/src/SUMMARY.md b/src/SUMMARY.md index 5ac26f150..2d9aca108 100644 --- a/src/SUMMARY.md +++ b/src/SUMMARY.md @@ -33,7 +33,6 @@ - [with Windows Performance Analyzer](./profiling/wpa_profiling.md) - [crates.io Dependencies](./crates-io.md) - # Contributing to Rust - [Contribution Procedures](./contributing.md) @@ -58,12 +57,18 @@ - [Licenses](./licenses.md) - [Editions](guides/editions.md) +# Bootstrapping + +- [Prologue](./building/bootstrapping/intro.md) +- [What Bootstrapping does](./building/bootstrapping/what-bootstrapping-does.md) +- [How Bootstrap does it](./building/bootstrapping/how-bootstrap-does-it.md) +- [Common Bootstrap commands](./building/bootstrapping/common-commands.md) + # High-level Compiler Architecture - [Prologue](./part-2-intro.md) - [Overview of the compiler](./overview.md) - [The compiler source code](./compiler-src.md) -- [Bootstrapping](./building/bootstrapping.md) - [Queries: demand-driven compilation](./query.md) - [The Query Evaluation Model in Detail](./queries/query-evaluation-model-in-detail.md) - [Incremental compilation](./queries/incremental-compilation.md) diff --git a/src/building/bootstrapping/common-commands.md b/src/building/bootstrapping/common-commands.md new file mode 100644 index 000000000..83766d39f --- /dev/null +++ b/src/building/bootstrapping/common-commands.md @@ -0,0 +1,15 @@ +# Common `x` commands + +| Command | Explanation | Remarks | +|------------------------------------------------|---------------------------------------------|-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------| +| `x build --keep-stage 0` | Assume stage0 std can be reused | | +| `x build --stage 1` | Build up to stage1 std and create a sysroot | | +| `x clean` | Invalidate build config cache | Does not clean LLVM cache | +| `x dist rustc-dev` | Build rustc-dev | | +| `x test --stage 0 library/std` | Test standard library | | +| `x check library/std` | Only check standard library | | +| `x check` | Check compiler/rustc | | +| `x check --all-targets` | Cross-compile for a lot of targets | Can be made less failure-prone by putting the imports inside the function where they're used rather than using separate configs; creates a problem of missing functions | +| `x build library/std --stage 0` | If only working on standard library | Builds library/std then compiler/rustc then library/std again | +| `x test --stage 1` | Test the compiler | | +| `x test --stage 1 --keep-stage-std 1 tests/ui` | Run UI tests with cached stage1 std | | diff --git a/src/building/bootstrapping/how-bootstrap-does-it.md b/src/building/bootstrapping/how-bootstrap-does-it.md new file mode 100644 index 000000000..b8d7636ba --- /dev/null +++ b/src/building/bootstrapping/how-bootstrap-does-it.md @@ -0,0 +1,48 @@ +# How Bootstrap does it + +The core concept in Bootstrap is a build [`Step`], which are chained together +by [`Builder::ensure`]. [`Builder::ensure`] takes a [`Step`] as input, and runs +the [`Step`] if and only if it has not already been run. Let's take a closer +look at [`Step`]. + +## Synopsis of [`Step`] + +A [`Step`] represents a granular collection of actions involved in the process +of producing some artifact. It can be thought of like a rule in Makefiles. +The [`Step`] trait is defined as: + +```rs,no_run +pub trait Step: 'static + Clone + Debug + PartialEq + Eq + Hash { + type Output: Clone; + + const DEFAULT: bool = false; + const ONLY_HOSTS: bool = false; + + // Required methods + fn run(self, builder: &Builder<'_>) -> Self::Output; + fn should_run(run: ShouldRun<'_>) -> ShouldRun<'_>; + + // Provided method + fn make_run(_run: RunConfig<'_>) { ... } +} +``` + +- `run` is the function that is responsible for doing the work. + [`Builder::ensure`] invokes `run`. +- `should_run` is the command-line interface, which determines if an invocation + such as `x build foo` should run a given [`Step`]. In a "default" context + where no paths are provided, then `make_run` is called directly. +- `make_run` is invoked only for things directly asked via the CLI and not + for steps which are dependencies of other steps. + +## The entry points + +There's a couple of preliminary steps before core Bootstrap code is reached: + +1. Shell script: [`./x`](https://github.com/rust-lang/rust/blob/master/x) or [`./x.ps1`](https://github.com/rust-lang/rust/blob/master/x.ps1) +2. Convenience wrapper script: [`x.py`](https://github.com/rust-lang/rust/blob/master/x.py) +3. [`src/bootstrap/bootstrap.py`](https://github.com/rust-lang/rust/blob/master/src/bootstrap/bootstrap.py) +4. [`src/bootstrap/src/bin/main.rs`](https://github.com/rust-lang/rust/blob/master/src/bootstrap/src/bin/main.rs) + +[`Step`]: https://doc.rust-lang.org/nightly/nightly-rustc/bootstrap/core/builder/trait.Step.html +[`Builder::ensure`]: https://doc.rust-lang.org/nightly/nightly-rustc/bootstrap/core/builder/struct.Builder.html#method.ensure diff --git a/src/building/bootstrapping/intro.md b/src/building/bootstrapping/intro.md new file mode 100644 index 000000000..50fa59038 --- /dev/null +++ b/src/building/bootstrapping/intro.md @@ -0,0 +1,23 @@ +# Bootstrapping the compiler + +[*Bootstrapping*][boot] is the process of using a compiler to compile itself. +More accurately, it means using an older compiler to compile a newer version +of the same compiler. + +This raises a chicken-and-egg paradox: where did the first compiler come from? +It must have been written in a different language. In Rust's case it was +[written in OCaml][ocaml-compiler]. However it was abandoned long ago and the +only way to build a modern version of rustc is a slightly less modern +version. + +This is exactly how `x.py` works: it downloads the current beta release of +rustc, then uses it to compile the new compiler. + +In this section, we give a high-level overview of +[what Bootstrap does](./what-bootstrapping-does.md), followed by a high-level +introduction to [how Bootstrap does it](./how-bootstrap-does-it.md). A +[listing of common bootstrap commands](./common-commands.md) is also provided +for convenience. + +[boot]: https://en.wikipedia.org/wiki/Bootstrapping_(compilers) +[ocaml-compiler]: https://github.com/rust-lang/rust/tree/ef75860a0a72f79f97216f8aaa5b388d98da6480/src/boot diff --git a/src/building/bootstrapping.md b/src/building/bootstrapping/what-bootstrapping-does.md similarity index 96% rename from src/building/bootstrapping.md rename to src/building/bootstrapping/what-bootstrapping-does.md index 7bee8d3aa..6920ee526 100644 --- a/src/building/bootstrapping.md +++ b/src/building/bootstrapping/what-bootstrapping-does.md @@ -1,20 +1,7 @@ -# Bootstrapping the compiler +# What Bootstrapping does -[*Bootstrapping*][boot] is the process of using a compiler to compile itself. -More accurately, it means using an older compiler to compile a newer version -of the same compiler. - -This raises a chicken-and-egg paradox: where did the first compiler come from? -It must have been written in a different language. In Rust's case it was -[written in OCaml][ocaml-compiler]. However it was abandoned long ago and the -only way to build a modern version of rustc is a slightly less modern -version. - -This is exactly how `x.py` works: it downloads the current beta release of -rustc, then uses it to compile the new compiler. - Note that this documentation mostly covers user-facing information. See [bootstrap/README.md][bootstrap-internals] to read about bootstrap internals. From e1ea8d5f9936af4d94fbdb8ae322be712a67c57a Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?=E8=AE=B8=E6=9D=B0=E5=8F=8B=20Jieyou=20Xu=20=28Joe=29?= Date: Wed, 13 Mar 2024 17:39:37 +0000 Subject: [PATCH 2/4] Drop common commands page (use `x --help` instead) --- src/SUMMARY.md | 1 - src/building/bootstrapping/common-commands.md | 15 --------------- src/building/bootstrapping/intro.md | 4 +--- 3 files changed, 1 insertion(+), 19 deletions(-) delete mode 100644 src/building/bootstrapping/common-commands.md diff --git a/src/SUMMARY.md b/src/SUMMARY.md index 2d9aca108..b905e7ef3 100644 --- a/src/SUMMARY.md +++ b/src/SUMMARY.md @@ -62,7 +62,6 @@ - [Prologue](./building/bootstrapping/intro.md) - [What Bootstrapping does](./building/bootstrapping/what-bootstrapping-does.md) - [How Bootstrap does it](./building/bootstrapping/how-bootstrap-does-it.md) -- [Common Bootstrap commands](./building/bootstrapping/common-commands.md) # High-level Compiler Architecture diff --git a/src/building/bootstrapping/common-commands.md b/src/building/bootstrapping/common-commands.md deleted file mode 100644 index 83766d39f..000000000 --- a/src/building/bootstrapping/common-commands.md +++ /dev/null @@ -1,15 +0,0 @@ -# Common `x` commands - -| Command | Explanation | Remarks | -|------------------------------------------------|---------------------------------------------|-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------| -| `x build --keep-stage 0` | Assume stage0 std can be reused | | -| `x build --stage 1` | Build up to stage1 std and create a sysroot | | -| `x clean` | Invalidate build config cache | Does not clean LLVM cache | -| `x dist rustc-dev` | Build rustc-dev | | -| `x test --stage 0 library/std` | Test standard library | | -| `x check library/std` | Only check standard library | | -| `x check` | Check compiler/rustc | | -| `x check --all-targets` | Cross-compile for a lot of targets | Can be made less failure-prone by putting the imports inside the function where they're used rather than using separate configs; creates a problem of missing functions | -| `x build library/std --stage 0` | If only working on standard library | Builds library/std then compiler/rustc then library/std again | -| `x test --stage 1` | Test the compiler | | -| `x test --stage 1 --keep-stage-std 1 tests/ui` | Run UI tests with cached stage1 std | | diff --git a/src/building/bootstrapping/intro.md b/src/building/bootstrapping/intro.md index 50fa59038..f829884fb 100644 --- a/src/building/bootstrapping/intro.md +++ b/src/building/bootstrapping/intro.md @@ -15,9 +15,7 @@ rustc, then uses it to compile the new compiler. In this section, we give a high-level overview of [what Bootstrap does](./what-bootstrapping-does.md), followed by a high-level -introduction to [how Bootstrap does it](./how-bootstrap-does-it.md). A -[listing of common bootstrap commands](./common-commands.md) is also provided -for convenience. +introduction to [how Bootstrap does it](./how-bootstrap-does-it.md). [boot]: https://en.wikipedia.org/wiki/Bootstrapping_(compilers) [ocaml-compiler]: https://github.com/rust-lang/rust/tree/ef75860a0a72f79f97216f8aaa5b388d98da6480/src/boot From 78f270a3158b4a8398ac8e7c52a1d62c49959ea7 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?=E8=AE=B8=E6=9D=B0=E5=8F=8B=20Jieyou=20Xu=20=28Joe=29?= Date: Wed, 13 Mar 2024 17:42:35 +0000 Subject: [PATCH 3/4] Add `make` as an alternative entry point --- src/building/bootstrapping/how-bootstrap-does-it.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/building/bootstrapping/how-bootstrap-does-it.md b/src/building/bootstrapping/how-bootstrap-does-it.md index b8d7636ba..93a502379 100644 --- a/src/building/bootstrapping/how-bootstrap-does-it.md +++ b/src/building/bootstrapping/how-bootstrap-does-it.md @@ -39,7 +39,7 @@ pub trait Step: 'static + Clone + Debug + PartialEq + Eq + Hash { There's a couple of preliminary steps before core Bootstrap code is reached: -1. Shell script: [`./x`](https://github.com/rust-lang/rust/blob/master/x) or [`./x.ps1`](https://github.com/rust-lang/rust/blob/master/x.ps1) +1. Shell script or `make`: [`./x`](https://github.com/rust-lang/rust/blob/master/x) or [`./x.ps1`](https://github.com/rust-lang/rust/blob/master/x.ps1) or `make` 2. Convenience wrapper script: [`x.py`](https://github.com/rust-lang/rust/blob/master/x.py) 3. [`src/bootstrap/bootstrap.py`](https://github.com/rust-lang/rust/blob/master/src/bootstrap/bootstrap.py) 4. [`src/bootstrap/src/bin/main.rs`](https://github.com/rust-lang/rust/blob/master/src/bootstrap/src/bin/main.rs) From 8840ddcf17b60cc9ecb0f96f532bd329263ffb8c Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?=E8=AE=B8=E6=9D=B0=E5=8F=8B=20Jieyou=20Xu=20=28Joe=29?= Date: Wed, 13 Mar 2024 17:51:41 +0000 Subject: [PATCH 4/4] Add src/bootstrap/README.md link --- src/building/bootstrapping/how-bootstrap-does-it.md | 3 +++ 1 file changed, 3 insertions(+) diff --git a/src/building/bootstrapping/how-bootstrap-does-it.md b/src/building/bootstrapping/how-bootstrap-does-it.md index 93a502379..6b32845f0 100644 --- a/src/building/bootstrapping/how-bootstrap-does-it.md +++ b/src/building/bootstrapping/how-bootstrap-does-it.md @@ -44,5 +44,8 @@ There's a couple of preliminary steps before core Bootstrap code is reached: 3. [`src/bootstrap/bootstrap.py`](https://github.com/rust-lang/rust/blob/master/src/bootstrap/bootstrap.py) 4. [`src/bootstrap/src/bin/main.rs`](https://github.com/rust-lang/rust/blob/master/src/bootstrap/src/bin/main.rs) +See [src/bootstrap/README.md](https://github.com/rust-lang/rust/blob/master/src/bootstrap/README.md) +for a more specific description of the implementation details. + [`Step`]: https://doc.rust-lang.org/nightly/nightly-rustc/bootstrap/core/builder/trait.Step.html [`Builder::ensure`]: https://doc.rust-lang.org/nightly/nightly-rustc/bootstrap/core/builder/struct.Builder.html#method.ensure