-
-
Notifications
You must be signed in to change notification settings - Fork 18k
express #208478 as assertions #210019
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
express #208478 as assertions #210019
Conversation
|
It probably depends on how one looks at the stage's description. If stage's description says "resulting stage1 stdenv: binutils from nixpkgs" it can mean 2 things:
I think I used Just to trace through one example: if we pick Checking: I think that checks out fine: |
Oh I see. If The choice is arbitrary, and both choices have confusing corner cases. I've updated the commit message. |
Somehow this reminds me of @trofi's avatar artwork. 😄 |
|
Resolved merge conflict. |
|
This PR has been absorbed into #209870, which is now ready for review. |
PR #208478 added a lot of documentation about which packages were rebuilt in each stage of the stdenv bootstrap. However nothing checks that these comments agree with reality; they can bitrot over time. This PR rewrites those comments as assertions, so they cannot bitrot. This conversion did expose some ambiguity in our scheme for naming the stages. Suppose that `pkgs.stdenv.name=="stdenv-stage4", then which of these is "the stage4 coreutils"? ``` pkgs.coreutils pkgs.stdenv.__bootPackages.coreutils ``` The choice is arbitrary, and both choices have confusing corner cases. We should revisit this at some point.
|
Squashed. |
|
I just wanted to say that I really that we add more safe guards to the bootstrapping process to make it less error prone for changes. |
#### Summary
By default, when you type `make`, GCC will compile itself three
times. This PR inhibits that behavior by configuring GCC with
`--disable-bootstrap`, and reimplements the triple-rebuild using
Nix rather than `make`/`sh`.
#### Immediate Benefits
- Allow `gcc11` and `gcc12` on `aarch64` (without needing new
`bootstrapFiles`)
- Faster stdenv rebuilds: the third compilation of gcc
(i.e. stageCompare) is no longer a `drvInput` of the final stdenv.
This allows Nix to build stageCompare in parallel with the rest of
nixpkgs instead of in series.
- No more copying `libgcc_s` out of the bootstrap-files or other
derivations
- No more Frankenstein compiler: the final gcc and the libraries it
links against (mpfr, mpc, isl, glibc) are all built by the same
compiler (xgcc) instead of a mixture of the bootstrapFiles'
compiler and xgcc.
- No more [static lib{mpfr,mpc,gmp,isl}.a hack]
- Many other small `stdenv` hacks eliminated
- `gcc` and `clang` share the same codepath for more of `cc-wrapper`.
#### Future Benefits
- This should allow using a [foreign] `bootstrap-files` so long as
`hostPlatform.canExecute bootstrapFiles`.
- This should allow each of the libraries that ship with `gcc`
(lib{backtrace, atomic, cc1, decnumber, ffi, gomp, iberty,
offloadatomic, quadmath, sanitizer, ssp, stdc++-v3, vtv}) to be
built in separate (one-liner) derivations which `inherit src;`
from `gcc`, much like NixOS/nixpkgs#132343
#### Incorporates
- NixOS/nixpkgs#210004
- NixOS/nixpkgs#36948 (unreverted)
- NixOS/nixpkgs#210325
- NixOS/nixpkgs#210118
- NixOS/nixpkgs#210132
- NixOS/nixpkgs#210109
- NixOS/nixpkgs#213909
- NixOS/nixpkgs#216136
- NixOS/nixpkgs#216237
- NixOS/nixpkgs#210019
- NixOS/nixpkgs#216232
- NixOS/nixpkgs#216016
- NixOS/nixpkgs#217977
- NixOS/nixpkgs#217995
#### Closes
- Closes #108305
- Closes #108111
- Closes #201254
- Closes #208412
#### Credits
This project was made possible by three important insights, none of
which were mine:
1. @Ericson2314 was the first to advocate for this change, and
probably the first to appreciate its advantages. Nix-driven
(external) bootstrap is "cross by default".
2. @trofi has figured out a lot about how to get gcc to not mix up
the copy of `libstdc++` that it depends on with the copy that it
builds, by moving the `bootstrapFiles`' `libstdc++` into a
[versioned directory]. This allows a Nix-driven bootstrap of gcc
without the final gcc would still having references to the
`bootstrapFiles`.
3. Using the undocumented variable [`user-defined-trusted-dirs`]
when building glibc. When glibc `dlopen()`s `libgcc_s.so`, it
uses a completely different and totally special set of rules for
finding `libgcc_s.so`. This trick is the only way we can put
`libgcc_s.so` in its own separate outpath without creating
circular dependencies or dependencies on the bootstrapFiles. I
would never have guessed to use this (or that it existed!) if it
were not for a [comment in guix] which @Mic92 [mentioned].
My own role in this PR was basically: being available to go on a
coding binge at an opportune moment, so we wouldn't waste a
[crisis].
[aarch64-compare-ofborg]: https://github.com/NixOS/nixpkgs/pull/209870/checks?check_run_id=10662822938
[amd64-compare-ofborg]: https://github.com/NixOS/nixpkgs/pull/209870/checks?check_run_id=10662825857
[nonexistent sysroot]: NixOS/nixpkgs#210004
[versioned directory]: NixOS/nixpkgs#209054
[`user-defined-trusted-dirs`]: https://sourceware.org/legacy-ml/libc-help/2013-11/msg00026.html
[comment in guix]: https://github.com/guix-mirror/guix/blob/5e4ec8218142eee8e6e148e787381a5ef891c5b1/gnu/packages/gcc.scm#L253
[mentioned]: NixOS/nixpkgs#210112 (comment)
[crisis]: NixOS/nixpkgs#108305
[foreign]: NixOS/nixpkgs#170857 (comment)
[static lib{mpfr,mpc,gmp,isl}.a hack]: https://github.com/NixOS/nixpkgs/blob/2f1948af9c984ebb82dfd618e67dc949755823e2/pkgs/stdenv/linux/default.nix#L380
Description of changes
PR #208478 added a lot of documentation about which packages were rebuilt in each stage of the stdenv bootstrap. However nothing checks that these comments agree with reality; they can bitrot over time. This PR rewrites those comments as assertions, so they cannot bitrot.
This conversion did expose some ambiguity in our scheme for naming the stages. Suppose that
pkgs.stdenv.name=="stdenv-stage4". Which of these is "the stage4 coreutils"?The choice is arbitrary, and both choices have confusing corner cases. We should revisit this at some point.
Things done