Skip to content

Conversation

@ghost
Copy link

@ghost ghost commented Jan 10, 2023

Description of changes

PR #208478 added a lot of documentation about which packages were rebuilt in each stage of the stdenv bootstrap. However nothing checks that these comments agree with reality; they can bitrot over time. This PR rewrites those comments as assertions, so they cannot bitrot.

This conversion did expose some ambiguity in our scheme for naming the stages. Suppose that pkgs.stdenv.name=="stdenv-stage4". Which of these is "the stage4 coreutils"?

pkgs.coreutils
pkgs.stdenv.__bootPackages.coreutils

The choice is arbitrary, and both choices have confusing corner cases. We should revisit this at some point.

Things done
  • Built on platform(s)
    • x86_64-linux
    • powerpc64le-linux
    • aarch64-linux
  • Fits CONTRIBUTING.md.

@ghost ghost requested review from Ericson2314 and matthewbauer as code owners January 10, 2023 11:05
@github-actions github-actions bot added the 6.topic: stdenv Standard environment label Jan 10, 2023
@ghost ghost mentioned this pull request Jan 10, 2023
4 tasks
@ofborg ofborg bot added 8.has: clean-up This PR removes packages or removes other cruft 10.rebuild-darwin: 0 This PR does not cause any packages to rebuild on Darwin. 10.rebuild-linux: 0 This PR does not cause any packages to rebuild on Linux. labels Jan 10, 2023
@ghost
Copy link
Author

ghost commented Jan 13, 2023

@trofi, I think this conversion-to-asserts caught a small mistake in #208478 -- the descriptions of the stages were all correct, but were shifted forward by one stdenv stage. Does this sound right?

@trofi
Copy link
Contributor

trofi commented Jan 13, 2023

It probably depends on how one looks at the stage's description. If stage's description says "resulting stage1 stdenv: binutils from nixpkgs" it can mean 2 things:

  1. this stdenv is constructed with rebuilt binutils in stdenv.cc.bintools (n)
  2. this stage provides a rebuilt by current stdenv binutils (or binutils-unwrapped) attribute for future use in next stdenv.cc.bintools (n+1)

I think I used 1. definition as I validated it by checking stdenv -> $pkg arrow. But might have mixed up with 2. as well.

Just to trace through one example: if we pick bootsrrap-stage2 it says resulting stage2 stdenv: binutils: from nixpkgs.

Checking:

# making sure it's stage2:
$ nix-instantiate -A stdenv.__bootPackages.stdenv.__bootPackages.stdenv.__bootPackages.stdenv
/nix/store/p95wb46d5bwzc9g0sr2i4dli5m9fsmmk-bootstrap-stage2-stdenv-linux.drv

$ nix-store --query --graph $(nix-instantiate -A stdenv.__bootPackages.stdenv.__bootPackages.stdenv.__bootPackages.stdenv) | grep 'stdenv.* -> .*binutils-2.39.drv'
"v14npkxf15hrzzl0a79dqvj5mzcvdc49-bootstrap-stage1-stdenv-linux.drv" -> "psjzwzbiawhg02h2by803micrd6n2y2m-binutils-2.39.drv" [color = "black"];

I think that checks out fine: stage2 itself pulls in new binutils here. If we validate bootstrap-stage1 stdenv closure it will not be there. Looks OK.

@ghost
Copy link
Author

ghost commented Jan 14, 2023

It probably depends on how one looks at the stage's description.

Oh I see. If pkgs.stdenv.name=="stdenv-stage4", then which of these is "the stage4 coreutils"?

pkgs.coreutils
pkgs.stdenv.__bootPackages.coreutils

The choice is arbitrary, and both choices have confusing corner cases.

I've updated the commit message.

@ghost
Copy link
Author

ghost commented Jan 14, 2023

The choice is arbitrary

Somehow this reminds me of @trofi's avatar artwork. 😄

@ghost
Copy link
Author

ghost commented Jan 15, 2023

Resolved merge conflict.

@ghost
Copy link
Author

ghost commented Jan 16, 2023

This PR has been absorbed into #209870, which is now ready for review.

@ghost ghost closed this Jan 16, 2023
@ghost ghost reopened this Feb 13, 2023
@ghost ghost marked this pull request as draft February 13, 2023 21:29
@ghost ghost marked this pull request as ready for review February 13, 2023 21:54
PR #208478 added a lot of documentation about which packages were
rebuilt in each stage of the stdenv bootstrap.  However nothing
checks that these comments agree with reality; they can bitrot over
time.  This PR rewrites those comments as assertions, so they cannot
bitrot.

This conversion did expose some ambiguity in our scheme for naming
the stages.   Suppose that `pkgs.stdenv.name=="stdenv-stage4", then
which of these is "the stage4 coreutils"?

```
pkgs.coreutils
pkgs.stdenv.__bootPackages.coreutils
```

The choice is arbitrary, and both choices have confusing corner
cases.  We should revisit this at some point.
@ghost
Copy link
Author

ghost commented Feb 21, 2023

Squashed.

@ghost ghost requested review from trofi and removed request for Ericson2314 and matthewbauer February 21, 2023 07:27
@trofi trofi merged commit 3057968 into NixOS:staging Feb 21, 2023
@SuperSandro2000
Copy link
Member

I just wanted to say that I really that we add more safe guards to the bootstrapping process to make it less error prone for changes.

ConnorBaker pushed a commit to nixos-cuda/cuda-legacy that referenced this pull request Apr 14, 2025
 #### Summary

By default, when you type `make`, GCC will compile itself three
times.  This PR inhibits that behavior by configuring GCC with
`--disable-bootstrap`, and reimplements the triple-rebuild using
Nix rather than `make`/`sh`.

 #### Immediate Benefits

- Allow `gcc11` and `gcc12` on `aarch64` (without needing new
  `bootstrapFiles`)
- Faster stdenv rebuilds: the third compilation of gcc
  (i.e. stageCompare) is no longer a `drvInput` of the final stdenv.
  This allows Nix to build stageCompare in parallel with the rest of
  nixpkgs instead of in series.
- No more copying `libgcc_s` out of the bootstrap-files or other
  derivations
- No more Frankenstein compiler: the final gcc and the libraries it
  links against (mpfr, mpc, isl, glibc) are all built by the same
  compiler (xgcc) instead of a mixture of the bootstrapFiles'
  compiler and xgcc.
- No more [static lib{mpfr,mpc,gmp,isl}.a hack]
- Many other small `stdenv` hacks eliminated
- `gcc` and `clang` share the same codepath for more of `cc-wrapper`.

 #### Future Benefits

- This should allow using a [foreign] `bootstrap-files` so long as
  `hostPlatform.canExecute bootstrapFiles`.
- This should allow each of the libraries that ship with `gcc`
  (lib{backtrace, atomic, cc1, decnumber, ffi, gomp, iberty,
  offloadatomic, quadmath, sanitizer, ssp, stdc++-v3, vtv}) to be
  built in separate (one-liner) derivations which `inherit src;`
  from `gcc`, much like NixOS/nixpkgs#132343

 #### Incorporates

- NixOS/nixpkgs#210004
- NixOS/nixpkgs#36948 (unreverted)
- NixOS/nixpkgs#210325
- NixOS/nixpkgs#210118
- NixOS/nixpkgs#210132
- NixOS/nixpkgs#210109
- NixOS/nixpkgs#213909
- NixOS/nixpkgs#216136
- NixOS/nixpkgs#216237
- NixOS/nixpkgs#210019
- NixOS/nixpkgs#216232
- NixOS/nixpkgs#216016
- NixOS/nixpkgs#217977
- NixOS/nixpkgs#217995

 #### Closes

- Closes #108305
- Closes #108111
- Closes #201254
- Closes #208412

 #### Credits

This project was made possible by three important insights, none of
which were mine:

1. @Ericson2314 was the first to advocate for this change, and
   probably the first to appreciate its advantages.  Nix-driven
   (external) bootstrap is "cross by default".

2. @trofi has figured out a lot about how to get gcc to not mix up
   the copy of `libstdc++` that it depends on with the copy that it
   builds, by moving the `bootstrapFiles`' `libstdc++` into a
   [versioned directory].  This allows a Nix-driven bootstrap of gcc
   without the final gcc would still having references to the
   `bootstrapFiles`.

3. Using the undocumented variable [`user-defined-trusted-dirs`]
   when building glibc.  When glibc `dlopen()`s `libgcc_s.so`, it
   uses a completely different and totally special set of rules for
   finding `libgcc_s.so`.  This trick is the only way we can put
   `libgcc_s.so` in its own separate outpath without creating
   circular dependencies or dependencies on the bootstrapFiles.  I
   would never have guessed to use this (or that it existed!) if it
   were not for a [comment in guix] which @Mic92 [mentioned].

My own role in this PR was basically: being available to go on a
coding binge at an opportune moment, so we wouldn't waste a
[crisis].

[aarch64-compare-ofborg]: https://github.com/NixOS/nixpkgs/pull/209870/checks?check_run_id=10662822938
[amd64-compare-ofborg]: https://github.com/NixOS/nixpkgs/pull/209870/checks?check_run_id=10662825857
[nonexistent sysroot]: NixOS/nixpkgs#210004
[versioned directory]: NixOS/nixpkgs#209054
[`user-defined-trusted-dirs`]: https://sourceware.org/legacy-ml/libc-help/2013-11/msg00026.html
[comment in guix]: https://github.com/guix-mirror/guix/blob/5e4ec8218142eee8e6e148e787381a5ef891c5b1/gnu/packages/gcc.scm#L253
[mentioned]: NixOS/nixpkgs#210112 (comment)
[crisis]: NixOS/nixpkgs#108305
[foreign]: NixOS/nixpkgs#170857 (comment)
[static lib{mpfr,mpc,gmp,isl}.a hack]: https://github.com/NixOS/nixpkgs/blob/2f1948af9c984ebb82dfd618e67dc949755823e2/pkgs/stdenv/linux/default.nix#L380
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

6.topic: stdenv Standard environment 8.has: clean-up This PR removes packages or removes other cruft 10.rebuild-darwin: 0 This PR does not cause any packages to rebuild on Darwin. 10.rebuild-linux: 0 This PR does not cause any packages to rebuild on Linux.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants