dontmerge: #209870 (external bootstrap) + #206907 (gcc12) #213931
Closed
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Co-authored-by: Weijia Wang <9713184+wegank@users.noreply.github.com>
On a native (host==build==target) build, `${with_build_sysroot}` can
be omitted. In that case, the line modified by this commit is
correct.
However, if `--with-build-sysroot=` is passed to `./configure`, then
`${with_build_sysroot}` will be used, similar to the cross case, and
the headers will be taken from:
${with_build_sysroot}${native_system_header_dir}
See:
https://gcc.gnu.org/git/?p=gcc.git;a=blob;f=gcc/configure.ac;h=62bc908b991ff0b700225aef32743cd3d7c4daaf;hb=HEAD#l2439
This commit adjusts the comment to account for the case where
`--with-build-sysroot` is passed to a native compiler.
#### Immediate Benefits
- Allow `gcc11` on `aarch64`
- No more copying `libgcc_s` out of the bootstrap-files or other
derivations
- No more [static `lib{mpfr,mpc,gmp,isl}.a`
hack](https://github.com/NixOS/nixpkgs/blob/2f1948af9c984ebb82dfd618e67dc949755823e2/pkgs/stdenv/linux/default.nix#L380)
- *Zero* additional `gcc` builds (stage1+stage2+stageCompare)
- The `gcc` derivation builds `gcc` once instead of three times.
- The libraries that are linked into the final `pkgs.gcc` (`mpfr`,
`mpc`, `gmp`, `isl`, `glibc`) are built by
`stdenv.__bootPkgs.gcc` rather than by the `bootstrapFiles`. No
more Frankenstein compiler!
- stageCompare runs **concurrently** with (not in series with)
with `stdenv`'s dependees.
- Many other `stdenv` hacks eliminated.
- `gcc` and `clang` share the same codepath for more of
`cc-wrapper`.
- Makes the cross and native codepaths much more similar --
another step towards "cross by default".
Note that *all* the changes in this PR are controlled by flags; no
old codepaths need to be removed until/if we're completely certain
that this is the right way to go.
#### Future Benefits
- This should allow using a [foreign] `bootstrap-files` so long as
`hostPlatform.canExecute bootstrapFiles`.
- There will be an "avalanche of simplification" when we set
`enableGccExternalBootstrap=true` and run dead code elimination.
It's really quite a huge amount of code that goes away.
Native-gcc has its own special codepath in so many places, while
cross-gcc and clang work the same way (and are much simpler).
- This should allow each of the libraries that ship with `gcc`
(`lib{backtrace,atomic,cc1,decnumber,ffi,gomp,iberty,offloadatomic,quadmath,sanitizer,ssp,stdc++-v3,vtv}`)
to be built in separate (one-liner) derivations which `inherit
src;` from `gcc`.
- Building `libstdc++-v3` in a separate derivation will eliminate
a lot of accidental-reference-to-the-`bootstrapFiles` landmines.
#### Incorporates
- #209054
- #210004
- #36948 (unreverted)
- #210325
- #210118
- #210132
- #210109
#### Closes
- Closes #208412
- Closes #108111
- Closes #108305
- Closes #201254
#### Build history
- First successful builds (stage1/stage2):
- powerpc64le-linux at 9c7e9ef256714455914180c0bcb434e014e5b75a
- x86_64-linux at 9c7e9ef256714455914180c0bcb434e014e5b75a
- aarch64-linux at 4d5bc7dabfbe1f8758559390e373b91dda9d401e
- First successful comparisons (stageCompare):
- at 81949cffa3272f8f9bdc8cfda8effb34be517d2f
- [aarch64-linux][aarch64-compare-ofborg]
- [x86\_64-linux][amd64-compare-ofborg]
#### Credits
This project was made possible by three important insights, none of
which were mine:
1. @Ericson2314 was the first to advocate for this change, and
probably the first to appreciate its advantages. External
bootstrap is "cross by default".
2. @trofi has figured out a lot about how to get gcc to not mix up
the copy of `libstdc++` that it depends on with the copy that it
builds. Now that gcc is written in C++, it depends on
`libstdc++`, builds a copy of `libstdc++`, and builds auxiliary
products (like `libplugin`) which depend on `libstdc++`. @trofi
developed two important techniques for keeping this straight: the
use of a [nonexistent sysroot] and moving the `bootstrapFiles`'
`libstdc++` into a [versioned directory]. Without these two
discoveries, external bootstrap would be impossible, because the
final gcc would still have references to the `bootstrapFiles`.
3. Using the undocumented variable [`user-defined-trusted-dirs`]
when building glibc. When glibc `dlopen()`s `libgcc_s.so`, it
uses a completely different and totally special set of rules for
finding `libgcc_s.so`. This trick is the only way we can put
`libgcc_s.so` in its own separate outpath without creating
circular dependencies or dependencies on the bootstrapFiles. I
would never have guessed to use this (or that it existed!) if it
were not for a [comment in guix] which @Mic92 [mentioned].
My own role in this PR was basically: being available to go on a
coding binge at an opportune moment, so we wouldn't waste a
[crisis].
[aarch64-compare-ofborg]: https://github.com/NixOS/nixpkgs/pull/209870/checks?check_run_id=10662822938
[amd64-compare-ofborg]: https://github.com/NixOS/nixpkgs/pull/209870/checks?check_run_id=10662825857
[nonexistent sysroot]: #210004
[versioned directory]: #209054
[`user-defined-trusted-dirs`]: https://sourceware.org/legacy-ml/libc-help/2013-11/msg00026.html
[comment in guix]: https://github.com/guix-mirror/guix/blob/5e4ec8218142eee8e6e148e787381a5ef891c5b1/gnu/packages/gcc.scm#L253
[mentioned]: #210112 (comment)
[crisis]: #108305
[foreign]: #170857 (comment)
GCC searches for headers in:
${with_build_sysroot}${native_system_header_dir}
Ordinarily we set
with_build_sysroot=/
native_system_header_dir="${lib.getDev stdenv.cc.libc}/include"
This commit causes the `xgcc` stage-1 compiler (not the final
compiler) to use a slightly different partitioning:
with_build_sysroot="${lib.getDev stdenv.cc.libc}"
native_system_header_dir="/include"
The concatenated result is the same, but `sysroot` is much easier to
override.
This reverts commit 250c9e5.
github-actions
bot
added
6.topic: emacs
Member
|
@ofborg build stdenv |
Member
|
It works! |
ofborg
bot
added
10.rebuild-darwin-stdenv
ghost
closed this
ghost
deleted the
This pull request was closed.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
For OfBorg. Merge commits used to bring #206907 up to latest
staging.