cc-wrapper: deunify clang/gcc handling of -B flag

[ghost]

Closes #225779
Closes #225780

NIX_CFLAGS_COMPILE+="v"-then-nuke-refs-then-diff gives:

https://github.com/gcc-mirror/gcc/blob/5582ad0afb051a76231b2959487f4ef1746df283/gcc/gcc.cc#L549-L551

Which is implemented as:

https://github.com/gcc-mirror/gcc/blob/5582ad0afb051a76231b2959487f4ef1746df283/gcc/gcc.cc#L6423-L6435

Which comes from:

https://github.com/gcc-mirror/gcc/blob/5582ad0afb051a76231b2959487f4ef1746df283/gcc/gcc.cc#L4505-L4506

So the root problem is the first red-text block in the image above. Once that -B is in there, gcc will automatically add the offending -isystem flags due to the specs file.

So it's just that one extra -B that's causing all the trouble.

[ghost]

Running a rebuild with this.

It definitely fixes llvmPackages_13.compiler-rt-libc when applied selectively using wrapCCWith to compiler-rt-libc. This PR applies it globally.

I think the big lesson from this issue and #225220 is that the cc-wrapper codepath for non-self-bootstrapping compilers had a fair bit of clang-specific stuff in it. This PR and that PR carve out those bits and mark them as being strictly isClang-isms.

[trofi]

Yeah, that's unfortunate. I think nixpkgs uses -Blib/gcc/${target}/${version} only to redirect the lookup of crt*.o files. include/ change was unexpected. The rest is handled by -I/-isystem/-idirafter and -L flags.

Longer term we might need to avoid aliasing *.o / include by storing them in different directories. Or by tweaking gcc not to use -isystem injection for this case. Or by changing the order of how cc-wrapper injects -I/-isystem/-idirafter to play nicer with -B option (rare external firmware packages likely use it as well).

[ghost]

Fixed merge conflict.

[ghost]

Confirmed that this fixes #225779 and #225846.

[ghost]

I think nixpkgs uses -Blib/gcc/${target}/${version} only to redirect the lookup of crt*.o files.

At some point (well after the 23.05 release) I would like to try putting crt*.o either in their own output, or else in the libgcc output.

They are effectively compiler intrinsics just like libgcc... the main difference is that they are always statically linked (there is no .so version of them).

[ghost]

Added a revert of d0bb9ed.

@vcunat would you please check that this PR reverts all the temporary fixes in staging-next?

[vcunat]

Yes all of them. I'm not sure if the rocm-thunk one is related to this PR.

[ghost]

@ofborg eval

[ghost]

The branch this PR will merge in to does not cleanly evaluate

Well that's not good.

[ghost]

Yes all of them. I'm not sure if the rocm-thunk one is related to this PR.

It's kinda hard to figure out what that commit does since it doesn't have a PR... it just got lumped in with a 250-commit merge in #224806

[vcunat]

@ofborg eval

[vcunat]

I merged master to staging, which should hopefully fix evaluation, but the very same line also caused a conflict here.

[ghost]

@vcunat will this make it into the 23.05 release?

This causes a mass rebuild, so if it doesn't make the release it is unlikely to be backported and we'll be stuck with the wrapCCWith kludge until 23.11.

[vcunat]

There's still a whole month until schedule says restricting breaking changes on staging.

[matthewpi]

Reverting d0bb9ed caused rocm-thunk to fail with the same error again. I'm building my NixOS config using this PR at commit e2340a5b8595d40b93e4463d24992f34974bdc91 and so far I have the following build errors.

Details

Apologies for the image, I cannot copy text from the output without cancelling the remaining rebuilds.

---

Edit: after building the branch over the past two days, I am left with the following failures.

Details

nixos-config on master [!] 
❯ nix build .\#nixosConfigurations.matthew-desktop.config.system.build.toplevel --keep-going --keep-failed                          
warning: Git tree '/home/matthew/code/matthewpi/nixos-config' is dirty
warning: Git tree '/home/matthew/code/matthewpi/nixos-config' is dirty
note: keeping build directory '/tmp/nix-build-rocm-thunk-5.4.4.drv-2'
error: builder for '/nix/store/qz4qp88qhfhd0ksvn6sf17r8cq216cf6-rocm-thunk-5.4.4.drv' failed with exit code 1;
       last 10 log lines:
       > -- Checking for module 'libdrm'
       > --   Found libdrm, version 2.4.115
       > -- Checking for module 'libdrm_amdgpu'
       > --   Found libdrm_amdgpu, version 2.4.115
       > CMake Error at CMakeLists.txt:191 (find_library):
       >   Could not find LIBGCC using the following names: libgcc_s.so.1
       >
       > 
       > -- Configuring incomplete, errors occurred!
       > See also "/build/source/build/CMakeFiles/CMakeOutput.log".
       For full logs, run 'nix log /nix/store/qz4qp88qhfhd0ksvn6sf17r8cq216cf6-rocm-thunk-5.4.4.drv'.
error: 1 dependencies of derivation '/nix/store/42hjrci9c9ymrr9wmrlivclh22vspxfd-rocm-runtime-5.4.3.drv' failed to build
error: 1 dependencies of derivation '/nix/store/ighbsjqns1bsyp4gs21yi8jpd40lvppd-rocm-opencl-runtime-5.4.4.drv' failed to build
error: 1 dependencies of derivation '/nix/store/2hmylj8a7kvir51x5i6w4qd1mnn8hcns-rocm-opencl-icd-5.4.4.drv' failed to build
error: 2 dependencies of derivation '/nix/store/wbnlhlbrwjdi58j9m618d7if1k6zxg86-opengl-drivers.drv' failed to build
error: 1 dependencies of derivation '/nix/store/p1xld12pfnalqbiqyz118wzvkard13wn-nixos-tmpfiles.d.drv' failed to build
error: 1 dependencies of derivation '/nix/store/yigjd1akncjzlq0yc53jz5143vmrzfia-tmpfiles.d.drv' failed to build
note: keeping build directory '/tmp/nix-build-mangohud-0.6.8.drv-2'
error: builder for '/nix/store/9r4vwggxggpnq1vb11vli3h9yil4qdyn-mangohud-0.6.8.drv' failed with exit code 1;
       last 10 log lines:
       >       |                                                                                 ^~~~
       >       |                                                                                 ;
       > [46/66] Compiling C++ object src/libMangoHud.so.p/overlay.cpp.o
       > [47/66] Compiling C++ object src/libMangoHud.so.p/battery.cpp.o
       > [48/66] Compiling C++ object src/libMangoHud.so.p/gamepad.cpp.o
       > [49/66] Compiling C++ object subprojects/imgui-1.81/libimgui.a.p/imgui_widgets.cpp.o
       > [50/66] Compiling C++ object subprojects/spdlog-1.8.5/src/libspdlog.a.p/fmt.cpp.o
       > [51/66] Compiling C++ object subprojects/imgui-1.81/libimgui.a.p/imgui.cpp.o
       > [52/66] Compiling C++ object subprojects/spdlog-1.8.5/src/libspdlog.a.p/spdlog.cpp.o
       > ninja: build stopped: subcommand failed.
       For full logs, run 'nix log /nix/store/9r4vwggxggpnq1vb11vli3h9yil4qdyn-mangohud-0.6.8.drv'.
error: 1 dependencies of derivation '/nix/store/mgjrmr46svs28svhmawrhnyf6g200jg1-opengl32-nix-workaround.patch.drv' failed to build
error: 2 dependencies of derivation '/nix/store/yy0g9q16zaggbjn34jw75r9p2h181lgx-mangohud-0.6.8.drv' failed to build
error: 1 dependencies of derivation '/nix/store/62jy9jmnknk65dl650zns8r4bkqls5rr-bottles-unwrapped-51.5.drv' failed to build
error: 1 dependencies of derivation '/nix/store/h33ikl48hr4qrl4zm744xd1rlk8z1xd5-bottles-cli-usr-target.drv' failed to build
error: 1 dependencies of derivation '/nix/store/xz091yxhbk9qb1m5yimwp4amjjfgjwnz-bottles-usr-target.drv' failed to build
error: 1 dependencies of derivation '/nix/store/gd6sviywva5bmhdlg4xjbmnzzfiqqd2z-bottles-cli-fhs.drv' failed to build
error: 1 dependencies of derivation '/nix/store/fb6knqlb7rdka27jvf1sy5cv6hn4djpl-bottles-fhs.drv' failed to build
error: 1 dependencies of derivation '/nix/store/lg070b9zm88d787ia86va5rh65dlnxgm-bottles-bwrap.drv' failed to build
error: 1 dependencies of derivation '/nix/store/43a0x5db9r8lyjky2s7m55qzhnv90d67-bottles-cli-bwrap.drv' failed to build
error: 1 dependencies of derivation '/nix/store/blzklmjj0hq31l32p2fdfws20d4bsv16-bottles-cli.drv' failed to build
error: 1 dependencies of derivation '/nix/store/yfnkp593whw8387fvcsmcix0vhwkjmyg-bottles.drv' failed to build
error: 3 dependencies of derivation '/nix/store/zg7l64xgl9rlgm5j88wdkx7815k7dx79-bottles.drv' failed to build
error: 2 dependencies of derivation '/nix/store/xpngmbqmswaiyd8mciiihbm831k0clmm-home-manager-path.drv' failed to build
error: 1 dependencies of derivation '/nix/store/8bsvhx5z5164svdzrn2lnbmih671446j-hm_fontconfigconf.d10hmfonts.conf.drv' failed to build
error: 1 dependencies of derivation '/nix/store/y78bqjmiy58bb6z87p0dvml7rkiawk79-user-environment.drv' failed to build
error: 1 dependencies of derivation '/nix/store/25727hq7ilqdg5y8n6qhs32c89l4zi52-home-manager-files.drv' failed to build
error: 2 dependencies of derivation '/nix/store/klkwzq2f71vyfllxbzz2zjdcycgqamlr-home-manager-generation.drv' failed to build
error: 1 dependencies of derivation '/nix/store/xndx1c9f9fl8hdc9hx224dw8279qyrql-unit-home-manager-matthew.service.drv' failed to build
error: 1 dependencies of derivation '/nix/store/svggi2biq3z9zm94kzszywirq7xix3p3-system-units.drv' failed to build
error: 3 dependencies of derivation '/nix/store/0p9l78g3qdwzicj5z2j3llghliv637fw-etc.drv' failed to build
error: 1 dependencies of derivation '/nix/store/yl7xrs7w111nnplvfbrycwvcn7y1ypp7-nixos-system-matthew-desktop-23.05.20230412.e2340a5.drv' failed to build

Full build logs for failing derivations https://gist.github.com/matthewpi/647099ecaa6083abe64e11eef2506417

[ghost]

Reverting d0bb9ed caused rocm-thunk to fail with the same error again.

@matthewpi thank you for checking this. And for spending two days of builder time on it. This was my mistake, I should not have reverted that commit.

Yes all of them. I'm not sure if the rocm-thunk one is related to this PR.

@vcunat, sorry for not looking at this more closely.

I have dropped the revert from this PR.

[vcunat]

@matthewpi: mangohud was an earlier unrelated regression: #224485 (comment)

[matthewpi]

@matthewpi: mangohud was an earlier unrelated regression: #224485 (comment)

I wasn't sure, but luckily these are the only two failures I've noticed so far.

---

I'll look into building the new changes "tomorrow", hopefully without anymore issues.

[matthewpi]

Fixes the rocm and LLVM-related build failures I've had on staging.

[orivej]

Does this also fix building androidsdk_9_0? Recently it fails with:

auto-patchelf: 2 dependencies could not be satisfied
error: auto-patchelf could not satisfy dependency libgcc_s.so.1 wanted by emulator-check
error: auto-patchelf could not satisfy dependency libgcc_s.so.1 wanted by emulator

[vcunat]

No, this PR does not address auto-patchelf errors, though they were introduced by the PR with gcc bootstrapping changes. Packages not built on Hydra get easily missed. Individual packages are easy to fix (e.g. see ab1cc58), but maybe we should have a generic fix for autoPatchelfHook instead.

[ghost]

Does this also fix building androidsdk_9_0? Recently it fails with:

@orivej can you please open an issue with a bug report for this build failure (including a specific nixpkgs commit and exact nix-build or nix build command which failed)?

It's probably my fault. Please assign the issue to me and I will fix it.

but maybe we should have a generic fix for autoPatchelfHook instead.

I think so too:

• autoPatchelfHook does not understand dlopen() #225963

If @orivej's problem turns out to be due to this then that will be the second issue report, and I will start working on a general autoPatchelfHook fix. I'd like to have two test cases in order to do that (already have one).