Hermetic nixpkgs_cc_toolchain#5976
Conversation
aherrmann-da
force-pushed
the
hermetic-cc-toolchain
branch
4 times, most recently
from
fb972e2 to
f61232f
Compare
cocreature
left a comment
cocreature
left a comment
There was a problem hiding this comment.
Great work, thank you so much!
| buildEnv { | ||
| name = "bazel-cc-toolchain"; | ||
| paths = [ customStdenv.cc ] ++ (if stdenv.isDarwin then [ ] else [ binutils ]); | ||
| paths = [ customStdenv.cc ] ++ (if stdenv.isDarwin then [ darwin.binutils ] else [ binutils ]); |
There was a problem hiding this comment.
Is this necessary now? For what?
There was a problem hiding this comment.
Yes, the old toolchain just picked /usr/bin/strip and friends. To make these parts hermetic as well, we want to use nixpkgs provided tools. On MacOS one use to use darwin.binutils instead of binutils. The latter includes a bin/strip that chokes on MACH-O binaries. See https://github.com/NixOS/nixpkgs-channels/blob/10100a97c8964e82b30f180fda41ade8e6f69e41/pkgs/os-specific/darwin/binutils/default.nix#L32
| +Use `nixpkgs_cc_configure_hermetic` instead. | ||
| + | ||
| +This uses Bazel's autoconfigure toolchain under the hood, which is | ||
| +inhermetic. In particular, system include directories specified in the |
There was a problem hiding this comment.
The description says “overriding this autodetection makes builds more hermetic” while here you then (correctly) specify that it is “inhermetic”. While that’s not really contradictory it does sound a bit confusing (no need to address this here but maybe worth keeping in mind for the rules_nixpkgs PR).
There was a problem hiding this comment.
Yeah, the docstrings could use a second pass. It is more hermetic than the builtin autodetection in that at least it doesn't use whatever cc is in PATH but the one provided by nixpkgs. However, it is still inhermetic in that it is influenced by some environment variables.
| + Returns: | ||
| + The string with all instances of `$(location )` replaced by paths. | ||
| + """ | ||
| + num = string.count("$(location ") |
There was a problem hiding this comment.
This looks like it is inconsistent with how Bazel handles this kind of stuff where you have to escape the $ if you want it to not have special meaning. Probably not a big deal.
There was a problem hiding this comment.
I don't follow. We do want $(location ) to have special meaning, just that here we define that special meaning with _expand_location. Note $(location ) expension is not automatic, a rule has to invoke ctx.expand_location explicitly. However, here we are within a repository rule, so ctx.expand_location is not available to us.
There was a problem hiding this comment.
What I mean is that only $(location …) has special meaning. In other contexts in Bazel where $(location …) has special meaning, everything starting with $ has special meaning and you have to escape something like $abc whereas here you do not need the escaping.
There was a problem hiding this comment.
Oh, I see, yes that would be more consistent. location expansion is not enabled by default, so it's not a big deal here. But, yes, this should be improved upstream.
CHANGELOG_BEGIN CHANGELOG_END
When using --incompatible_enable_cc_toolchain_resolution instead cc actions still depend on `external/local_config_cc/builtin_include_directory_paths` as well as `external/nixpkgs_cc_toolchain_config/builtin_include_directory_paths`.
The toolchain only considers `bin/cc` and having the other symlinks around could lead to confusion
aherrmann-da
force-pushed
the
hermetic-cc-toolchain
branch
from
ac4476f to
4408eca
Compare
|
I can confirm that this PR fixes rebuilding when within a Nix shell. Thanks! |
aherrmann-da
force-pushed
the
hermetic-cc-toolchain
branch
from
64e794d to
f0fb522
Compare
3 participants
So far we were using
nixpkgs_cc_toolchainofrules_nixpkgswhich uses Bazel's auto configured CC toolchain applied to a Nix provided compiler under the hood. This is not hermetic as the auto configure toolchain picks up header include paths defined in the environment. These may change for example within anix-shellenvironment. These builtin include paths affect the cache keys of all actions that depend on the cc toolchain and cause cache misses on mismatch.This updates rules_nixpkgs and uses the new
nixpkgs_cc_toolchain_hermetic(via patch until tweag/rules_nixpkgs#128 is merged). This new toolchain is hermetic in that it configures the cc toolchain during anix-buildwithin Nix's sandbox.Pull Request Checklist
CHANGELOG_BEGINandCHANGELOG_ENDtagsNOTE: CI is not automatically run on non-members pull-requests for security
reasons. The reviewer will have to comment with
/AzurePipelines runtotrigger the build.