Skip to content

add force-build-new-FODs.sh maintainer script#313248

Open
risicle wants to merge 1 commit intoNixOS:masterfrom
risicle:ris-force-build-new-fods
Open

add force-build-new-FODs.sh maintainer script#313248
risicle wants to merge 1 commit intoNixOS:masterfrom
risicle:ris-force-build-new-fods

Conversation

@risicle
Copy link
Contributor

@risicle risicle commented May 20, 2024

Description of changes

See the later discussions in NixOS/nix#969, NixOS/ofborg#68, NixOS/rfcs#171 around concerns that the nixos binary cache could be "seeded" with a malicious source package in a quiet under-reviewed corner of nixpkgs which then gets re-used in a more security-critical place through a hash confusion attack.

The following script can be used as a mitigation to this, enabling reviewers to check whether new or changed Fixed-Output-Derivations actually produce the purported output hash when not pulled from a cache. It actually works by producing a "tainted" copy of any new FODs with a random hash and checking what "got:" hash nix lists when the hashes fail to agree.

There are obvious synergies here with what ofborg does - particularly the most computationally intensive part, performing the full nixpkgs evaluation. However it appears that the ofborg team don't have the capacity to act on this now, so it's probably better to get a standalone copy of this script in more peoples' hands.

The script in fact uses a modified version of ofborg's outpaths.nix, along with a few other embedded sibling scripts. There are of course downsides with using embedded scripts, but I chose to go this way because it keeps the script very self-contained and means I don't have to solve problems like locating these other scripts at runtime.

Things done

  • Built on platform(s)
    • x86_64-linux
    • aarch64-linux
    • x86_64-darwin
    • aarch64-darwin
  • For non-Linux: Is sandboxing enabled in nix.conf? (See Nix manual)
    • sandbox = relaxed
    • sandbox = true
  • Tested, as applicable:
  • Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
  • Tested basic functionality of all binary files (usually in ./result/bin/)
  • 24.05 Release Notes (or backporting 23.05 and 23.11 Release notes)
    • (Package updates) Added a release notes entry if the change is major or breaking
    • (Module updates) Added a release notes entry if the change is significant
    • (Module addition) Added a release notes entry if adding a new NixOS module
  • Fits CONTRIBUTING.md.

Add a 👍 reaction to pull requests you find important.

this can be used to check whether newly-introduced FODs in, for
instance, a pull request, produce the purported output hash
despite that output hash/path possibly already existing in
a binary cache.
@risicle risicle requested a review from a team May 20, 2024 21:56
@ofborg ofborg bot added 10.rebuild-darwin: 0 This PR does not cause any packages to rebuild on Darwin. 10.rebuild-linux: 0 This PR does not cause any packages to rebuild on Linux. labels May 20, 2024
@tobiasBora
Copy link
Contributor

Thanks, I will try to give it a try when I've some time. Would be great to integrate it in nixpkgs review if ofborg is not going to run it automatically. But just wanted to drop a line saying that this is a nice to have and "easy" (compared to fixing nix directly) to implement mitigation, but a true fix would be to do it directly at nix's level as discussed in NixOS/nix#969 (comment)

For instance, I guess that this script would be unable to detect this kind of attack targeting a different architecture than the one of the current user?

@mweinelt mweinelt added the 1.severity: security Issues which raise a security issue, or PRs that fix one label May 21, 2024
@jonringer
Copy link
Contributor

How long does it take to run this? usually doing a nixpkgs-wide eval is pretty ram and time intensive for me.

@risicle
Copy link
Contributor Author

risicle commented Jun 19, 2024

😁

It takes just about as long as you suspect, which is why I was interested in piggy-backing on ofborg - it already does the required evals.

There's actually no limit to how long it can take when e.g. a common fetcher's hash changes - it can technically want to build quite a few non-FOD derivations in order to be able to build the FOD derivations. But ofborg itself isn't immune to this sort of thing and usually people ignore crazy ofborg results for staging PRs and the like.

It occurred to me the other day that this script would still have some value if it were simply run periodically against master. Might not catch things pre-merge, but it's better than the current situation.

@wegank wegank added the 2.status: stale https://github.com/NixOS/nixpkgs/blob/master/.github/STALE-BOT.md label Jan 2, 2025
@nixpkgs-ci nixpkgs-ci bot removed the 2.status: stale https://github.com/NixOS/nixpkgs/blob/master/.github/STALE-BOT.md label Jun 25, 2025
@nixpkgs-ci nixpkgs-ci bot added the 12.approvals: 1 This PR was reviewed and approved by one person. label Jan 16, 2026
@risicle
Copy link
Contributor Author

risicle commented Jan 17, 2026

This was written quite a while ago, it's worth checking it still works.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

1.severity: security Issues which raise a security issue, or PRs that fix one 10.rebuild-darwin: 0 This PR does not cause any packages to rebuild on Darwin. 10.rebuild-linux: 0 This PR does not cause any packages to rebuild on Linux. 12.approvals: 1 This PR was reviewed and approved by one person.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants