Experimentally return treeHash on git and github fetchers

[Ericson2314]

Motivation

We already had some #if 0 for this, indicating a sort of "TODO" experiment. I would like to make this part of the git-hashing experimental feature.

Context

Priorities and Process

Add 👍 to pull requests you find important.

The Nix maintainer team uses a GitHub project board to schedule and track reviews.

[roberth]

This seems similarly unprincipled to #9025.
We have at least these kinds of tree hashes:

• derived from tarball
• taken from git commit
• derived after doing whatever transformation on the source
• maybe putting a tree hash into rev works

Some of these may coincide, but I'm pretty sure we'll have at least two, so the attributes should be qualified.

I don't think we should commit to anything before we know more about the new github: approach discussed in https://discourse.nixos.org/t/2024-03-25-nix-team-meeting-minutes-133/42167, especially because additions to the lock file are breaking changes.

[roberth]

• How is it useful?
• getSubmodulesAttr(input) doesn't tell us anything about whether the repo has submodules. false doesn't imply anything, and true only correlates.

[Ericson2314]

Yeah I agree it is lacking in principles. I want to take incremental steps towards supporting the "no submdules, tree hash is consistent through the entire process including of CA of store object" happy path, but I am not sure how. Per the end of the call maybe it is ignoring the per-fetcher stuff and starting with outputHashMode for fetchTree.

[thufschmitt]

Triaged during the Nix maintainers team meeting: Not ready yet.

Details

• @Ericson2314: explains some of the issues that he and Robert had discussed

• @roberth points at unresolved questions about just how much we want to trust fetchers today. Laziness complicates that

  • @Ericson2314: Agreed. E.g. if we are no longer always putting the entire source accessor in the store as one big store object, it is unclear when we would recheck a narHash in the lock file.

  • @roberth: We could use an in-memory overlay store for both .drvs (optimize the I/O latency) and as a store for lazy sources. This would allow the narHash to be computed without I/O and storage in the real store.

• @roberth would be nice to just store commit objects so we can faithfully use original hash for commits (root commit object) and submodules (nested commit object)

  • two options for the extra data

    • store with dot files in tree (like the case hack)
    • store in DB (store object, but not file system object)

    Awkward to pollute tree, but also awkward to store arbitrary amounts of extra data in DB.

• @Ericson2314: In any event, can side-step this unexplored design space for now by focusing on getting outputHashMode argument attributes on all the built-ins that need it (fetchTree, path, etc.). That is an easier-to-think-about next step, and will buy us some time to mull on these issues.

[nixos-discourse]

This pull request has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/2023-04-08-nix-team-meeting-136/42963/1