Store provenance info in the NAR info disk cache

[edolstra]

Motivation

Note: this bumps the cache version to binary-cache-v8.sqlite.

Context

Summary by CodeRabbit

• New Features

  • Provenance is now stored with NAR metadata and surfaced from the local cache for improved traceability.
  • Database schema upgraded to version 8 to support provenance fields; provenance handling is behind an experimental feature toggle.

• Tests

  • Added functional tests that validate provenance is recorded and retrieved across HTTP and cached store workflows, including copy-from-cache scenarios.

[coderabbitai]

📝 Walkthrough

Walkthrough

Adds provenance storage and plumbing to the NAR info disk cache: schema bumped to binary-cache-v8.sqlite with a new provenance column; SQL, lookup, and upsert paths updated to read/write provenance JSON guarded by the experimental Provenance feature; functional tests added to validate provenance caching behavior.

Changes

Cohort / File(s)	Summary
Provenance Disk Cache Implementation src/libstore/nar-info-disk-cache.cc	Bumped DB path to binary-cache-v8.sqlite; added provenance column to NARs table and SQL statements; extend lookupNarInfo to parse provenance and upsertNarInfo to serialize/write provenance when Xp::Provenance is enabled.
Provenance Functional Tests tests/functional/flakes/provenance.sh	Force HTTP cache in test, add provenance assertions after build/store and after copying from binary cache, and cleanup env flag; verifies non-empty provenance in path-info outputs.

Sequence Diagram(s)

sequenceDiagram
    autonumber
    participant Client
    participant NarInfoDiskCache
    participant ExperimentalFeatureToggle
    participant SQLiteDB

    Client->>NarInfoDiskCache: request upsert or lookup (path info)
    NarInfoDiskCache->>ExperimentalFeatureToggle: isEnabled(Xp::Provenance)?
    ExperimentalFeatureToggle-->>NarInfoDiskCache: true/false
    alt Provenance enabled
        NarInfoDiskCache->>SQLiteDB: INSERT/SELECT ... provenance (JSON)
        SQLiteDB-->>NarInfoDiskCache: row with provenance (or ack)
        NarInfoDiskCache-->>Client: NarInfo with provenance populated
    else Provenance disabled
        NarInfoDiskCache->>SQLiteDB: INSERT/SELECT without provenance
        SQLiteDB-->>NarInfoDiskCache: row (no provenance)
        NarInfoDiskCache-->>Client: NarInfo without provenance
    end

Loading

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~20 minutes

Possibly related PRs

• Add subcommand 'nix provenance show' #340: touches provenance data flow and provenance display/typing — likely interacts with the DB provenance additions.
• BinaryCacheStore::queryPathInfoUncached(): Ensure noexcept #336: modifies NarInfo lookup handling and error paths — related to NarInfo population changes.

Suggested reviewers

• cole-h

Poem

🐇 I hop through rows of SQLite light,

I tuck JSON crumbs where NARs sleep tight,
A provenance trail left neat and bright,
V8 hums softly through day and night,
Hooray — the cache remembers right! 🥕

🚥 Pre-merge checks | ✅ 2 | ❌ 1

❌ Failed checks (1 warning)

Check name	Status	Explanation	Resolution
Docstring Coverage	⚠️ Warning	Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%.	Write docstrings for the functions missing them to satisfy the coverage threshold.

✅ Passed checks (2 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The pull request title 'Store provenance info in the NAR info disk cache' directly and clearly summarizes the main change: adding provenance information storage to the NAR info disk cache, which is the core objective of the changeset.

_{✏️ Tip: You can configure your own custom pre-merge checks in the settings.}

✨ Finishing touches

• 📝 Generate docstrings

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Post copyable unit tests in a comment
• Commit unit tests in branch nar-info-cache-provenance

---

No actionable comments were generated in the recent review. 🎉

🧹 Recent nitpick comments

src/libstore/nar-info-disk-cache.cc (1)

337-347: Avoid serializing provenance when the feature is disabled.

to_json_str() is evaluated even when Xp::Provenance is off. Consider precomputing only when enabled to avoid unnecessary work.

♻️ Suggested refactor

+                const bool provenanceEnabled = experimentalFeatureSettings.isEnabled(Xp::Provenance);
+                const bool hasProvenance = provenanceEnabled && info->provenance;
+                const std::string provenanceJson = hasProvenance ? info->provenance->to_json_str() : "";
+
                 state->insertNAR
                     .use()(cache.id)(hashPart) (std::string(info->path.name()))(
                         narInfo ? narInfo->url : "", narInfo != 0)(narInfo ? narInfo->compression : "", narInfo != 0)(
                         narInfo && narInfo->fileHash ? narInfo->fileHash->to_string(HashFormat::Nix32, true) : "",
                         narInfo && narInfo->fileHash)(
                         narInfo ? narInfo->fileSize : 0, narInfo != 0 && narInfo->fileSize)(info->narHash.to_string(
                         HashFormat::Nix32, true))(info->narSize)(concatStringsSep(" ", info->shortRefs()))(
                         info->deriver ? std::string(info->deriver->to_string()) : "",
                         (bool) info->deriver)(concatStringsSep(" ", info->sigs))(renderContentAddress(info->ca))(
-                        info->provenance ? info->provenance->to_json_str() : "",
-                        experimentalFeatureSettings.isEnabled(Xp::Provenance) && info->provenance)(time(0))
+                        provenanceJson, hasProvenance)(time(0))
                     .exec();

---

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

[coderabbitai]

Actionable comments posted: 2

🤖 Fix all issues with AI agents

In `@src/libstore/nar-info-disk-cache.cc`:
- Line 284: The code calls queryNAR.getStr(12) unguarded when assigning
narInfo->provenance via Provenance::from_json_str_optional, which can throw if
the column is NULL; change this to first check queryNAR.isNull(12) and only call
Provenance::from_json_str_optional(queryNAR.getStr(12)) when not null, otherwise
set narInfo->provenance to nullptr/optional empty value. Update the assignment
site where narInfo and provenance are used to handle the empty/null case
accordingly.
- Line 40: The DB schema change added a new column named provenance but existing
DBs created with the prior schema won’t get that column (CREATE TABLE IF NOT
EXISTS won’t alter existing tables), causing the SELECT ... provenance prepare
to fail; update the open/init path in nar-info-disk-cache.cc to detect and
migrate old DBs by either checking PRAGMA table_info(<table>) for the provenance
column or using PRAGMA user_version, and if missing run a lightweight ALTER
TABLE <table> ADD COLUMN provenance TEXT (or an equivalent migration SQL)
immediately after the schema exec so subsequent calls that SELECT provenance
succeed. Ensure the migration runs once on open before any prepares that
reference provenance and log or handle errors from the ALTER/MIGRATION step.

🧹 Nitpick comments (1)

tests/functional/flakes/provenance.sh (1)

90-95: Restore any pre-existing _NIX_FORCE_HTTP value to avoid cross-test leakage.

The script unsets the variable unconditionally, which can clobber a pre-set value in the test environment. Save and restore instead.

♻️ Suggested tweak

-export _NIX_FORCE_HTTP=1 # force use of the NAR info disk cache
+old_force_http="${_NIX_FORCE_HTTP-__unset__}"
+export _NIX_FORCE_HTTP=1 # force use of the NAR info disk cache
 ...
-unset _NIX_FORCE_HTTP
+if [[ "$old_force_http" == "__unset__" ]]; then
+  unset _NIX_FORCE_HTTP
+else
+  export _NIX_FORCE_HTTP="$old_force_http"
+fi

Also applies to: 131-131

[github-actions]

🚀 Deployed on https://698ba8100296b60530c9f320--determinate-nix.netlify.app