Skip to content

Deduplicate listNar and MemorySourceAccessor::File#14598

Merged
Ericson2314 merged 4 commits intomasterfrom
nar-listing-dedup
Nov 20, 2025
Merged

Deduplicate listNar and MemorySourceAccessor::File#14598
Ericson2314 merged 4 commits intomasterfrom
nar-listing-dedup

Conversation

@Ericson2314
Copy link
Member

Motivation

listNar did the not-so-pretty thing of going straight to JSON. Now it uses MemorySourceAccessor::File, or rather variations of it, to go to a C++ data type first, and only JSON second.

Context

To accomplish this we add some type parameters to the File data type. Actually, we need to do two rounds of this, because shallow NAR listings. There is FileT and DirectoryT accordingly.

The above descriptions are from the last commit, before that I have some tiny cleanups which should be self-explanatory. Review commit-by-commit to get better diffs.


Add 👍 to pull requests you find important.

The Nix maintainer team uses a GitHub project board to schedule and track reviews.

@Ericson2314 Ericson2314 requested a review from xokdvium November 20, 2025 01:47
@github-actions github-actions bot added new-cli Relating to the "nix" command store Issues and pull requests concerning the Nix store labels Nov 20, 2025
Copy link
Contributor

@xokdvium xokdvium left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There seems to be some tricky changes to the serialisation logic around handling of nullopt values. Looks like a pretty big footgun if we are not very careful. A missing key is very different from zero value.

Under which circumstances can st.narOffset and st.fileSize can be nullopt even?

@xokdvium
Copy link
Contributor

My comments might turn out inconsequential, but that seems like a slight behavior change. Not sure whether it matters in practice though. Just want to better understand what's going on there

This matches the "NAR Listing" JSON format, and also helps distinguish
from regular file contents.

Why we want to match that will become clear in the next comments, when
we will in fact use (variations of) this data type for NAR listings.
File-system-object-layer functionality doesn't depend on store-layer
concets, and therefore doesn't need to live inside there.
`listNar` did the not-so-pretty thing of going straight to JSON. Now it
uses `MemorySourceAccessor::File`, or rather variations of it, to go to
a C++ data type first, and only JSON second.

To accomplish this we add some type parameters to the `File` data type.
Actually, we need to do two rounds of this, because shallow NAR
listings. There is `FileT` and `DirectoryT` accordingly.
Copy link
Contributor

@xokdvium xokdvium left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Seems equivalent to me now.

@Ericson2314 Ericson2314 added this pull request to the merge queue Nov 20, 2025
Merged via the queue into master with commit 50407ab Nov 20, 2025
20 checks passed
@Ericson2314 Ericson2314 deleted the nar-listing-dedup branch November 20, 2025 21:49
@edolstra edolstra mentioned this pull request Dec 9, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

new-cli Relating to the "nix" command store Issues and pull requests concerning the Nix store

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants