Add skip() method to Source interface to allow efficient seeks#14291
Add skip() method to Source interface to allow efficient seeks#14291
Conversation
This allows FdSource to efficiently skip data we don't care about.
|
@edolstra tests seem to fail with sanitizers |
|
I was just thinking about this when I did #14273 For now, it ended up passing GetNarBytes which special cases it but something like this would make it unecessary. |
|
@xokdvium Fixed. |
| sink.preallocateContents(size); | ||
|
|
||
| if (sink.skipContents) { | ||
| source.skip(size + (size % 8 ? 8 - (size % 8) : 0)); |
There was a problem hiding this comment.
This just does an alignUp with alignment = 8? Seems like a useful utility that we could move to libutil in a follow-up?
There was a problem hiding this comment.
(for my own edification) why does it align on 8 byte boundaries ?
Is too specific to x86_64 ?
What about other ISAs or 32bit.
There was a problem hiding this comment.
Okay you will hate this idea; but let's write a spec for https://kaitai.io/#what-is-it for fun as a side-project.
There was a problem hiding this comment.
Yes, that would be useful as a utility.
There was a problem hiding this comment.
Okay you will hate this idea; but let's write a spec for https://kaitai.io/#what-is-it for fun as a side-project.
https://chatgpt.com/share/68f6a47c-add8-800c-9bcc-bf17799ea0f3 hope it can be improved! from a quickest of glances that has some cruft that ought not to be there
There was a problem hiding this comment.
I'll work on this tonight if you want to hack on it.
There was a problem hiding this comment.
I can't join tonight, but yes I do think that would be a nice complement to the JSON Schema stuff @roberth and I have been doing.
4 participants
Motivation
Taken from DeterminateSystems#232.
When listing the contents of a NAR or nario, if the input is a seekable file descriptor, we can save a lot of time by seeking over the bytes that we're not interested in. For instance, this allows speeding up
nix nario list --no-contentsfrom 7.42s to 0.42s on a cold page cache for a 15 GB system closure.In a followup PR this will allow a cleaner alternative to #14273.
Context
Add 👍 to pull requests you find important.
The Nix maintainer team uses a GitHub project board to schedule and track reviews.