libstore: avoid PathSet allocation in derivation unparse#14823
libstore: avoid PathSet allocation in derivation unparse#14823Zaczero wants to merge 4 commits intoNixOS:masterfrom
Conversation
- Derivation::unparse() built a temporary PathSet via printStorePathSet(inputSrcs) only to immediately serialize it, which was already flagged as // FIXME: slow. - Emit the quoted inputSrcs paths directly into the output string instead, keeping the exact on-disk format while avoiding the intermediate container and extra iteration. - This reduces allocations/copies on a hot path during derivation (un)parsing; in local microbenchmarks (BM_UnparseRealDerivationFile/hello) it improves runtime by ~8%.
|
SGTM. I suppose we also skip the potential string escaping because derivation names cannot contain any characters that need escapes, right? |
|
@xokdvium I don't think the original was escaping anything (I didn't review whether it's correct or not, I assume it is). It did build a set of strings and then used printUnquotedStrings to print each as-is. The new approach simply iterates and prints directly without allocations in this hot path. |
Ericson2314
left a comment
There was a problem hiding this comment.
See also my review in #14825 (review)
Here's what I would like
-
Move
concatStringsfromutil.hhtostrings.hh. That is my mistake not doing that when I broke up most ofutil.hh. (Feel free to move some other things there while you are at it. -
Just as we have
basicSplitStringIntovsbasicSplitStringinstrings-inline.hh, we should have a version ofconcatStrings(and the underlyingconcatStringSep) that appends to an existing string. -
I think you can just change
printStorePathSetto do the appending way, and that will benefit most call sites, and for the remainder you can either just make the initial empty string manually, or make a small overload wrapper that does it.
How does that sound? That allows us to have or cake (perf) and eat it (code reuse) too.
Sounds good, I'll dig into it. |
StoreDirConfig::printStorePath() previously built an intermediate string via storeDir + "/" and then appended the store-path basename, which can trigger an extra allocation/copy.
Derivation::unparse() built a temporary PathSet via printStorePathSet(inputSrcs) only to immediately serialize it, which was already flagged as // FIXME: slow.
This reduces allocations/copies on a hot path during derivation (un)parsing; in local microbenchmarks (BM_UnparseRealDerivationFile/hello) it improves runtime by ~8%.
Motivation
Context
Add 👍 to pull requests you find important.
The Nix maintainer team uses a GitHub project board to schedule and track reviews.