Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Reduce parent stat-per-file to stat-per-dir #1839

Merged
merged 1 commit into from
May 8, 2019

Conversation

rbtcollins
Copy link
Contributor

This is an installation microoptimisation for ~ 3% performance during
unpack performance: stat() calls are not free, involving path traversal,
ACL checking and so forth.

Before: 34s consistently to install rust-docs.
After: 32s-33s to install rust-docs.

A better still approach would be to handle creation errors by creating
the containing directories just in time, but that requires replacing
the tar extraction logic, which I haven't yet figured out how to layer
in cleanly.

This is an installation microoptimisation for ~ 3% performance during
unpack performance: stat() calls are not free, involving path traversal,
ACL checking and so forth.

Before: 34s consistently to install rust-docs.
After: 32s-33s to install rust-docs.

A better still approach would be to handle creation errors by creating
the containing directories just in time, but that requires replacing
the tar extraction logic, which I haven't yet figured out how to layer
in cleanly.
Copy link
Contributor

@kinnison kinnison left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me, I'm not bothered about the concern, as you say it's unlikely to be a problem since tarballs are usually semi-sorted thanks to filesystem layout.

Some(parent) => {
if !checked_parents.contains(parent) {
checked_parents.insert(parent.clone().to_owned());
// It would be nice to optimise this stat out, but the tar could be like so:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

One moderately neat way to do that would be to re-use the components iterator from above, joining and checking as you go; but this is sufficient for now

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That would result in stating every element from the top down, which is more than stating from the bottom up.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(Or I misunderstood :). Thanks for merging it!)

@kinnison kinnison merged commit 4e833c0 into rust-lang:master May 8, 2019
@rbtcollins rbtcollins deleted the stat-per-dir branch March 13, 2021 21:22
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants