Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add support for artifact size profiling #87404

Merged
merged 2 commits into from
Oct 20, 2021

Conversation

rylev
Copy link
Member

@rylev rylev commented Jul 23, 2021

This adds support for profiling artifact file sizes (incremental compilation artifacts and query cache to begin with).

Eventually we want to track this in perf.rlo so we can ensure that file sizes do not change dramatically on each pull request.

This relies on support in measureme: rust-lang/measureme#169. Once that lands we can update this PR to not point to a git dependency.

This was worked on together with @michaelwoerister.

r? @wesleywiser

@rust-highfive rust-highfive added the S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. label Jul 23, 2021
@rust-log-analyzer

This comment has been minimized.

@inquisitivecrystal inquisitivecrystal added the T-compiler Relevant to the compiler team, which will review and decide on the PR/issue. label Aug 24, 2021
@bors
Copy link
Contributor

bors commented Sep 7, 2021

☔ The latest upstream changes (presumably #83214) made this pull request unmergeable. Please resolve the merge conflicts.

@jyn514
Copy link
Member

jyn514 commented Oct 3, 2021

@rylev the measureme PR was merged it looks like :) do you know when you'll get a chance to follow up?

@jyn514 jyn514 added S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Oct 3, 2021
@rylev
Copy link
Member Author

rylev commented Oct 5, 2021

@jyn514 I will be getting to this soon. We first wanted to make sure our changes to measureme and the related tooling don’t break perf.rlo. This should be easy to test, I just haven’t found the time yet.

@rylev rylev force-pushed the artifact-size-profiling branch from f32353e to 757f76e Compare October 7, 2021 13:08
@rylev
Copy link
Member Author

rylev commented Oct 7, 2021

An update: we're waiting on rust-lang/rustc-perf#1063 to be merged and to see that no issues arise from it.

Once that happens, we should be good to go here. One concern is that miri and rustc-ap-rustc_data_structures still use 0.9.2. I believe rustc-ap-rustc_data_structures auto-updates but I'm not sure.

@rylev rylev marked this pull request as ready for review October 8, 2021 15:50
@rylev
Copy link
Member Author

rylev commented Oct 8, 2021

This is good to go. I'd like to see this land, and then we can start working on rustc-perf to expose some of these stats in the perf.rlo UI.

encoder.flush()
let result = encoder.flush();
// FIXME(rylev): we hardcode the dep graph file name so we don't need a dependency on
// rustc_incremental just for that.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

rustc_incremental already depends on rustc_query_impl (directly or through rustc_middle). You can export a const to be accessed by rustc_incremental.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think that would actually introduce a circular dependency, right?

rustc_query_system -> rustc_incremental -> rustc_middle
         ^                                      |
         |                                      |
         +--------------------------------------+

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I believe the failure mode here if the dep graph file name changes is just that we have the wrong file name in the self-profile data which doesn't seem important enough to add the dependency, so this seems fine.

Copy link
Member

@wesleywiser wesleywiser left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just one question. r=me when you're ready to merge

encoder.flush()
let result = encoder.flush();
// FIXME(rylev): we hardcode the dep graph file name so we don't need a dependency on
// rustc_incremental just for that.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I believe the failure mode here if the dep graph file name changes is just that we have the wrong file name in the self-profile data which doesn't seem important enough to add the dependency, so this seems fine.

@wesleywiser
Copy link
Member

@bors r+

@bors
Copy link
Contributor

bors commented Oct 17, 2021

📌 Commit 757f76e has been approved by wesleywiser

@bors bors added S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. and removed S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. labels Oct 17, 2021
bors added a commit to rust-lang-ci/rust that referenced this pull request Oct 19, 2021
Rollup of 10 pull requests

Successful merges:

 - rust-lang#86479 (Automatic exponential formatting in Debug)
 - rust-lang#87404 (Add support for artifact size profiling)
 - rust-lang#87769 (Alloc features cleanup)
 - rust-lang#88789 (remove unnecessary bound on Zip specialization impl)
 - rust-lang#88860 (Deduplicate panic_fmt)
 - rust-lang#90009 (Make more `From` impls `const` (libcore))
 - rust-lang#90018 (Fix rustdoc UI for very long type names)
 - rust-lang#90025 (Revert rust-lang#86011 to fix an incorrect bound check)
 - rust-lang#90036 (Remove border-bottom from most docblocks.)
 - rust-lang#90060 (Update RELEASES.md)

Failed merges:

r? `@ghost`
`@rustbot` modify labels: rollup
@bors bors merged commit 3d95330 into rust-lang:master Oct 20, 2021
@rustbot rustbot added this to the 1.58.0 milestone Oct 20, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. T-compiler Relevant to the compiler team, which will review and decide on the PR/issue.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

10 participants