Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Build the stdlib from rust-src sources. #645

Merged
merged 3 commits into from
Apr 16, 2020

Conversation

eddyb
Copy link
Member

@eddyb eddyb commented Apr 9, 2020

See rust-lang/rust#69060 (comment) for background.

I definitely feels slower than previous. We might want to build libstd only once and place it in the sysroot (and I believe that would mean we can skip downloading rust-std altogether).

EDIT I moved from -Zbuild-std to a custom cargo build of the stdlib sources, no slowdown left.

However, we only need this for a couple weeks, until rust-lang/cargo#8073 (hopefully) reaches beta.
Then all of the hacks here will be unnecessary, since Rust itself will be built fully deterministically.

@eddyb
Copy link
Member Author

eddyb commented Apr 9, 2020

rust-lang/cargo#8073 landed, so we shouldn't need the "fake version" trick anymore.
(well, once rust-lang/rust updates src/tools/cargo to include that)

@eddyb
Copy link
Member Author

eddyb commented Apr 9, 2020

This took 5h09m to finish on a 24-core/48-thread server. Somehow I don't think we can do it this way :).

@eddyb
Copy link
Member Author

eddyb commented Apr 10, 2020

Tried the same run but without -Zbuild-std, it finished in 1h20.
So this PR makes perf runs almost four times slower. That's unacceptable.

@eddyb eddyb changed the title Use -Zbuilt-std to build libstd on the fly from rust-src sources. Use -Zbuild-std to build libstd on the fly from rust-src sources. Apr 10, 2020
@eddyb eddyb changed the title Use -Zbuild-std to build libstd on the fly from rust-src sources. Build the stdlib from rust-src sources. Apr 15, 2020
@eddyb
Copy link
Member Author

eddyb commented Apr 15, 2020

1h23 with the new approach. Also, for fun, I looked inside the rlibs built here and saw:

 00000000: 7275 7374 0000 0005 0047 bd06 2b72 7573  rust.....G..+rus
 00000010: 7463 2031 2e34 342e 302d 6e69 6768 746c  tc 1.44.0-nightl
-00000020: 7920 2838 3335 3432 3863 3335 2032 3032  y (835428c35 202
+00000020: 7920 2864 6637 3638 6335 6338 2032 3032  y (df768c5c8 202
 00000030: 302d 3034 2d31 3529 0463 6f72 65da 81b4  0-04-15).core...

(this is expected, since rustc outputs that into the rmeta format to be able to detect incompatibilities)

What's nice is that there are no other differences remaining.

Copy link
Member

@Mark-Simulacrum Mark-Simulacrum left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks! I want to take another look in the morning and merge and deploy then.

@eddyb
Copy link
Member Author

eddyb commented Apr 16, 2020

So the tests I did locally are with a recent noop change for which we now have official results too.

For "syn-opt clean":

Before After Δ%
perf.rust-lang.org 37,490,787,593 38,584,396,789 2.9170%
This PR (locally) 37,521,722,991 37,521,495,334 -0.0006%

So I believe this PR should remove the problem (but we should keep an eye on results for a while).

@eddyb
Copy link
Member Author

eddyb commented Apr 16, 2020

Oddly enough, this had a beneficial perf effect.
EDIT: looks like it's entirely from shaving ~10ms off of run_linker with locally-built stdlib somehow.
EDIT2: just remembered our locally-built stdlib has no debuginfo, that might be it.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants