Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Copy bin/* and lib/*.dylib files to stage0-sysroot #107956

Merged
merged 1 commit into from
Feb 17, 2023

Conversation

KittyBorgX
Copy link
Member

Fixes #101691

@rustbot
Copy link
Collaborator

rustbot commented Feb 12, 2023

Thanks for the pull request, and welcome! The Rust team is excited to review your changes, and you should hear from @ozkanonur (or someone else) soon.

Please see the contribution instructions for more information.

@rustbot rustbot added S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. T-bootstrap Relevant to the bootstrap subteam: Rust's build system (x.py and src/bootstrap) labels Feb 12, 2023
@KittyBorgX
Copy link
Member Author

r? @jyn514

@rustbot rustbot assigned jyn514 and unassigned onur-ozkan Feb 12, 2023
@jyn514
Copy link
Member

jyn514 commented Feb 12, 2023

I don't have time for reviews right now.

r? @ozkanonur

@rustbot rustbot assigned onur-ozkan and unassigned jyn514 Feb 12, 2023
@onur-ozkan
Copy link
Member

PR can't be passed in the pipelines currently. Please make sure all the pipelines gets passed successfully then we can review the changes.

@onur-ozkan onur-ozkan added S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Feb 12, 2023
@KittyBorgX
Copy link
Member Author

@ozkanonur I've fixed the breaking changes. Do I add a new commit or ammend it to the existing one?

@onur-ozkan
Copy link
Member

@ozkanonur I've fixed the breaking changes. Do I add a new commit or ammend it to the existing one?

Ammend it please :)

@jyn514
Copy link
Member

jyn514 commented Feb 12, 2023

@rustbot ready

@rustbot rustbot added S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. and removed S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. labels Feb 12, 2023
@onur-ozkan
Copy link
Member

onur-ozkan commented Feb 12, 2023

ref from the issue:

  1. x dist --stage 0 doesn't work:
thread 'main' panicked at 'could not read dir "/home/dawn/projects/rust/build/x86_64-unknown-linux-gnu/stage0-sysroot/bin": Os { code: 2, kind: NotFound, message: "No such file or directory" }', lib.rs:1573:25

Still doesn't work

 remotes/KittyBorgX/master $ ./x dist --stage 0

thread 'main' panicked at 'could not read dir "/home/nimda/devspace/personal/rust2/build/x86_64-unknown-linux-gnu/stage0-sysroot/lib/rustlib/x86_64-unknown-linux-gnu/codegen-backends": Os { code: 2, kind: NotFound, message: "No such file or directory" }', lib.rs:1522:25
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace

config:

# Includes one of the default files in src/bootstrap/defaults
profile = "compiler"
changelog-seen = 2

[build]
# patch-binaries-for-nix = true
# extended = true
#cargo-native-static = false
# sanitizers = false


[rust]
deny-warnings = false
download-rustc = false

Also, stage0-sysroot doesn't work as toolchain built from x build command.

Here is the output of running cargo with using stage0-sysroot as a toolchain on simple hello world project:

 ~/devspace/stuff/hello  stable $ cargo +issue-101691 run
error: process didn't exit successfully: `rustc -vV` (exit status: 127)
--- stderr
/home/nimda/.rustup/toolchains/issue-101691/bin/rustc: error while loading shared libraries: librustc_driver-c78d16bcb49dc5d0.so: cannot open shared object file: No such file or directory

@onur-ozkan onur-ozkan added S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Feb 12, 2023
@KittyBorgX
Copy link
Member Author

I see the issue. The original pr (#101711) broke when rustc and cargo was set which works fine with my PR.
Here's a screenshot:
image

As to codegen-backends which failed in the above comment:

 remotes/KittyBorgX/master $ ./x dist --stage 0
thread 'main' panicked at 'could not read dir "/home/nimda/devspace/personal/rust2/build/x86_64-unknown-linux-gnu/stage0-sysroot/lib/rustlib/x86_64-unknown-linux-gnu/codegen-backends": Os { code: 2, kind: NotFound, message: "No such file or directory" }', lib.rs:1522:25
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace

I followed : https://rust-lang.zulipchat.com/#narrow/stream/326414-t-infra.2Fbootstrap/topic/Copying.20binaries.20to.20stage0-sysroot/near/326813495

and didn't copy it. I will work on the same and update this pr.
Sidenote : On like 463, in my current pr, it is x.starts_with(sysroot_lib_dir.join("rustlib").into_boxed_path()) which passes the ci and passes the test locally but doesn't do the actual job that is intended and fails like how you mentioned in the second half of the above comment. But having !x.starts_with(sysroot_lib_dir.join("rustlib").into_boxed_path()) does the job of copying correctly but fails the CI and doesn't pass the tests. This has hit me with a roadblock which forces me to use another option and look into it deeply. Can I take some more time to work on this issue, if that's alright?

@onur-ozkan
Copy link
Member

I see the issue. The original pr (#101711) broke when rustc and cargo was set which works fine with my PR. Here's a screenshot: image

This doesn't work on x86_64.

~  $ rustc +issue-101691 --v
/home/nimda/.rustup/toolchains/issue-101691/bin/rustc: error while loading shared libraries: librustc_driver-c78d16bcb49dc5d0.so: cannot open shared object file: No such file or directory

Can I take some more time to work on this issue, if that's alright?

Please take your time

@KittyBorgX
Copy link
Member Author

@ozkanonur Can you help me? I've found a fix which works and passes the tests but I pulled the upstream changes and git(hub) added a commit. Is there any way I can delete the commit? Or should I close this PR and open a new one?

@onur-ozkan
Copy link
Member

@ozkanonur Can you help me? I've found a fix which works and passes the tests but I pulled the upstream changes and git(hub) added a commit. Is there any way I can delete the commit? Or should I close this PR and open a new one?

git reset --hard d8d95b5c0dbfea63b37a390252c5023fd7041bc8
git push --force

This will remove the diffs and commit you mentioned. If you want to merge these 2 commits, do soft reset instead.

@KittyBorgX KittyBorgX force-pushed the master branch 3 times, most recently from d8d95b5 to 982ab4d Compare February 13, 2023 11:12
@KittyBorgX
Copy link
Member Author

KittyBorgX commented Feb 13, 2023

Thank you so much! I've made the necessary changes and pushed it.

@bors
Copy link
Contributor

bors commented Feb 17, 2023

📌 Commit 6990ab9 has been approved by ozkanonur

It is now in the queue for this repository.

@bors bors added S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Feb 17, 2023
@KittyBorgX
Copy link
Member Author

@ozkanonur and @jyn514 Tysm for helping me throught the PR! I couldn't have done it without your help :D

@onur-ozkan
Copy link
Member

onur-ozkan commented Feb 17, 2023

congrats and thank you for your first contribution :)

bors added a commit to rust-lang-ci/rust that referenced this pull request Feb 17, 2023
…iaskrgr

Rollup of 6 pull requests

Successful merges:

 - rust-lang#107592 (Default `repr(C)` enums to `c_int` size)
 - rust-lang#107956 (Copy `bin/*` and `lib/*.dylib` files to `stage0-sysroot`)
 - rust-lang#108126 (fix a line, and do a consistency fix)
 - rust-lang#108144 (Add compiler-errors to a few more triagebot groups)
 - rust-lang#108149 (typo)
 - rust-lang#108154 (`BasicBlock::new(0)` -> `START_BLOCK` [no functional changes])

Failed merges:

r? `@ghost`
`@rustbot` modify labels: rollup
@bors bors merged commit 9956766 into rust-lang:master Feb 17, 2023
@rustbot rustbot added this to the 1.69.0 milestone Feb 17, 2023
@KittyBorgX
Copy link
Member Author

@bors try @rust-timer queue

@rust-timer

This comment has been minimized.

@albertlarsan68
Copy link
Member

@rust-timer build 89394208c043edd910fa6c950968ca75fcf349c0

@rust-timer

This comment has been minimized.

@rust-timer
Copy link
Collaborator

Finished benchmarking commit (89394208c043edd910fa6c950968ca75fcf349c0): comparison URL.

Overall result: no relevant changes - no action needed

Benchmarking this pull request likely means that it is perf-sensitive, so we're automatically marking it as not fit for rolling up. While you can manually mark this PR as fit for rollup, we strongly recommend not doing so since this PR may lead to changes in compiler perf.

@bors rollup=never
@rustbot label: -S-waiting-on-perf -perf-regression

Warning ⚠: The following benchmark(s) failed to build:

  • rustc

Instruction count

This benchmark run did not return any relevant results for this metric.

Max RSS (memory usage)

This benchmark run did not return any relevant results for this metric.

Cycles

Results

This is a less reliable metric that may be of interest but was not used to determine the overall result at the top of this comment.

mean range count
Regressions ❌
(primary)
- - 0
Regressions ❌
(secondary)
- - 0
Improvements ✅
(primary)
-2.2% [-2.2%, -2.2%] 1
Improvements ✅
(secondary)
- - 0
All ❌✅ (primary) -2.2% [-2.2%, -2.2%] 1

@albertlarsan68
Copy link
Member

@KittyBorgX ^
This PR does break rustc benches, but not the bootstrap timings, it has them.

@KittyBorgX
Copy link
Member Author

ah alright.
@albertlarsan68 thank you so much for running it :)

@Kobzol
Copy link
Contributor

Kobzol commented Feb 20, 2023

It's the same thing AFAIK, bootstrap timings = building rustc. The timings there are from a previous run, the "B column" is missing.

bors added a commit to rust-lang-ci/rust that referenced this pull request Jul 9, 2023
…crum

Copy stage0 `rustc` binaries to `stage0-sysroot`

This is basically a revival of rust-lang#101711 and rust-lang#107956, with an added check that the full sysroot will only be created if the original rustc comes from `stage0/bin`.

What is/should be tested:

- [x] `rustup toolchain link stage0` (new libstd is used correctly)
- [x]  `python3 x.py fmt dist --stage 0`
- [x] Custom rustc/cargo in `config.toml` (in this case this logic is ignored)
- [x]  Perfbot (try perf run has succeeded)
- [x] Real use case (rust-lang/backtrace-rs#542)

(Hopefully) fixes: rust-lang#101691

This is not the "end all, be all" solution to this problem, but as long as it resolves the basic use-case, and doesn't break perfbot, I say ship it. This code will probably be nuked anyway Soon™ because of the stage redesign.
RalfJung pushed a commit to RalfJung/miri that referenced this pull request Jul 10, 2023
Copy stage0 `rustc` binaries to `stage0-sysroot`

This is basically a revival of rust-lang/rust#101711 and rust-lang/rust#107956, with an added check that the full sysroot will only be created if the original rustc comes from `stage0/bin`.

What is/should be tested:

- [x] `rustup toolchain link stage0` (new libstd is used correctly)
- [x]  `python3 x.py fmt dist --stage 0`
- [x] Custom rustc/cargo in `config.toml` (in this case this logic is ignored)
- [x]  Perfbot (try perf run has succeeded)
- [x] Real use case (rust-lang/backtrace-rs#542)

(Hopefully) fixes: rust-lang/rust#101691

This is not the "end all, be all" solution to this problem, but as long as it resolves the basic use-case, and doesn't break perfbot, I say ship it. This code will probably be nuked anyway Soon™ because of the stage redesign.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. T-bootstrap Relevant to the bootstrap subteam: Rust's build system (x.py and src/bootstrap)
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Copy stage0 binaries into stage0-sysroot
9 participants