-
Notifications
You must be signed in to change notification settings - Fork 12.9k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Set ninja=true by default #74922
Set ninja=true by default #74922
Conversation
(rust_highfive has picked a reviewer for you, use r? to override) |
This seems fine, but it looks like our CI does not have ninja installed -- presumably, that might mean that we could speed up CI builds as well, though they're all cached anyway (using sccache). I suspect we can at least get ninja relatively easily on the linux builders by adding a bunch of |
@Mark-Simulacrum Our CI should test the default configuration to the extent possible, so I think we should add ninja to the |
Okay, sounds good. I don't know how easy it is to obtain Ninja on macOS or Windows. It might be good to change the default for those platforms, at least for now. It doesn't look like Ninja is included in the GHA environments on either platform (https://github.com/actions/virtual-environments/blob/main/images/macos/macos-10.15-Readme.md or https://github.com/actions/virtual-environments/blob/main/images/win/Windows2019-Readme.md). |
@Mark-Simulacrum Our CI for Windows, at least, appears to install Ninja by default, so it should work fine there. |
@Mark-Simulacrum I disabled it for macOS CI (via the configure invocation), and installed it in all the CI |
72e815f
to
8c22f2b
Compare
Fixing... |
We do install ninja on Windows builders but really old version: https://github.com/rust-lang/rust/blob/b497e18995d6b6992f97512c6b86b5cb3f2f34f5/src/ci/scripts/install-ninja.sh |
8c22f2b
to
a9b52b4
Compare
@mati865 I saw that, yeah. We should probably update that, but if it works, we can do that separately. |
Looks like it works now. @Mark-Simulacrum Does this seem reasonable now that it passes CI? |
There was some discussion on Zulip about how Ninja may be somewhat hard to obtain on older linux (e.g. RHEL7) and the story on macOS and Windows seems like it'd be similar. I would like to change two things:
|
Within windows-gnu environment it's as simple as on Linux: |
@Mark-Simulacrum Building on older enterprise distributions seems like a case where it'd be reasonable for people to need to explicitly set On Windows at least, the comment immediately after where you linked indicates that we very much want to use ninja on Windows if at all possible. Ninja provides downloadable binaries for Windows, macOS, and Linux. I have no objection to making the default different for different host platforms, if it would otherwise be an imposition to people rather than a help. Is it actually hard to install Ninja on Windows or Mac, sufficiently so that most people would rather set |
In fact Windows has even more reasons to use Ninja than other platforms: Lines 108 to 119 in 8611e52
|
Downloadable binaries sounds about right to me. I'm happy with leaving the default on-by-default but let's definitely add a better error message. Maybe on Windows we'd want to give a message that's more cautionary (though maybe those problems have since been fixed, too). |
r=me with nit fixed, thanks! (As an aside, I wish we had an |
149de3d
to
6789762
Compare
@bors r+ |
📌 Commit 67897629fa74bd53b2de64282ce09d13539d3ba3 has been approved by |
⌛ Testing commit 67897629fa74bd53b2de64282ce09d13539d3ba3 with merge 9ffd86d7dc3810b3a7a5ce5cc4e75effb03de661... |
💔 Test failed - checks-actions |
Ah, that is indeed a reasonable theory. @bors r- @joshtriplett I am not sure of the best way to fix this. I suspect the easy fix is to just not do the ninja check if builder.config.dry_run is true... |
Ah, that does look like a better fix. Hopefully it works :) @bors r+ |
📌 Commit 8b501e3 has been approved by |
…rk-Simulacrum Set ninja=true by default Ninja substantially improves LLVM build time. On a 96-way system, using Make took 248s, and using Ninja took 161s, a 35% improvement. We already require a variety of tools to build Rust. If someone wants to build without Ninja (for instance, to minimize the set of packages required to bootstrap a new target), they can easily set `ninja=false` in `config.toml`. Our defaults should help people build Rust (and LLVM) faster, to speed up development.
⌛ Testing commit 8b501e3 with merge dac556e3380a47863b5797405312361e1ef7bd1f... |
The job Click to expand the log.
I'm a bot! I can only do what humans tell me to, so if this was not helpful or you have suggestions for improvements, please ping or otherwise contact |
💔 Test failed - checks-actions |
Looks like a spurious download failure. @bors retry |
☀️ Test successful - checks-actions, checks-azure |
@joshtriplett Why not use samurai instead of ninja, it is ninja-compatible but simpler so maybe it is faster? |
Ninja is more popular and available to quickly install basically everywhere. |
Clean up some of the pkgsrc Makefile, there's still lots in here that should just be deleted though. Switch SunOS to the illumos bootstrap by default. Version 1.48.0 (2020-11-19) ========================== Language -------- - [The `unsafe` keyword is now syntactically permitted on modules.][75857] This is still rejected *semantically*, but can now be parsed by procedural macros. Compiler -------- - [Stabilised the `-C link-self-contained=<yes|no>` compiler flag.][76158] This tells `rustc` whether to link its own C runtime and libraries or to rely on a external linker to find them. (Supported only on `windows-gnu`, `linux-musl`, and `wasi` platforms.) - [You can now use `-C target-feature=+crt-static` on `linux-gnu` targets.][77386] Note: If you're using cargo you must explicitly pass the `--target` flag. - [Added tier 2\* support for `aarch64-unknown-linux-musl`.][76420] \* Refer to Rust's [platform support page][forge-platform-support] for more information on Rust's tiered platform support. Libraries --------- - [`io::Write` is now implemented for `&ChildStdin` `&Sink`, `&Stdout`, and `&Stderr`.][76275] - [All arrays of any length now implement `TryFrom<Vec<T>>`.][76310] - [The `matches!` macro now supports having a trailing comma.][74880] - [`Vec<A>` now implements `PartialEq<[B]>` where `A: PartialEq<B>`.][74194] - [The `RefCell::{replace, replace_with, clone}` methods now all use `#[track_caller]`.][77055] Stabilized APIs --------------- - [`slice::as_ptr_range`] - [`slice::as_mut_ptr_range`] - [`VecDeque::make_contiguous`] - [`future::pending`] - [`future::ready`] The following previously stable methods are now `const fn`'s: - [`Option::is_some`] - [`Option::is_none`] - [`Option::as_ref`] - [`Result::is_ok`] - [`Result::is_err`] - [`Result::as_ref`] - [`Ordering::reverse`] - [`Ordering::then`] Cargo ----- Rustdoc ------- - [You can now link to items in `rustdoc` using the intra-doc link syntax.][74430] E.g. ``/// Uses [`std::future`]`` will automatically generate a link to `std::future`'s documentation. See ["Linking to items by name"][intradoc-links] for more information. - [You can now specify `#[doc(alias = "<alias>")]` on items to add search aliases when searching through `rustdoc`'s UI.][75740] Compatibility Notes ------------------- - [Promotion of references to `'static` lifetime inside `const fn` now follows the same rules as inside a `fn` body.][75502] In particular, `&foo()` will not be promoted to `'static` lifetime any more inside `const fn`s. - [Associated type bindings on trait objects are now verified to meet the bounds declared on the trait when checking that they implement the trait.][27675] - [When trait bounds on associated types or opaque types are ambiguous, the compiler no longer makes an arbitrary choice on which bound to use.][54121] - [Fixed recursive nonterminals not being expanded in macros during pretty-print/reparse check.][77153] This may cause errors if your macro wasn't correctly handling recursive nonterminal tokens. - [`&mut` references to non zero-sized types are no longer promoted.][75585] - [`rustc` will now warn if you use attributes like `#[link_name]` or `#[cold]` in places where they have no effect.][73461] - [Updated `_mm256_extract_epi8` and `_mm256_extract_epi16` signatures in `arch::{x86, x86_64}` to return `i32` to match the vendor signatures.][73166] - [`mem::uninitialized` will now panic if any inner types inside a struct or enum disallow zero-initialization.][71274] - [`#[target_feature]` will now error if used in a place where it has no effect.][78143] - [Foreign exceptions are now caught by `catch_unwind` and will cause an abort.][70212] Note: This behaviour is not guaranteed and is still considered undefined behaviour, see the [`catch_unwind`] documentation for further information. Internal Only ------------- These changes provide no direct user facing benefits, but represent significant improvements to the internals and overall performance of rustc and related tools. - [Building `rustc` from source now uses `ninja` by default over `make`.][74922] You can continue building with `make` by setting `ninja=false` in your `config.toml`. - [cg_llvm: `fewer_names` in `uncached_llvm_type`][76030] - [Made `ensure_sufficient_stack()` non-generic][76680] [78143]: rust-lang/rust#78143 [76680]: rust-lang/rust#76680 [76030]: rust-lang/rust#76030 [70212]: rust-lang/rust#70212 [27675]: rust-lang/rust#27675 [54121]: rust-lang/rust#54121 [71274]: rust-lang/rust#71274 [77386]: rust-lang/rust#77386 [77153]: rust-lang/rust#77153 [77055]: rust-lang/rust#77055 [76275]: rust-lang/rust#76275 [76310]: rust-lang/rust#76310 [76420]: rust-lang/rust#76420 [76158]: rust-lang/rust#76158 [75857]: rust-lang/rust#75857 [75585]: rust-lang/rust#75585 [75740]: rust-lang/rust#75740 [75502]: rust-lang/rust#75502 [74880]: rust-lang/rust#74880 [74922]: rust-lang/rust#74922 [74430]: rust-lang/rust#74430 [74194]: rust-lang/rust#74194 [73461]: rust-lang/rust#73461 [73166]: rust-lang/rust#73166 [intradoc-links]: https://doc.rust-lang.org/rustdoc/linking-to-items-by-name.html [`catch_unwind`]: https://doc.rust-lang.org/std/panic/fn.catch_unwind.html [`Option::is_some`]: https://doc.rust-lang.org/std/option/enum.Option.html#method.is_some [`Option::is_none`]: https://doc.rust-lang.org/std/option/enum.Option.html#method.is_none [`Option::as_ref`]: https://doc.rust-lang.org/std/option/enum.Option.html#method.as_ref [`Result::is_ok`]: https://doc.rust-lang.org/std/result/enum.Result.html#method.is_ok [`Result::is_err`]: https://doc.rust-lang.org/std/result/enum.Result.html#method.is_err [`Result::as_ref`]: https://doc.rust-lang.org/std/result/enum.Result.html#method.as_ref [`Ordering::reverse`]: https://doc.rust-lang.org/std/cmp/enum.Ordering.html#method.reverse [`Ordering::then`]: https://doc.rust-lang.org/std/cmp/enum.Ordering.html#method.then [`slice::as_ptr_range`]: https://doc.rust-lang.org/std/primitive.slice.html#method.as_ptr_range [`slice::as_mut_ptr_range`]: https://doc.rust-lang.org/std/primitive.slice.html#method.as_mut_ptr_range [`VecDeque::make_contiguous`]: https://doc.rust-lang.org/std/collections/struct.VecDeque.html#method.make_contiguous [`future::pending`]: https://doc.rust-lang.org/std/future/fn.pending.html [`future::ready`]: https://doc.rust-lang.org/std/future/fn.ready.html
Pkgsrc changes: * Compensate for files being moved around upstream. * Introduce optional, on-by-default semi-static building of cargo, using the internal curl and openssl sources. This reduces the dynamic dependencies of cargo and therefore the rust package itself. Ref. options.mk. * The 1.47.0 bootstrap kits have been re-built with the above option turned on, so no longer depends on curl or openssl from pkgsrc and/or from earlier OS or pkgsrc versions. This should hopefully fix installation of rust with non-default PREFIX, ref. PR#54453. Upstream changes: Version 1.48.0 (2020-11-19) ========================== Language -------- - [The `unsafe` keyword is now syntactically permitted on modules.][75857] This is still rejected *semantically*, but can now be parsed by procedural macros. Compiler -------- - [Stabilised the `-C link-self-contained=<yes|no>` compiler flag.][76158] This tells `rustc` whether to link its own C runtime and libraries or to rely on a external linker to find them. (Supported only on `windows-gnu`, `linux-musl`, and `wasi` platforms.) - [You can now use `-C target-feature=+crt-static` on `linux-gnu` targets.] [77386] Note: If you're using cargo you must explicitly pass the `--target` flag. - [Added tier 2\* support for `aarch64-unknown-linux-musl`.][76420] \* Refer to Rust's [platform support page][forge-platform-support] for more information on Rust's tiered platform support. Libraries --------- - [`io::Write` is now implemented for `&ChildStdin` `&Sink`, `&Stdout`, and `&Stderr`.][76275] - [All arrays of any length now implement `TryFrom<Vec<T>>`.][76310] - [The `matches!` macro now supports having a trailing comma.][74880] - [`Vec<A>` now implements `PartialEq<[B]>` where `A: PartialEq<B>`.][74194] - [The `RefCell::{replace, replace_with, clone}` methods now all use `#[track_caller]`.][77055] Stabilized APIs --------------- - [`slice::as_ptr_range`] - [`slice::as_mut_ptr_range`] - [`VecDeque::make_contiguous`] - [`future::pending`] - [`future::ready`] The following previously stable methods are now `const fn`'s: - [`Option::is_some`] - [`Option::is_none`] - [`Option::as_ref`] - [`Result::is_ok`] - [`Result::is_err`] - [`Result::as_ref`] - [`Ordering::reverse`] - [`Ordering::then`] Cargo ----- Rustdoc ------- - [You can now link to items in `rustdoc` using the intra-doc link syntax.][74430] E.g. ``/// Uses [`std::future`]`` will automatically generate a link to `std::future`'s documentation. See ["Linking to items by name"][intradoc-links] for more information. - [You can now specify `#[doc(alias = "<alias>")]` on items to add search aliases when searching through `rustdoc`'s UI.][75740] Compatibility Notes ------------------- - [Promotion of references to `'static` lifetime inside `const fn` now follows the same rules as inside a `fn` body.][75502] In particular, `&foo()` will not be promoted to `'static` lifetime any more inside `const fn`s. - [Associated type bindings on trait objects are now verified to meet the bounds declared on the trait when checking that they implement the trait.][27675] - [When trait bounds on associated types or opaque types are ambiguous, the compiler no longer makes an arbitrary choice on which bound to use.][54121] - [Fixed recursive nonterminals not being expanded in macros during pretty-print/reparse check.][77153] This may cause errors if your macro wasn't correctly handling recursive nonterminal tokens. - [`&mut` references to non zero-sized types are no longer promoted.][75585] - [`rustc` will now warn if you use attributes like `#[link_name]` or `#[cold]` in places where they have no effect.][73461] - [Updated `_mm256_extract_epi8` and `_mm256_extract_epi16` signatures in `arch::{x86, x86_64}` to return `i32` to match the vendor signatures.][73166] - [`mem::uninitialized` will now panic if any inner types inside a struct or enum disallow zero-initialization.][71274] - [`#[target_feature]` will now error if used in a place where it has no effect.][78143] - [Foreign exceptions are now caught by `catch_unwind` and will cause an abort.][70212] Note: This behaviour is not guaranteed and is still considered undefined behaviour, see the [`catch_unwind`] documentation for further information. Internal Only ------------- These changes provide no direct user facing benefits, but represent significant improvements to the internals and overall performance of rustc and related tools. - [Building `rustc` from source now uses `ninja` by default over `make`.][74922] You can continue building with `make` by setting `ninja=false` in your `config.toml`. - [cg_llvm: `fewer_names` in `uncached_llvm_type`][76030] - [Made `ensure_sufficient_stack()` non-generic][76680] [78143]: rust-lang/rust#78143 [76680]: rust-lang/rust#76680 [76030]: rust-lang/rust#76030 [70212]: rust-lang/rust#70212 [27675]: rust-lang/rust#27675 [54121]: rust-lang/rust#54121 [71274]: rust-lang/rust#71274 [77386]: rust-lang/rust#77386 [77153]: rust-lang/rust#77153 [77055]: rust-lang/rust#77055 [76275]: rust-lang/rust#76275 [76310]: rust-lang/rust#76310 [76420]: rust-lang/rust#76420 [76158]: rust-lang/rust#76158 [75857]: rust-lang/rust#75857 [75585]: rust-lang/rust#75585 [75740]: rust-lang/rust#75740 [75502]: rust-lang/rust#75502 [74880]: rust-lang/rust#74880 [74922]: rust-lang/rust#74922 [74430]: rust-lang/rust#74430 [74194]: rust-lang/rust#74194 [73461]: rust-lang/rust#73461 [73166]: rust-lang/rust#73166 [intradoc-links]: https://doc.rust-lang.org/rustdoc/linking-to-items-by-name.html [`catch_unwind`]: https://doc.rust-lang.org/std/panic/fn.catch_unwind.html [`Option::is_some`]: https://doc.rust-lang.org/std/option/enum.Option.html#method.is_some [`Option::is_none`]: https://doc.rust-lang.org/std/option/enum.Option.html#method.is_none [`Option::as_ref`]: https://doc.rust-lang.org/std/option/enum.Option.html#method.as_ref [`Result::is_ok`]: https://doc.rust-lang.org/std/result/enum.Result.html#method.is_ok [`Result::is_err`]: https://doc.rust-lang.org/std/result/enum.Result.html#method.is_err [`Result::as_ref`]: https://doc.rust-lang.org/std/result/enum.Result.html#method.as_ref [`Ordering::reverse`]: https://doc.rust-lang.org/std/cmp/enum.Ordering.html#method.reverse [`Ordering::then`]: https://doc.rust-lang.org/std/cmp/enum.Ordering.html#method.then [`slice::as_ptr_range`]: https://doc.rust-lang.org/std/primitive.slice.html#method.as_ptr_range [`slice::as_mut_ptr_range`]: https://doc.rust-lang.org/std/primitive.slice.html#method.as_mut_ptr_range [`VecDeque::make_contiguous`]: https://doc.rust-lang.org/std/collections/struct.VecDeque.html#method.make_contiguous [`future::pending`]: https://doc.rust-lang.org/std/future/fn.pending.html [`future::ready`]: https://doc.rust-lang.org/std/future/fn.ready.html
Ninja substantially improves LLVM build time. On a 96-way system, using
Make took 248s, and using Ninja took 161s, a 35% improvement.
We already require a variety of tools to build Rust. If someone wants to
build without Ninja (for instance, to minimize the set of packages
required to bootstrap a new target), they can easily set
ninja=false
in
config.toml
. Our defaults should help people build Rust (and LLVM)faster, to speed up development.