Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Provide a way to run cargo subcommands on all crates in the workspace #2878

Closed
euclio opened this issue Jul 14, 2016 · 33 comments
Closed

Provide a way to run cargo subcommands on all crates in the workspace #2878

euclio opened this issue Jul 14, 2016 · 33 comments

Comments

@euclio
Copy link
Contributor

euclio commented Jul 14, 2016

This is mentioned as a "future extension" in RFC 1525.

@alexcrichton
Copy link
Member

Thanks for the issue @euclio! This was indeed intended to have an issue. And to expand a bit the thought was to add --all flags to many subcommands so executions like:

cargo test --all

would test all the crates in a workspace, or

cargo publish --all

would publish all crates in a workspace that haven't already been published.

@DrRibosome
Copy link

just wondering if anyone is still working on this

@alexcrichton
Copy link
Member

@jdc2172 I don't believe the currently is, but if you'd like to take it feel free!

@retep998
Copy link
Member

cargo doc --all would be exceptionally useful.

@euclio
Copy link
Contributor Author

euclio commented Sep 30, 2016

I'd be interested in tackling this, but I think we need to hammer out which commands should get this flag.

Also, what sort of implementation approach should be taken?

@steveklabnik
Copy link
Member

cargo build and cargo test are the biggies for me.

@DrRibosome
Copy link

Cargo test definitely needs this.

Not sure cargo build is as high priority given that it already builds path
deps in the crate

On Fri, Sep 30, 2016, 6:32 PM Steve Klabnik [email protected]
wrote:

cargo build and cargo test are the biggies for me.


You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
#2878 (comment),
or mute the thread
https://github.com/notifications/unsubscribe-auth/ABHmfeh16DgMThxKntuIleFSaX_eVtI-ks5qvY34gaJpZM4JMpzT
.

@steveklabnik
Copy link
Member

Oh yeah. That's what I get for hasty commenting! On my project that uses workspaces, those are the only two sub commands I use, so I didn't think it through 😅😞

On Sep 30, 2016, 18:57 -0400, Jack Crawford [email protected], wrote:

Cargo test definitely needs this.

Not sure cargo build is as high priority given that it already builds path
deps in the crate

On Fri, Sep 30, 2016, 6:32 PM Steve Klabnik [email protected]
wrote:

cargo build and cargo test are the biggies for me.


You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
#2878 (comment),
or mute the thread
https://github.com/notifications/unsubscribe-auth/ABHmfeh16DgMThxKntuIleFSaX_eVtI-ks5qvY34gaJpZM4JMpzT
.


You are receiving this because you commented.
Reply to this email directly, view it on GitHub (#2878 (comment)), or mute the thread (https://github.com/notifications/unsubscribe-auth/AABsijei0_YDV9nu6hCIS8o69VRvBBEFks5qvZPkgaJpZM4JMpzT).

@alexcrichton
Copy link
Member

@euclio to me yeah test and doc are the biggies to cover here. I think publish would also be great but that has other tricky semantics we'd want to cover later.

@euclio
Copy link
Contributor Author

euclio commented Oct 18, 2016

@alexcrichton Trying to get my feet wet with test. What's the best plan of attack here? Modifying ops::compile to take a flag to compile all members of the workspace instead of the current?

@alexcrichton
Copy link
Member

@euclio yeah there's a few points where the -p flag is passed down, but that could just be an enum of either a -p flag or a --all flag. When that needs to be resolved we can then just iterate over all packages in a workspace.

@euclio
Copy link
Contributor Author

euclio commented Oct 20, 2016

@alexcrichton How should I handle the ambiguity when the member has the same name as a dependency?

@alexcrichton
Copy link
Member

Oh that's ok, -p is a "package id specification" which will uniquely identify a crate in the dependency graph

@cbiffle
Copy link
Contributor

cbiffle commented Nov 4, 2016

Is there a reason not to make --all the default in "virtual" workspaces?

It seems like a more reasonable default than the current behavior, which is:

$ cargo build
error: manifest path `/path/embrs/Cargo.toml` is a virtual manifest, but this command requires running against an actual package in this workspace

@alexcrichton
Copy link
Member

@cbiffle perhaps yeah, I figured it'd be good to ask for an explicit --all but there's not really any other interpretation of a cargo build against a virtual manifest, so just defaulting seems reasonable.

@Arnavion
Copy link

I've had a PS function cargoall in my local profile to do cargoall { cargo test }. Perhaps an approach to add a cargo all subcommand (eg cargo all test or cargo all cargo test) will be preferable to adding --all to every existing subcommand?

function cargoall ([ScriptBlock] $ScriptBlock) {
    (cargo metadata --no-deps | ConvertFrom-Json).packages |
        %{ ([System.IO.FileInfo] $_.manifest_path).Directory } |
        %{
            $_.FullName
            Push-Location $_
            &$ScriptBlock
            Pop-Location
        }
}

(This runs cargo metadata --no-deps, parses the JSON output, then executes the parameter as a command inside the directory of each package.)

@rocallahan
Copy link

cargo build --all would be very helpful to us. We have a collection of crates, several of which are --bin, and it would be nice to have a single cargo command to build them all. That would let us get a parallel build.

bors added a commit that referenced this issue Dec 8, 2016
Add `--all` flag to `cargo test`

Work towards #2878.

This flag allows a user to test all members of a workspace by using `cargo test --all`. The command is also supported when using a virtual workspace manifest.
@sdroege
Copy link
Contributor

sdroege commented Feb 2, 2017

cargo build --all (#3511) and cargo doc --all (#3515) also exist now

@Dushistov
Copy link

@sdroege
It would be nice to exclude some crates from build --all, for example
I have bunch of crates and one of them specific for android, it depends on android NDK,
and it is possible to build it only with cargo build --target=arm-linux-androideabi,
so cargo build --all failed, it would be nice to have something like cargo build --all --exclude android

@gifnksm
Copy link

gifnksm commented May 1, 2017

It would be very nice to have cargo bench --all.

@sdroege
Copy link
Contributor

sdroege commented May 1, 2017

It would be very nice to have cargo bench --all.

Indeed, I'll take a look at that tomorrow if nobody else is faster. What other commands would be useful to have with an --all variant?

bors added a commit that referenced this issue May 2, 2017
Add support for benchmarking all members of a workspace with "bench --all"

Same behaviour as "build --all" and others.

See #2878 (comment)
@alexcrichton
Copy link
Member

Thanks @sdroege!

@kvark
Copy link

kvark commented May 2, 2017

@Dushistov

It would be nice to exclude some crates from build --all, for example
I have bunch of crates and one of them specific for android, it depends on android NDK,

This is very applicable to gfx-rs as well, we have dependencies on Windows, OSX, and other stuff, that can't be ON at the same time.

@torkleyy
Copy link
Contributor

I created a PR for the --exclude flag ( #4031 ), would somebody like to review?

bors added a commit that referenced this issue May 27, 2017
Add --exclude flag

Allows to exclude packages in conjunction
with --all.

Addresses #2878
@azriel91
Copy link

azriel91 commented Jun 7, 2017

Hi, does this issue cover supporting --all to any arbitrary custom subcommand to run that subcommand across all workspace crates? For example, cargo fmt works on the current crate, but I'd like to have a way to run it against all workspace members.

The way the current code looks (to my untrained eyes), each subcommand determines whether or not it supports running against the workspace via the --all flag. I'm wondering if it makes sense to have a command that switches the order like so:

cargo workspace <subcommand> [opts] [args]
# this would run the following against each crate in the workspace:
cargo subcommand [opts] [args]

That way any custom subcommands will be able to be run against all crates in the workspace without needing to implement it.

@alexcrichton
Copy link
Member

@azriel91 I'm not currently thinking that we'd have such a subcommand or interpretation of this, it'd be up to subcommands to interpret and act on --all if they'd like.

@gnzlbg
Copy link
Contributor

gnzlbg commented Sep 17, 2017

cargo fmt --all and cargo clippy --all would be really useful.

@carols10cents
Copy link
Member

A bunch of commands have --all flags now, and the remaining ones should get their own issue, like publish does. Any 3rd party subcommands would need to implement this on their own, so issues should be filed on those commands' repos.

So I'm closing this!

@gnzlbg
Copy link
Contributor

gnzlbg commented Sep 26, 2017

@carols10cents

and the remaining ones should get their own issue

Did you fill these issues?

@azriel91
Copy link

azriel91 commented Sep 26, 2017

@gnzlbg that's up to the subcommands (and we probably don't know all of them), at least these subcommands support it:

Another cargo command which I found useful is cargo make, which supports running arbitrary commands in each workspace member. The default configuration is pretty good, and you can add your own if you wish.

@gnzlbg
Copy link
Contributor

gnzlbg commented Sep 26, 2017

That's good news, 9 days ago clippy and fmt still did not support --all. I think the only other one missing from the ones mentioned here is cargo bench --all.

@carols10cents
Copy link
Member

@gnzlbg

$ cargo --version
cargo 0.21.0 (5b4b8b2ae 2017-08-12)
$ cargo bench --help
Execute all benchmarks of a local package

Usage:
    cargo bench [options] [--] [<args>...]

...
    --all                        Benchmark all packages in the workspace
...

It looks like cargo bench supports --all to me.

@azriel91
Copy link

This ugly command tells you what subcommands you have installed have the --all option:

cargo --list | grep -vF 'Installed' | xargs -I {} bash -c 'echo -n {}: ; cargo {} --help 2>&1 | grep -m 1 "^.\+--all[^-a-z]" || echo' | grep '.\+--all'

Sample output:

bench:    --all                        Benchmark all packages in the workspace
build:    --all                        Build all packages in the workspace
check:    --all                        Check all packages in the workspace
doc:    --all                        Document all packages in the workspace
fmt:        --all           format all packages (only usable in workspaces)
test:    --all                        Test all packages in the workspace

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests