Publishing a crate with unpublished dependencies

[ysimonson]

I'd like to publish a crate that depends on various crates from tower-rs, which are not yet themselves published. I'm not sure how to do this.

---

If I specify the dependencies in Cargo.toml as sourced from git:

tower-grpc = { git = "https://github.com/tower-rs/tower-grpc", rev = "94a7b51" }
tower-grpc-build = { git = "https://github.com/tower-rs/tower-grpc", rev = "94a7b51" }
tower-add-origin = { git = "https://github.com/tower-rs/tower-http", rev = "6d7a9fd" }
tower-h2 = { git = "https://github.com/tower-rs/tower-h2", rev = "71effe1" }
tower-service = "0.2"
tower-util = { git = "https://github.com/tower-rs/tower", rev = "50fc5e8" }

I can get as far as packaging, but not publish:

$ cargo publish
    Updating crates.io index
error: crates cannot be published with dependencies sourced from a repository
either publish `tower-grpc` as its own crate and specify a version as a dependency or pull it into this repository and specify it with a path and version
(crate `tower-grpc` has repository path `https://github.com/tower-rs/tower-grpc?rev=94a7b51`)

---

If I pull in the dependencies as submodules and use paths in Cargo.toml:

tower-grpc = { path = "tower/tower-grpc", version = "0.1.0" }
tower-grpc-build = { path = "tower/tower-grpc/tower-grpc-build", version = "0.1.0" }
tower-add-origin = { path = "tower/tower-http/tower-add-origin", version = "0.1.0" }
tower-h2 = { path = "tower/tower-h2", version = "0.1.0" }
tower-service = "0.2"
tower-util = { path = "tower/tower/tower-util", version = "0.1.0" }

I can't package because it's trying to pull the crates from crates.io:

$ cargo package
...
error: failed to verify package tarball

Caused by:
  failed to select a version for the requirement `tower-grpc = "^0.1.0"`
  candidate versions found which didn't match: 0.0.0
  location searched: crates.io index
required by package `pachyderm v0.1.0 (/Users/yusuf/work/rust-pachyderm/target/package/pachyderm-0.1.0)`

---

If I use patch:

tower-grpc = "0.1.0"
tower-grpc-build = "0.1.0"
tower-add-origin = "0.1.0"
tower-h2 = "0.1.0"
tower-service = "0.2"
tower-util = "0.1.0"

[patch.crates-io]
tower-grpc = { path = "tower/tower-grpc" }
tower-grpc-build = { path = "tower/tower-grpc/tower-grpc-build" }
tower-add-origin = { path = "tower/tower-http/tower-add-origin" }
tower-h2 = { path = "tower/tower-h2" }
tower-util = { path = "tower/tower/tower-util" }

I get the same verification error when packaging.

Observed on stable v1.33.0.

[sfackler]

You cannot publish a crate that relies on git dependencies.

[ysimonson]

Right, but the error message implies that you can pull in submodules, which don't work either.

[ehuss]

When you have multiple crates to publish, you have to publish from the leaves first, and then work up to the root. There is a long-standing request (such as #1169) to be able to publish multiple packages at once, which would handle that automatically.

[ysimonson]

This isn't trying to publish multiple crates; rather, it's trying publish a single crate with multiple dependencies, some of which are themselves unpublished.

[ehuss]

A crate cannot be published with unpublished dependencies. Can you say more about why you want to do that? It wouldn't be usable with missing dependencies.

[ysimonson]

The dependencies aren't missing per se, but they are unpublished. Based off the error, I guess I would've thought that you can pull in dependencies that are not on crates.io via git, or path.

[peterhuene]

I'm in the same boat as @ysimonson and initially thought pull it into this repository and specify it with a path and version was implying there was some way of "vendoring" the unpublished dependency so that cargo publish would work. Thus the error seems misleading if the only way we can publish a crate with unpublished dependencies is to publish those dependencies.

[2-complex]

I think I understand why you can't publish a project that depends on another github repo. But why can't you publish when it sources a local directory using "path="? Doing that seems essentially equivalent to incorporating the source-code of the package into your own, which presumably would work.

[mcobzarenco]

@2-complex Your cargo dependencies, path included, are different crates (i. e. different units of compilation).

Doing that seems essentially equivalent to incorporating the source-code of the package into your own, which presumably would work.

That's not right, a path dependency is a different crate that gets compiled separately and then linked together.

[jeikabu]

You also can't publish if the unpublished crate is optional = true and the feature is not activated.
Not sure if that's intentional for similar reasons. I suppose because the consumer could enable the feature in dependencies...

[nitinkhobragade]

Here at https://doc.rust-lang.org/cargo/reference/specifying-dependencies.html#multiple-locations it says

# Uses `my-bitflags` when used locally, and uses
# version 1.0 from crates.io when published.
bitflags = { path = "my-bitflags", version = "1.0" }

When I provide path dependency then cargo package asks for version number, and when I provide both path and version it starts looking into crate.io for the package.
Stuck! Please help!!

[omarabid]

Here at https://doc.rust-lang.org/cargo/reference/specifying-dependencies.html#multiple-locations it says

# Uses `my-bitflags` when used locally, and uses
# version 1.0 from crates.io when published.
bitflags = { path = "my-bitflags", version = "1.0" }

When I provide path dependency then cargo package asks for version number, and when I provide both path and version it starts looking into crate.io for the package. Stuck! Please help!!

I'm only guessing: It probably only uses the path dependency if you have a version that's yet to be published on crates.io (presumably to speed up development). If bitflags has 1.0 published, then it's going to ignore your local path. (and rightfully so, I guess).

[Gowee]

I realize by design all deps should be published before a crate is published for now.

But in my case, I make some minor patches to a dependency that won't be accepted by the upstream. I just want to keep it as-is without publishing since it is useless for general use. Could it be possible to hide some crates from appearing in the search results of crates.io? Otherwise, it might confuse others.

[Levitanus]

I also don't see the point in publishing a patched crate, which I don't own. And, actually, I can't imagine how to manage a situation.

[soultice]

Say I patch a dependency which itself has multiple workspaces. I'd have to publish each workspace I've applied patches to individually?