Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add API support additional to Git as a Crate registry index #9364

Closed
secana opened this issue Apr 16, 2021 · 2 comments
Closed

Add API support additional to Git as a Crate registry index #9364

secana opened this issue Apr 16, 2021 · 2 comments
Labels
C-feature-request Category: proposal for a feature. Before PR, ping rust-lang/cargo if this is not `Feature accepted`

Comments

@secana
Copy link

secana commented Apr 16, 2021

Status Quo

Currently, cargo expects a crate index in the form of a git repository where all crates are listed. See Cargo Book: Registries. This has some disadvantages:

  • The git index needs to be pulled and updated before a build. Especially on build servers, where the build starts "fresh", this adds multiple minutes to each build.
  • Switching the implementation behind the index is not possible. While the registry HTTP API defines a clear API and the implementation is hidden (doesn't matter of written in rust, python, ...), the index has to be a git repository. Using a DB, S3 or similar, to store the crate metainformation, is not easily possible.
  • Authentication is limited. While the HTTP API to publish crates can easily support enterprise authentication like LDAP, the git index cannot. This is problematic for private registry, where authentication is needed to pull crates.

Possible solution

Add a feature to cargo that handles the pull of a crate or the search for a crate the same way the push to a registry is handled. Define a HTTP API which allows cargo to request if the registry has a crate in version x instead of cloning the git index and then search for the crate.

This would improve the performance of builds, as cargo can start directly to request only needed crates instead of cloning the whole index. The crate metadata store itself would be a hidden implementation detail, e.g. a DB or S3

It would allow to use the same token authorization approach to pull samples a it's used to push samples to a registry. Where the token is stored and how the authorization is done is a hidden implementation detail, which would support enterprise authentication mechanisms for private registries.

Notes

This is not a demand that someone implements such a feature, I would use this issue for a discussion and if there is interest create a proper RFC. As I whish for such a feature, I would like to help it implement, too. So please, let me know what you think!

Furthermore to make my motivation clear: I work for a company (no name, as I don't want to advertise here) which sells a private crate registry and in talks with our customers we got multiple requests for enterprise authentication and faster build times, especially on build servers.

@secana secana added the C-feature-request Category: proposal for a feature. Before PR, ping rust-lang/cargo if this is not `Feature accepted` label Apr 16, 2021
@ehuss
Copy link
Contributor

ehuss commented Apr 16, 2021

This sounds a bit like RFC 2789, have you checked that out? There has been some preliminary work on it over at #8890.

@secana
Copy link
Author

secana commented Apr 16, 2021

Hi @ehuss, thanks for the RFC link. I did not find that issue before. Looks exactly like what I need. I'll close this issue and add my comments to the RFC.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
C-feature-request Category: proposal for a feature. Before PR, ping rust-lang/cargo if this is not `Feature accepted`
Projects
None yet
Development

No branches or pull requests

2 participants