-
Notifications
You must be signed in to change notification settings - Fork 2.4k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add support local mirrors of registries, take 2 #2857
Commits on Aug 1, 2016
-
Refactor URL parsing, be more robust for decoding errors
URL parsing now returns a `CargoResult` and also change a few `unwrap()` calls to returning a `CargoResult` when decoding various bits and pieces of information.
Configuration menu - View commit details
-
Copy full SHA for f730def - Browse repository at this point
Copy the full SHA f730defView commit details -
Configuration menu - View commit details
-
Copy full SHA for 71eff06 - Browse repository at this point
Copy the full SHA 71eff06View commit details -
Configuration menu - View commit details
-
Copy full SHA for 7ff9cbe - Browse repository at this point
Copy the full SHA 7ff9cbeView commit details -
Add convenience helpers to map source ids
Should help easily mapping packages from one source to another
Configuration menu - View commit details
-
Copy full SHA for 99ec335 - Browse repository at this point
Copy the full SHA 99ec335View commit details -
Configuration menu - View commit details
-
Copy full SHA for 2b19333 - Browse repository at this point
Copy the full SHA 2b19333View commit details -
This commit implements a scheme for .cargo/config files where sources can be redirected to other sources. The purpose of this will be to override crates.io for a few use cases: * Replace it with a mirror site that is sync'd to crates.io * Replace it with a "directory source" or some other local source This major feature of this redirection, however, is that none of it is encoded into the lock file. If one source is redirected to another then it is assumed that packages from both are exactly the same (e.g. `foo v0.0.1` is the same in both location). The lock file simply encodes the canonical soure (e.g. crates.io) rather than the replacement source. In the end this means that Cargo.lock files can be generated from any replacement source and shipped to other locations without the lockfile oscillating about where packages came from. Eventually this support will be extended to `Cargo.toml` itself (which will be encoded into the lock file), but that support is not implemented today. The syntax for what was implemented today looks like: # .cargo/config [source.my-awesome-registry] registry = 'https://example.com/path/to/index' [source.crates-io] replace-with = 'my-awesome-registry' Each source will have a canonical name and will be configured with the various keys underneath it (today just 'registry' and 'directory' will be accepted). The global `crates-io` source represents crates from the standard registry, and this can be replaced with other mirror sources. All tests have been modified to use this new infrastructure instead of the old `registry.index` configuration. This configuration is now also deprecated and will emit an unconditional warning about how it will no longer be used in the future. Finally, all subcommands now use this "source map" except for `cargo publish`, which will always publish to the default registry (in this case crates.io).
Configuration menu - View commit details
-
Copy full SHA for 8214bb9 - Browse repository at this point
Copy the full SHA 8214bb9View commit details -
Add sha256 checksums to the lockfile
This commit changes how lock files are encoded by checksums for each package in the lockfile to the `[metadata]` section. The previous commit implemented the ability to redirect sources, but the core assumption there was that a package coming from two different locations was always the same. An inevitable case, however, is that a source gets corrupted or, worse, ships a modified version of a crate to introduce instability between two "mirrors". The purpose of adding checksums will be to resolve this discrepancy. Each crate coming from crates.io will now record its sha256 checksum in the lock file. When a lock file already exists, the new checksum for a crate will be checked against it, and if they differ compilation will be aborted. Currently only registry crates will have sha256 checksums listed, all other sources do not have checksums at this time. The astute may notice that if the lock file format is changing, then a lock file generated by a newer Cargo might be mangled by an older Cargo. In anticipation of this, however, all Cargo versions published support a `[metadata]` section of the lock file which is transparently carried forward if encountered. This means that older Cargos compiling with a newer lock file will not verify checksums in the lock file, but they will carry forward the checksum information and prevent it from being removed. There are, however, a few situations where problems may still arise: 1. If an older Cargo takes a newer lockfile (with checksums) and updates it with a modified `Cargo.toml` (e.g. a package was added, removed, or updated), then the `[metadata]` section will not be updated appropriately. This modification would require a newer Cargo to come in and update the checksums for such a modification. 2. Today Cargo can only calculate checksums for registry sources, but we may eventually want to support other sources like git (or just straight-up path sources). If future Cargo implements support for this sort of checksum, then it's the same problem as above where older Cargos will not know how to keep the checksum in sync
Configuration menu - View commit details
-
Copy full SHA for 5430db6 - Browse repository at this point
Copy the full SHA 5430db6View commit details -
Refactor the RegistrySource implementation
Add an abstraction over which the index can be updated and downloads can be made. This is currently implemented for "remote" registries (e.g. crates.io), but soon there will be one for "local" registries as well.
Configuration menu - View commit details
-
Copy full SHA for f1e26ed - Browse repository at this point
Copy the full SHA f1e26edView commit details -
Implement a local registry type
This flavor of registry is intended to behave very similarly to the standard remote registry, except everything is contained locally on the filesystem instead. There are a few components to this new flavor of registry: 1. The registry itself is rooted at a particular directory, owning all structure beneath it. 2. There is an `index` folder with the same structure as the crates.io index describing the local registry (e.g. contents, versions, checksums, etc). 3. Inside the root will also be a list of `.crate` files which correspond to those described in the index. All crates must be of the form `name-version.crate` and be the same `.crate` files from crates.io itself. This support can currently be used via the previous implementation of source overrides with the new type: ```toml [source.crates-io] replace-with = 'my-awesome-registry' [source.my-awesome-registry] local-registry = 'path/to/registry' ``` I will soon follow up with a tool which can be used to manage these local registries externally.
Configuration menu - View commit details
-
Copy full SHA for c3a9599 - Browse repository at this point
Copy the full SHA c3a9599View commit details -
This flavor of source is intended to behave like a local registry except that its contents are unpacked rather than zipped up in `.crate` form. Like with local registries the only way to use this currently is via the `.cargo/config`-based source replacement currently, and primarily only to replace crates.io or other registries at the moment. A directory source is simply a directory which has many `.crate` files unpacked inside of it. The directory is not recursively traversed for changes, but rather it is just required that all elements in the directory are themselves directories of packages. This format is more suitable for checking into source trees, and it still provides guarantees around preventing modification of the original source from the upstream copy. Each directory in the directory source is required to have a `.cargo-checksum.json` file indicating the checksum it *would* have had if the crate had come from the original source as well as all of the sha256 checksums of all the files in the repo. It is intended that directory sources are assembled from a separately shipped subcommand (e.g. `cargo vendor` or `cargo local-registry`), so these checksum files don't have to be managed manually. Modification of a directory source is not the intended purpose, and if a modification is detected then the user is nudged towards solutions like `[replace]` which are intended for overriding other sources and processing local modifications.
Configuration menu - View commit details
-
Copy full SHA for 7fd5243 - Browse repository at this point
Copy the full SHA 7fd5243View commit details -
Configuration menu - View commit details
-
Copy full SHA for 4814a84 - Browse repository at this point
Copy the full SHA 4814a84View commit details -
Configuration menu - View commit details
-
Copy full SHA for 63ac9e1 - Browse repository at this point
Copy the full SHA 63ac9e1View commit details