Make publishing libthemis-src easier #419
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Remove unnecessary symlinks and whitelist the files which should go into libthemis-src crate. This ensures that we do not package anything extra. Unfortunately, it is still impossible to simply symlink the whole repo as libthemis-src/themis because Cargo is too smart. It sees a Cargo.toml in the repo root, thinks that this is some internal crate, and refuses to copy the whole "themis" subdirectory (despite that "include" list does not include themis/Cargo.toml). Well, maybe some day...
Just make sure that Cargo got it right and really copied Themis source
code into the package. It kinda likes to ignore symlinks on some systems
so double-check that. (I have been burnt by this once and published
a crate without Themis source code in it.)
"cargo package" does not run unit-tests so make this check a part of the
regular build. Try including a file into a dummy constant. If the file
is missing you will get an error message like this when packaging:
Compiling libthemis-src v0.0.3
error: couldn't read src/../themis/Makefile: No such file or directory (os error 2)
--> src/lib.rs:54:33
|
54 | const _THEMIS_MAKEFILE: &[u8] = include_bytes!("../themis/Makefile");
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
error: aborting due to previous error
error: failed to verify package tarball
Caused by:
Could not compile `libthemis-src`.
To learn more, run the command again with --verbose.
Rust constants are true compile-time constants so this does not
introduce any run-time overhead, does not use additional storage, etc.
We don't really need JNI, AFL, or Themis tests when building a vendored version of Themis from libthemis-src crate. Guard makefile inclusions with a CARGO variable defined by Cargo when building Rust code. It will be propagated into the make invocation.
vixentael
approved these changes
Mar 7, 2019
Actually, once we're here, reshuffle Cargo.toml files a bit in order to use standard file layout for Rust projects. Now the root Cargo.toml is a _virtual package_ which contains only a workspace. All Themis crates are now located in "src/wrappers/themis/rust" which is a bit easier to understand. This layout also allows us to avoid specifying all tests and examples explicitly. We use symlinks instead to keep the files at their expected locations. The root virtual package specifies only "themis" as a default member, meaning that commands like "cargo test" will affect "themis" by default. You'll need to specify other crates explicitly to act on them.
New crate layout does not make it easy to build integration tools which previously were a part of examples, but now they are not. Instead of hacking around, just add a new binary crate for them which depends on "themis" and build them separately when we need to run integration tests. Stop (ab)using "cargo install" for separating test binaries. It seems that cargo install does not work for crates that are not available on crates.io (even if --path) is used. Therefore just "cargo build" the binaries and find them in the target directory. This should be good enough for testing tools. The new crate is a part of the workspace so it will be checked and tested by CI along with all other Rust code. It just does not have a particular version and is marked as non-publishable to prevent accidental pushing of it to crates.io
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
libthemis-src crate includes Themis source code. Unfortunately, Cargo does not make publishing this crate easy so we have to use a certain amount of hacks. Improve them a bit so that they at least work out of the box for any developer. Running just
cargo publishshould be enough to publish all our crates now.Remove unnecessary symlinks and whitelist the files which should go into libthemis-src crate. This ensures that we do not package anything extra.
Unfortunately, it is still impossible to simply symlink the whole repo as libthemis-src/themis because Cargo is too smart. It sees a Cargo.toml in the repo root, thinks that this is some internal crate, and refuses to copy the whole "themis" subdirectory (despite that "include" list does not include themis/Cargo.toml). Well, maybe some day...
Just make sure that Cargo got it right and really copied Themis source code into the package. It kinda likes to ignore symlinks on some systems so double-check that. (I have been burnt by this once and published a crate without Themis source code in it.)
cargo packagedoes not run unit-tests so make this check a part of the regular build. Try including a file into a dummy constant. If the file is missing you will get an error message like this when packaging:Rust constants are true compile-time constants so this does not introduce any run-time overhead, does not use additional storage, etc.
We don't really need JNI, AFL, or Themis tests when building a vendored version of Themis from libthemis-src crate. Guard makefile inclusions with a CARGO variable defined by Cargo when building Rust code. It will be propagated into the make invocation.