Make publishing libthemis-src easier (#419)

* Whitelist libthemis-src crate files

Remove unnecessary symlinks and whitelist the files which should go into
libthemis-src crate. This ensures that we do not package anything extra.

Unfortunately, it is still impossible to simply symlink the whole repo
as libthemis-src/themis because Cargo is too smart. It sees a Cargo.toml
in the repo root, thinks that this is some internal crate, and refuses
to copy the whole "themis" subdirectory (despite that "include" list
does not include themis/Cargo.toml). Well, maybe some day...

* Compile-time check for Themis source code

Just make sure that Cargo got it right and really copied Themis source
code into the package. It kinda likes to ignore symlinks on some systems
so double-check that. (I have been burnt by this once and published
a crate without Themis source code in it.)

"cargo package" does not run unit-tests so make this check a part of the
regular build. Try including a file into a dummy constant. If the file
is missing you will get an error message like this when packaging:

       Compiling libthemis-src v0.0.3
    error: couldn't read src/../themis/Makefile: No such file or directory (os error 2)
      --> src/lib.rs:54:33
       |
    54 | const _THEMIS_MAKEFILE: &[u8] = include_bytes!("../themis/Makefile");
       |                                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

    error: aborting due to previous error

    error: failed to verify package tarball

    Caused by:
      Could not compile `libthemis-src`.

    To learn more, run the command again with --verbose.

Rust constants are true compile-time constants so this does not
introduce any run-time overhead, does not use additional storage, etc.

* Do not include submakefiles when building Rust

We don't really need JNI, AFL, or Themis tests when building a vendored
version of Themis from libthemis-src crate. Guard makefile inclusions
with a CARGO variable defined by Cargo when building Rust code. It will
be propagated into the make invocation.

* New Cargo.toml layout

Actually, once we're here, reshuffle Cargo.toml files a bit in order to
use standard file layout for Rust projects. Now the root Cargo.toml is
a _virtual package_ which contains only a workspace. All Themis crates
are now located in "src/wrappers/themis/rust" which is a bit easier to
understand.

This layout also allows us to avoid specifying all tests and examples
explicitly. We use symlinks instead to keep the files at their expected
locations.

The root virtual package specifies only "themis" as a default member,
meaning that commands like "cargo test" will affect "themis" by default.
You'll need to specify other crates explicitly to act on them.

* Separate crate for rust-themis integration tools

New crate layout does not make it easy to build integration tools
which previously were a part of examples, but now they are not.
Instead of hacking around, just add a new binary crate for them
which depends on "themis" and build them separately when we need
to run integration tests.

Stop (ab)using "cargo install" for separating test binaries. It seems
that cargo install does not work for crates that are not available on
crates.io (even if --path) is used. Therefore just "cargo build" the
binaries and find them in the target directory. This should be good
enough for testing tools.

The new crate is a part of the workspace so it will be checked and
tested by CI along with all other Rust code. It just does not have
a particular version and is marked as non-publishable to prevent
accidental pushing of it to crates.io

Cargo.toml

@@ -1,118 +1,10 @@
-[package]
-name = "themis"
-version = "0.0.3"
-edition = "2018"
-authors = ["rust-themis developers"]
-description = "High-level cryptographic services for storage and messaging"
-homepage = "https://www.cossacklabs.com/themis/"
-repository = "https://github.com/cossacklabs/themis"
-readme = "src/wrappers/themis/rust/README.md"
-keywords = ["crypto", "Themis"]
-categories = ["cryptography", "api-bindings"]
-license = "Apache-2.0"
-include = [
-    "Cargo.toml",
-    "docs/examples/rust/**/*",
-    "src/wrappers/themis/rust/**/*",
-    "tests/rust/**/*",
-]
-
-[badges]
-circle-ci = { repository = "cossacklabs/themis", branch = "master" }
-maintenance = { status = "actively-developed" }
-
 [workspace]
-
-[lib]
-path = "src/wrappers/themis/rust/src/lib.rs"
-
-[features]
-vendored = ["bindings/vendored"]
-
-[dependencies]
-bindings = { package = "libthemis-sys", path = "src/wrappers/themis/rust/libthemis-sys", version = "=0.0.3" }
-zeroize = "0.5.2"
-
-[dev-dependencies]
-base64 = "0.10.0"
-byteorder = "1.2.7"
-clap = "2.32"
-lazy_static = "1.2.0"
-log = "0.4.6"
-env_logger = "0.6.0"
-
-[package.metadata.docs.rs]
-features = ["vendored"]
-dependencies = ["libssl-dev"]
-
-[[example]]
-name = "keygen"
-path = "docs/examples/rust/keygen.rs"
-
-[[example]]
-name = "secure_cell"
-path = "docs/examples/rust/secure_cell.rs"
-
-[[example]]
-name = "secure_compare"
-path = "docs/examples/rust/secure_compare.rs"
-
-[[example]]
-name = "secure_message_client_encrypt"
-path = "docs/examples/rust/secure_message_client_encrypt.rs"
-
-[[example]]
-name = "secure_message_client_verify"
-path = "docs/examples/rust/secure_message_client_verify.rs"
-
-[[example]]
-name = "secure_message_server"
-path = "docs/examples/rust/secure_message_server.rs"
-
-[[example]]
-name = "secure_session_echo_server"
-path = "docs/examples/rust/secure_session_echo_server.rs"
-
-[[example]]
-name = "secure_session_echo_client"
-path = "docs/examples/rust/secure_session_echo_client.rs"
-
-[[example]]
-name = "keygen_tool"
-path = "tools/rust/keygen_tool.rs"
-
-[[example]]
-name = "scell_context_string_echo"
-path = "tools/rust/scell_context_string_echo.rs"
-
-[[example]]
-name = "scell_seal_string_echo"
-path = "tools/rust/scell_seal_string_echo.rs"
-
-[[example]]
-name = "scell_token_string_echo"
-path = "tools/rust/scell_token_string_echo.rs"
-
-[[example]]
-name = "smessage_encryption"
-path = "tools/rust/smessage_encryption.rs"
-
-[[test]]
-name = "keys"
-path = "tests/rust/keys.rs"
-
-[[test]]
-name = "secure_cell"
-path = "tests/rust/secure_cell.rs"
-
-[[test]]
-name = "secure_comparator"
-path = "tests/rust/secure_comparator.rs"
-
-[[test]]
-name = "secure_message"
-path = "tests/rust/secure_message.rs"
-
-[[test]]
-name = "secure_session"
-path = "tests/rust/secure_session.rs"
+members = [
+    "src/wrappers/themis/rust",
+    "src/wrappers/themis/rust/libthemis-sys",
+    "src/wrappers/themis/rust/libthemis-src",
+    "tools/rust",
+]
+default-members = [
+    "src/wrappers/themis/rust",
+]

Makefile

@@ -292,9 +292,11 @@ endif
 ifndef ERROR
 include src/soter/soter.mk
 include src/themis/themis.mk
+ifndef CARGO
 include src/wrappers/themis/jsthemis/jsthemis.mk
 include jni/themis_jni.mk
 endif
+endif
 
 JSTHEMIS_PACKAGE_VERSION=$(shell cat src/wrappers/themis/jsthemis/package.json \
   | grep version \
@@ -390,8 +392,10 @@ $(AUD_PATH)/%: $(SRC_PATH)/%
 	@echo -n "compile "
 	@$(BUILD_CMD)
 
+ifndef CARGO
 include tests/test.mk
 include tools/afl/fuzzy.mk
+endif
 
 err: ; $(ERROR)
 

src/wrappers/themis/rust/Cargo.toml

@@ -0,0 +1,43 @@
+[package]
+name = "themis"
+version = "0.0.3"
+edition = "2018"
+authors = ["rust-themis developers"]
+description = "High-level cryptographic services for storage and messaging"
+homepage = "https://www.cossacklabs.com/themis/"
+repository = "https://github.com/cossacklabs/themis"
+readme = "README.md"
+keywords = ["crypto", "Themis"]
+categories = ["cryptography", "api-bindings"]
+license = "Apache-2.0"
+include = [
+    "/Cargo.toml",
+    "/LICENSE",
+    "/README.md",
+    "/examples/**/*.rs",
+    "/src/**/*.rs",
+    "/tests/**/*.rs",
+]
+
+[badges]
+circle-ci = { repository = "cossacklabs/themis", branch = "master" }
+maintenance = { status = "actively-developed" }
+
+[features]
+vendored = ["bindings/vendored"]
+
+[dependencies]
+bindings = { package = "libthemis-sys", path = "libthemis-sys", version = "=0.0.3" }
+zeroize = "0.5.2"
+
+[dev-dependencies]
+base64 = "0.10.0"
+byteorder = "1.2.7"
+clap = "2.32"
+lazy_static = "1.2.0"
+log = "0.4.6"
+env_logger = "0.6.0"
+
+[package.metadata.docs.rs]
+features = ["vendored"]
+dependencies = ["libssl-dev"]

src/wrappers/themis/rust/examples

@@ -0,0 +1 @@
+../../../../docs/examples/rust

src/wrappers/themis/rust/libthemis-src/Cargo.toml

@@ -10,6 +10,16 @@ readme = "README.md"
 keywords = ["crypto", "Themis"]
 categories = ["development-tools::build-utils"]
 license = "Apache-2.0"
+include = [
+    "/Cargo.toml",
+    "/LICENSE",
+    "/README.md",
+    "/src/**/*.rs",
+    "/themis/LICENSE",
+    "/themis/Makefile",
+    "/themis/src/soter/**/*",
+    "/themis/src/themis/**/*",
+]
 
 [badges]
 circle-ci = { repository = "cossacklabs/themis", branch = "master" }

src/wrappers/themis/rust/libthemis-src/src/lib.rs

@@ -49,6 +49,9 @@ use std::fs;
 use std::path::{Path, PathBuf};
 use std::process::{Command, Stdio};
 
+// Compile-time check for that we really bundle Themis source code.
+const _THEMIS_MAKEFILE: &[u8] = include_bytes!("../themis/Makefile");
+
 /// A builder (literally!) for Themis, produces [`Library`].
 ///
 /// [`Library`]: struct.Library.html

src/wrappers/themis/rust/tests

@@ -0,0 +1 @@
+../../../../tests/rust

tests/test.mk

@@ -92,11 +92,9 @@ themispp_test: $(THEMISPP_TEST_OBJ)
 
 rustthemis_integration_tools:
 	@echo "make integration tools for rust-themis..."
-	@mkdir -p $(TEST_BIN_PATH)/rust
-	@cargo install --examples --path . --force --root $(TEST_BIN_PATH)/rust
+	@cargo build --package themis-integration-tools
 	@for tool in $(notdir $(foreach tool,$(wildcard tools/rust/*.rs),$(basename $(tool)))); \
-	do cp $(TEST_BIN_PATH)/rust/bin/$$tool tools/rust/$$tool.rust; done
-	@echo -n "integration tools for rust-themis "
+	do cp target/debug/$$tool tools/rust/$$tool.rust; done
 	@$(PRINT_OK_)
 
 prepare_tests_basic: soter_test themis_test

tools/rust/Cargo.toml

@@ -0,0 +1,30 @@
+[package]
+name = "themis-integration-tools"
+version = "0.0.0"
+edition = "2018"
+publish = false
+
+[dependencies]
+base64 = "0.10.0"
+clap = "2.32"
+themis = { path = "../../src/wrappers/themis/rust" }
+
+[[bin]]
+name = "keygen_tool"
+path = "keygen_tool.rs"
+
+[[bin]]
+name = "scell_context_string_echo"
+path = "scell_context_string_echo.rs"
+
+[[bin]]
+name = "scell_seal_string_echo"
+path = "scell_seal_string_echo.rs"
+
+[[bin]]
+name = "scell_token_string_echo"
+path = "scell_token_string_echo.rs"
+
+[[bin]]
+name = "smessage_encryption"
+path = "smessage_encryption.rs"