Pre-release Themis 0.15.0 (#1011)

* Fix rust issues (pin log, run bindgen) (#1005)

* rust: Pin log version to =0.4.18

The 0.4.19 requires rustc 1.60, but currently we support 1.58.
Pinning it is not a big deal since it's development dependecy for
tests and examples.

* rust: Regenerate and update lib.rs

bindgen was updated again and changed something which resulted in
new output (seems like some internal constants are removed).

* Pythemis: introduce `pyproject.toml` (#1006)

* pythemis: Add pyproject.toml

Since setup.py is deprecated, let's try moving to the pyproject.toml
and configuring it with the same data as in setup.py.

Use setuptools as a backend for no particular reasons ¯\_(ツ)_/¯,
just because the name is familiar and we have no reasons to not use
it or use something else.

Keep the old setup.py for backward compatibility so old systems can
try to build the package.

For now, keep 0.14.0, we will bump the version in another PR.

* makefile: Use pyproject.toml for installing pythemis

According to this [1] article, the correct command is

    pip install .

in the project's root. Let's try that. Also, the other option is

    python -m build --wheel

which builds the package but doesn't install it. We can provide
something like `pythemis_build` for it for example.

[1]: https://godatadriven.com/blog/a-practical-guide-to-setuptools-and-pyproject-toml/

* pythemis: Update classifiers to Python3.6+

With many hours and docker containers I tested that themis actually
works up to python 3.4. The other versions require some changes in
makefile so they are more like "grey area".

However, python3.5 is deprecated and it produces warning like
"DEPRECATION: Python 3.5 reached the end of its life on..." so many
libraries don't support it. Instead they start with 3.6 which will
do as well, I guess.

Though, actually python3.6 is also deprecated [1]. The same will be
true for python3.7 in a couple of days (Jun 27 2023), so the question
is, should we declare support of these versions?

[1]: https://devguide.python.org/versions/

* pythemis: Extend range of supported py versions

* Update changelog

* Run and pin bindgen (#1008)

* rust-themis: Update bindgen

It updated and broke something again 🤦

* rust-themis: Pin bindgen version

It is pretty unstable with its frequent releases, so let's pin it.

* Update changelog

* Bump wrapper versions to 0.15.0 (#1007)

* changelog: Add 0.15.0 summary

* themis-core: Update version

* pythemis: Update version

* pythemis: Fix 8-year old typo in AUTHORS :)

* rbthemis: Update version

* jsthemis: Update versions

* wasm-themis: Update versions

* android-themis: Update version

* rust-themis: Update versions

* react-native-themis: Update versions

* pythemis: https in AUTHORS

Co-authored-by: vixentael <[email protected]>

* rust-themis: Update bench versions

Somehow missed that.

* changelog: Forgot to mention rust 1.58

* changelog: Mention the new iteration count

---------

Co-authored-by: vixentael <[email protected]>

* Bump embedded BoringSSL (#1004)

* Bump BoringSSL and fix makefile

This is not the latest BoringSSL version yet, because there are
a couple of fixes. So, treat it as the first. Here we also fix
our makefile because the BoringSSL team fixed bug with the strange
behaviour of absolute path to symbols.txt [1].

[1]: https://boringssl.googlesource.com/boringssl/+/8c75ed046f799f1d8b805036b1dea9c5ec0a0fb5%5E%21/#F0

* Bump BoringSSL and fix opaque EVP

As OpenSSL, BoringSSL made many types opaque, so it will require
updating some of the code to not use fields.

* Bump BoringSSL again and fix RSA

The same issue - RSA type became opaque, so we need to use accessors
similar to what Openssl had.

* Bump BoringSSL once more

This is (hoperfully) the last bump. This time without issues but
we will see what CI says.

* Make bignum_to_bytes accept const bignum*

It will prevent some of the warnings. This function doesn't mutate
bignum anyway.

* Update changelog

* boringssl: Bump once again

* msys2: Update hashes temporarily

This are test values because we will move the tag. But for now,
let's just test it.

* phpthemis: Update version for the sake of testing

They will fail probably, but just out of curiosity let's try to run
the tests.

* Update date of the release

Solstice!

---------

Co-authored-by: vixentael <[email protected]>

.github/workflows/test-rust.yaml

@@ -215,7 +215,7 @@ jobs:
             ${{ runner.os }}-cargo-build-target-unit-tests-
             ${{ runner.os }}-cargo-build-target-
       - name: Install Bindgen
-        run: cargo install bindgen-cli
+        run: cargo install bindgen-cli --version 0.66.1 --force
       - name: Check out code
         uses: actions/checkout@v2
       - name: Check bindgen.sh output

CHANGELOG.md

@@ -4,13 +4,31 @@
 
 Changes that are currently in development and have not been released yet.
 
+## [0.15.0](https://github.com/cossacklabs/themis/releases/tag/0.15.0), June 21st 2023
+
+**TL;DR:**
+
+- Uncompressed EC public keys are now supported.
+- Increased PBKDF2 iteration count from 200000 to 314110 for Secure Cell passphrase mode.
+- OpenSSL 3.0 is now supported.
+- Pythemis now uses `pyproject.toml`.
+- And as usual: enhanced security measures and fixed bugs.
+
+**Breaking changes and deprecations:**
+- AndroidThemis build requires Gradle 7.3, Android SDK 11, Android NDK 25.
+- Some Soter functions are deprecated.
+- Node.js 8 is no longer supported.
+- Rust `SecureSessionTransport` implementations are now `Send`.
+- Rust 1.58 is now the minimum supported version.
+
 _Code:_
 
 - **Core**
 
   - Uncompressed EC public keys are now supported ([#959](https://github.com/cossacklabs/themis/pull/959), [#954](https://github.com/cossacklabs/themis/pull/954))
   - Themis will generate uncompressed EC public keys when `THEMIS_GEN_EC_KEY_PAIR_UNCOMPRESSED=1` environment variable is set ([#959](https://github.com/cossacklabs/themis/pull/959))
   - Increased PBKDF2 iteration count to maintain security of Secure Cell passphrase mode ([#976](https://github.com/cossacklabs/themis/pull/976)).
+  - Bumped embedded BoringSSL to the latest version ([#1004](https://github.com/cossacklabs/themis/pull/1004)).
 
   - **Soter** (low-level security core used by Themis)
 
@@ -30,6 +48,7 @@ _Code:_
 - **Python**
 
   - `pythemis.scomparator` and `pythemis.skeygen` are now imported with `from pythemis import *` ([#914](https://github.com/cossacklabs/themis/pull/914)).
+  - Pythemis supports `pyproject.toml` as a main way of building packages. The old `setup.py` is preserved for backwards compatibility ([#1006](https://github.com/cossacklabs/themis/pull/1006)).
 
 - **Ruby**
 
@@ -42,6 +61,7 @@ _Code:_
     This is technically a breaking change, but most reasonble implementations should be `Send` already. Please raise an issue if your code fails to build.
 
   - Minimum supported Rust version is now 1.58 ([#977](https://github.com/cossacklabs/themis/pull/977), [#984](https://github.com/cossacklabs/themis/pull/984)).
+  - Bindgen is pinned to 0.66.1 on CI ([#1008](https://github.com/cossacklabs/themis/pull/1008)).
 
 - **WebAssembly**
 

Makefile

@@ -598,7 +598,7 @@ ifdef PIP_VERSION
 PIP_THEMIS_INSTALL := $(shell pip freeze |grep themis)
 endif
 
-pythemis_install: CMD = cd src/wrappers/themis/python/ && python3 setup.py install --record files3.txt
+pythemis_install: CMD = cd src/wrappers/themis/python/ && pip3 install .
 pythemis_install:
 ifeq ($(PYTHON3_VERSION),)
 	@echo "python3 not found"

PKGBUILD.MSYS2

@@ -4,7 +4,7 @@
 
 pkgname=('themis' 'themis-devel')
 pkgbase=themis
-pkgver=0.14.0
+pkgver=0.15.0
 pkgrel=1
 
 pkgdesc="Data security library for network communication and data storage"
@@ -17,9 +17,9 @@ depends=('libopenssl>=1.1.1')
 makedepends=('tar' 'gcc' 'make' 'openssl-devel>=1.1.1')
 
 source=("https://github.com/cossacklabs/themis/archive/$pkgver.tar.gz")
-sha256sums=('2efb793e0ef604fb97258b07671a83135ad9229d83b92d7758b43510dcc6cb07')
-sha1sums=('6d89a69014c24f39aedea684a78fc10f6019e505')
-md5sums=('46a69d51d9e8a5d96ae919f3bf547ce9')
+sha256sums=('1c6082c6440b44eb1331637a39ffe3c5924fb99c28e630cd9adb300f5f46ed69')
+sha1sums=('7fa6ca58eed08030b7c68e18bc7eebea8660c39d')
+md5sums=('64dbed936994c402a337218854471a28')
 # TODO: verify package signature
 
 # Unfortunately, bsdtar cannot handle symlinks on MSYS2 [1] so we have to use

Themis.nsi

@@ -7,10 +7,10 @@ VIAddVersionKey "ProductName"     "Themis"
 VIAddVersionKey "CompanyName"     "Cossack Labs Limited"
 VIAddVersionKey "LegalCopyright"  "(c) Cossack Labs Limited"
 VIAddVersionKey "FileDescription" "Themis library installer"
-VIAddVersionKey "FileVersion"     "0.14.0"
-VIAddVersionKey "ProductVersion"  "0.14.0"
-VIFileVersion    0.14.0.0
-VIProductVersion 0.14.0.0
+VIAddVersionKey "FileVersion"     "0.15.0"
+VIAddVersionKey "ProductVersion"  "0.15.0"
+VIFileVersion    0.15.0.0
+VIProductVersion 0.15.0.0
 
 Page license
 Page directory

VERSION

@@ -1 +1 @@
-0.14.0
+0.15.0

benches/rust/Cargo.toml

@@ -5,7 +5,7 @@ edition = "2018"
 publish = false
 
 [dependencies]
-themis = { version = "0.14", path = "../../src/wrappers/themis/rust" }
+themis = { version = "0.15", path = "../../src/wrappers/themis/rust" }
 
 [dev-dependencies]
 criterion = { version = "0.3.4", features = ["cargo_bench_support", "html_reports"] }

benches/themis/Cargo.toml

@@ -5,8 +5,8 @@ edition = "2018"
 publish = false
 
 [dependencies]
-themis = { version = "0.14", path = "../../src/wrappers/themis/rust" }
-libthemis-sys = { version = "0.14", path = "../../src/wrappers/themis/rust/libthemis-sys" }
+themis = { version = "0.15", path = "../../src/wrappers/themis/rust" }
+libthemis-sys = { version = "0.15", path = "../../src/wrappers/themis/rust/libthemis-sys" }
 
 [dev-dependencies]
 criterion = { version = "0.3.4", features = ["cargo_bench_support", "html_reports"] }

gradle.properties

@@ -5,8 +5,8 @@
 org.gradle.configureondemand=true
 
 # Versions of AndroidThemis and JavaThemis packages.
-androidThemisVersion=0.14.0
-javaThemisVersion=0.14.0
+androidThemisVersion=0.15.0
+javaThemisVersion=0.15.0
 
 # Android Studio insists that this is set to use JUnit test runner.
 android.useAndroidX=true

src/soter/boringssl/soter.mk

@@ -87,14 +87,12 @@ ifeq ($(RENAME_BORINGSSL_SYMBOLS),yes)
 	 $(GO) run util/read_symbols.go -out $(abspath $(BIN_PATH)/boringssl/symbols.txt) \
 	     $(abspath $(BIN_PATH)/boringssl/stage-1/crypto/libcrypto.a) \
 	     $(abspath $(BIN_PATH)/boringssl/stage-1/decrepit/libdecrepit.a)
-	@# Path to symbols must be a relative one (relative to the build directory)
-	@# because absolute paths confuse BoringSSL's make.
 	@echo "building embedded BoringSSL again with renamed symbols..."
 	@mkdir -p $(BIN_PATH)/boringssl/stage-2
 	@cd $(BIN_PATH)/boringssl/stage-2 && \
 	 $(CMAKE) $(SOTER_ENGINE_CMAKE_FLAGS) \
 	     -DBORINGSSL_PREFIX=$(SOTER_BORINGSSL_PREFIX) \
-	     -DBORINGSSL_PREFIX_SYMBOLS=../symbols.txt \
+	     -DBORINGSSL_PREFIX_SYMBOLS=$(abspath $(BIN_PATH)/boringssl/symbols.txt) \
 	     $(abspath third_party/boringssl/src)
 ifeq ($(NINJA),)
 	@$(MAKE) -C $(BIN_PATH)/boringssl/stage-2 crypto decrepit

src/soter/boringssl/soter_rsa_key.c

@@ -101,7 +101,7 @@ static bool is_mod_size_supported(unsigned mod_size)
     }
 }
 
-static soter_status_t bignum_to_bytes(BIGNUM* bn, uint8_t* to, size_t to_length)
+static soter_status_t bignum_to_bytes(const BIGNUM* bn, uint8_t* to, size_t to_length)
 {
     size_t bn_size = (size_t)BN_num_bytes(bn);
     size_t bytes_copied;
@@ -159,16 +159,16 @@ soter_status_t soter_engine_specific_to_rsa_pub_key(const soter_engine_specific_
     }
 
     pub_exp = (uint32_t*)((unsigned char*)(key + 1) + rsa_mod_size);
-    if (BN_is_word(rsa->e, RSA_F4)) {
+    if (BN_is_word(RSA_get0_e(rsa), RSA_F4)) {
         *pub_exp = htobe32(RSA_F4);
-    } else if (BN_is_word(rsa->e, RSA_3)) {
+    } else if (BN_is_word(RSA_get0_e(rsa), RSA_3)) {
         *pub_exp = htobe32(RSA_3);
     } else {
         res = SOTER_INVALID_PARAMETER;
         goto err;
     }
 
-    res = bignum_to_bytes(rsa->n, (unsigned char*)(key + 1), rsa_mod_size);
+    res = bignum_to_bytes(RSA_get0_n(rsa), (unsigned char*)(key + 1), rsa_mod_size);
     if (SOTER_SUCCESS != res) {
         goto err;
     }
@@ -225,59 +225,59 @@ soter_status_t soter_engine_specific_to_rsa_priv_key(const soter_engine_specific
     }
 
     pub_exp = (uint32_t*)(curr_bn + ((rsa_mod_size * 4) + (rsa_mod_size / 2)));
-    if (BN_is_word(rsa->e, RSA_F4)) {
+    if (BN_is_word(RSA_get0_e(rsa), RSA_F4)) {
         *pub_exp = htobe32(RSA_F4);
-    } else if (BN_is_word(rsa->e, RSA_3)) {
+    } else if (BN_is_word(RSA_get0_e(rsa), RSA_3)) {
         *pub_exp = htobe32(RSA_3);
     } else {
         res = SOTER_INVALID_PARAMETER;
         goto err;
     }
 
     /* Private exponent */
-    res = bignum_to_bytes(rsa->d, curr_bn, rsa_mod_size);
+    res = bignum_to_bytes(RSA_get0_d(rsa), curr_bn, rsa_mod_size);
     if (SOTER_SUCCESS != res) {
         goto err;
     }
     curr_bn += rsa_mod_size;
 
     /* p */
-    res = bignum_to_bytes(rsa->p, curr_bn, rsa_mod_size / 2);
+    res = bignum_to_bytes(RSA_get0_p(rsa), curr_bn, rsa_mod_size / 2);
     if (SOTER_SUCCESS != res) {
         goto err;
     }
     curr_bn += rsa_mod_size / 2;
 
     /* q */
-    res = bignum_to_bytes(rsa->q, curr_bn, rsa_mod_size / 2);
+    res = bignum_to_bytes(RSA_get0_q(rsa), curr_bn, rsa_mod_size / 2);
     if (SOTER_SUCCESS != res) {
         goto err;
     }
     curr_bn += rsa_mod_size / 2;
 
     /* dp */
-    res = bignum_to_bytes(rsa->dmp1, curr_bn, rsa_mod_size / 2);
+    res = bignum_to_bytes(RSA_get0_dmp1(rsa), curr_bn, rsa_mod_size / 2);
     if (SOTER_SUCCESS != res) {
         goto err;
     }
     curr_bn += rsa_mod_size / 2;
 
     /* dq */
-    res = bignum_to_bytes(rsa->dmq1, curr_bn, rsa_mod_size / 2);
+    res = bignum_to_bytes(RSA_get0_dmq1(rsa), curr_bn, rsa_mod_size / 2);
     if (SOTER_SUCCESS != res) {
         goto err;
     }
     curr_bn += rsa_mod_size / 2;
 
     /* qp */
-    res = bignum_to_bytes(rsa->iqmp, curr_bn, rsa_mod_size / 2);
+    res = bignum_to_bytes(RSA_get0_iqmp(rsa), curr_bn, rsa_mod_size / 2);
     if (SOTER_SUCCESS != res) {
         goto err;
     }
     curr_bn += rsa_mod_size / 2;
 
     /* modulus */
-    res = bignum_to_bytes(rsa->n, curr_bn, rsa_mod_size);
+    res = bignum_to_bytes(RSA_get0_n(rsa), curr_bn, rsa_mod_size);
     if (SOTER_SUCCESS != res) {
         goto err;
     }

src/soter/boringssl/soter_sign_ecdsa.c

@@ -135,7 +135,7 @@ soter_status_t soter_sign_final_ecdsa_none_pkcs8(soter_sign_ctx_t* ctx,
     if (!pkey) {
         return SOTER_INVALID_PARAMETER;
     }
-    if (EVP_PKEY_type(pkey->type) != EVP_PKEY_EC) {
+    if (EVP_PKEY_type(EVP_PKEY_id(pkey)) != EVP_PKEY_EC) {
         return SOTER_INVALID_PARAMETER;
     }
     /* TODO: need review */

src/wrappers/themis/android/AndroidManifest.xml

@@ -1,3 +1,3 @@
 <?xml version="1.0" encoding="UTF-8"?>
-<manifest xmlns:android="http://schemas.android.com/apk/res/android" package="com.cossacklabs.themis" android:versionCode="1" android:versionName="0.14.0">
+<manifest xmlns:android="http://schemas.android.com/apk/res/android" package="com.cossacklabs.themis" android:versionCode="1" android:versionName="0.15.0">
 </manifest>

src/wrappers/themis/jsthemis/package.json

@@ -1,6 +1,6 @@
 {
   "name": "jsthemis",
-  "version": "0.14.0",
+  "version": "0.15.0",
   "description": "Themis is a convenient cryptographic library for data protection.",
   "main": "build/Release/jsthemis.node",
   "scripts": {

src/wrappers/themis/php/php_themis.h

@@ -17,7 +17,7 @@
 #ifndef _PHP_THEMIS_H_
 #define _PHP_THEMIS_H_
 
-#define PHP_THEMIS_VERSION "0.14.0"
+#define PHP_THEMIS_VERSION "0.15.0"
 #define PHP_THEMIS_EXTNAME "phpthemis"
 
 PHP_FUNCTION(phpthemis_secure_message_wrap);
@@ -38,5 +38,4 @@ PHP_FUNCTION(phpthemis_scell_context_imprint_decrypt);
 extern zend_module_entry phpthemis_module_entry;
 #define phpext_themis_ptr &phpthemis_module_entry
 
-
 #endif /* _PHP_THEMIS_H_ */

src/wrappers/themis/php7/php_themis.h

@@ -17,7 +17,7 @@
 #ifndef _PHP_THEMIS_H_
 #define _PHP_THEMIS_H_
 
-#define PHP_THEMIS_VERSION "0.14.0"
+#define PHP_THEMIS_VERSION "0.15.0"
 #define PHP_THEMIS_EXTNAME "phpthemis"
 
 extern zend_module_entry phpthemis_module_entry;

src/wrappers/themis/python/AUTHORS

@@ -1 +1 @@
-CossackLabs <def@cossacklabs.com> (http://cossacklabs.com/)
+CossackLabs <dev@cossacklabs.com> (https://cossacklabs.com/)

src/wrappers/themis/python/PKG-INFO

@@ -1,6 +1,6 @@
-Metadata-Version: 0.14.0
+Metadata-Version: 0.15.0
 Name: pythemis
-Version: 0.14.0
+Version: 0.15.0
 Summary: Data security library for network communication and data storage for Python
 Home-page: https://cossacklabs.com
 Author: Cossack Labs
@@ -26,5 +26,12 @@ Classifier: Programming Language :: Python :: 3
 Classifier: Programming Language :: Python :: 3.2
 Classifier: Programming Language :: Python :: 3.3
 Classifier: Programming Language :: Python :: 3.4
+Classifier: Programming Language :: Python :: 3.5
+Classifier: Programming Language :: Python :: 3.6
+Classifier: Programming Language :: Python :: 3.7
+Classifier: Programming Language :: Python :: 3.8
+Classifier: Programming Language :: Python :: 3.9
+Classifier: Programming Language :: Python :: 3.10
+Classifier: Programming Language :: Python :: 3.11
 Classifier: Programming Language :: Python :: Implementation :: CPython
 Classifier: Programming Language :: Python :: Implementation :: PyPy

src/wrappers/themis/python/pyproject.toml

@@ -0,0 +1,40 @@
+[build-system]
+requires = ["setuptools"]
+build-backend = "setuptools.build_meta"
+
+[project]
+name = "pythemis"
+version = "0.15.0"
+authors = [{ name = "CossackLabs", email = "[email protected]" }]
+description = "Themis is multi-platform library with a high-level and easy-to-use cryptographic toolkit for data protection"
+readme = "README.md"
+requires-python = ">=3.2"
+license = { file = "LICENSE" }
+dependencies = ["six", "enum34; python_version<'3.4'"]
+classifiers = [
+    "Intended Audience :: Developers",
+    "License :: OSI Approved :: Apache Software License",
+    "Natural Language :: English",
+    "Operating System :: MacOS :: MacOS X",
+    "Operating System :: POSIX",
+    "Operating System :: POSIX :: BSD",
+    "Operating System :: POSIX :: Linux",
+    "Programming Language :: Python",
+    "Programming Language :: Python :: 3",
+    "Programming Language :: Python :: 3.2",
+    "Programming Language :: Python :: 3.3",
+    "Programming Language :: Python :: 3.4",
+    "Programming Language :: Python :: 3.5",
+    "Programming Language :: Python :: 3.6",
+    "Programming Language :: Python :: 3.7",
+    "Programming Language :: Python :: 3.8",
+    "Programming Language :: Python :: 3.9",
+    "Programming Language :: Python :: 3.9",
+    "Programming Language :: Python :: 3.10",
+    "Programming Language :: Python :: 3.11",
+    "Programming Language :: Python :: Implementation :: CPython",
+    "Programming Language :: Python :: Implementation :: PyPy",
+]
+
+[tool.setuptools]
+packages = ["pythemis"]