Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
* Action: Themis Core
This is the first action we add so it sets an example. All testing
actions are triggered by either pushing to important integration
branches, or by submitting a pull request touching relevant files,
or on schedule at 06:00 UTC every day.
All of them are going to have at least "unit-tests" and "examples" jobs,
with some more as appropriate. Since Themis Core is the most diverse
thing that we have, it has quite a few jobs for checking it.
Code examples for Themis Core are self-contained tests, we just need to
build and run them. However, there were some minor issues with the code
producing warnings that we resolve for a clean build.
* Action: ThemisPP
The action itself is more or less trivial. Note though how we actually
test the examples, making sure they work as expected.
The examples themselves were kind of dated and did not work reliably
(especially the networked ones). Update them, clean up code style, and
make sure that they exit with non-zero code when failing.
* Action: GoThemis
This is one of the most straighforward actions -- but not so fast!
GoThemis build actually requires GOTHEMIS_IMPORT variable to be set,
just be aware of that. "make test_go" relies on it.
Take care to test against multiple versions of Go. We had issues with
that in the past so this environment is important. Unfortunately,
installing multiple versions of Go is not that easy with available
actions. But I don't want to write a new one (or some shell scripts)
so we simply run them in parallel.
Also, touch up code examples: gofmt them and make sure they os.Exit(1)
when then fail.
* Action: PHPThemis
Remember that we have two separate code bases for PHP 5 and PHP 7.
Furthermore, PHP is somewhat "old school" with installation so use
a convenient PPA to install various versions in parallel.
There is some weird interference between modules so we make sure to
*not* install php-fpm which seems to break PHP distribution. Anyhow,
thank you, Ondrej, for maintaining this repository!
Our Makefile expects PHP to be available as "php" in PATH, so we use
update-alternatives to fix up the symlinks in the system.
We test examples with the latest version only to reduce the matrix.
Also note that PHPThemis currently does not support PHP 7.3+.
* Action: JavaThemis
Well, actually, more like AndroidThemis right now, but the actions are
named after code bases, not platforms.
There are few nice words that I have for Android on CI. Let's just leave
it at that it's abysmally slow. Though, this script seems to do the job.
Most of the time. Sometimes we still fail to wait for emulator to boot.
If we're lucky, it boots in 3-4 minutes. If we're not lucky it gets
stuck for 15 minutes.
JavaThemis currently does not have a standlone unit-test suite so we
have to test it via instrumentation tests on Android by running the
entire emulator. However, we should still do these tests as Android
build process is tricky and sometimes it does fail.
* Action: ObjCThemis
iOS build automation is not much easier than Android, but at least iOS
Simulator on macOS supports x86. Thankfully, we are developing a library
and for tests we do not need code signing. Otherwise we'd dealing with
longstanding Apple policy of changing the way code signing works every
18 months.
However, most pain and suffering comes from the build systems popular
for iOS/macOS development. Note that CocoaPods cache. It shaves off
about 4 minutes and 850 MB of crap^W trunk reposistory that CocoaPods
pulls. It still takes about three minutes to download and unpack but
that's better than nothing. Though, we have to do it for every build.
Maybe some day we'll invent a shared cache, but until then let's just
ride upon Microsoft's generosity of providing free macOS runners.
(Otherwise we would be paying $2.08 per build.)
There are also various other issues with dependencies, like not having
a decent packaging of OpenSSL, which leads to us using GRKOpenSSL which
is not really maintained and causes podspec validation warnings. Eh...
I just give up, this is insurmountable, I wonder whether we should be
maintaining OpenSSL of our own instead. However, that will hurt dynamic
linking if people are using OpenSSL for other pods.
* Action: JsThemis
That's the one for Node.js. Well, it's more or less straightforward and
without any surprises. We test across multiple versions of Node.js so
there is some amount of NVM juggling involved.
* Action: WasmThemis
WasmThemis is more close to Themis Core than any other wrapper. It also
uses Node.js for runtime backend so we test with multiple versions.
WasmThemis curretly does not have any code examples. (The ones for
JsThemis should work with slight changes, but we don't test that.)
* Action: PyThemis
Here come the scripting languages! We need to test with both Python 2
and Python 3 so there is some complexity related to that. Also, some
code examples need a couple of services so make them running.
Right, examples...
- Update them all to be compatible with Python 3
- Avoid hardcoded IP addresses (use localhost)
- Make sure they exit with non-zero code when failing
- Add SO_REUSEADDR to sockets created in networked examaple servers
so that we can run them one after another without waiting for
Linux timeout on listening port reuse
- Also, some "pika" API has changed in the meantime, update that.
Now you see why it's important to test examples automatically?
* Action: RbThemis
Well this is easy. The only hard part is installing RVM which for some
reason really does not want to play nice, arbitrarily requiring you to
relogin into your shell to start working, etc. We can't to that on CI :(
There is a nice PPA -- thanks, Rael! -- which helps a bit, but there are
still some things that we need to do manually.
Also, tweak Secure Comparator example to actually stop once the
comparison is complete, not just sit there in an infinite loop.
* Action: RustThemis
RustThemis has a couple more native dependencies like pkg-config and
clang, make sure to install those.
Update the examples to exit cleanly on success and report failures via
the exit code. Also, do relay messages to the sender so that we have
something in stdout to test against.
Rust's Cargo uses CocoaPods-like approach with pulling the entire
package index history on clean builds. Rust build times are also quite
long, so caching *really* help here, turning 5 minutes into 15 seconds.
* Action: Code style
This action is for (relatively) quick code style check, mostly of C code
right now. It also runs clang-tidy static analysis.
We use really recent versions of Clang for that to catch as many issues
as we can with static analysis. Unfortunately, GitHub goes wa-a-ay
overboard with their prebundled repository lists and they fail to
install clang-tidy-8 without dependency issues. Use a clean container.
* Action: Integration testing
Run cross-language integration tests. Since anything anywhere can affect
these tests, they are running for every build. Refer to individual
language workflows for quirks.
Note that integration testing does not test *everything*, only those
that have tools in "tools" directory.
* Run AFL fuzzers on CI
Adapt recently added code from CircleCI to run AFL fuzzers on GitHub
Actions runners too. We use the same approach: build fuzzers and run
them for 30 seconds each, looking for some easy wins.
Note that while GitHub Actions support submitting directories as
artifacts, they choke on colons in filenames, therefore we zip AFL
reports manually to avoid stalling the build for 10 minutes.
* Check C compiler flags with AFL_CC when available
"supported" function is used to determine whether a flag is supported by
the compiler. Use AFL_CC instead of regular CC if available to correctly
check for flags when building AFL stuff.
Some time ago "supported" supported a second argument to select the
compiler to use, but this option does not seem to be used anymore.
* Zero-initialize structures with memset
Certain versions of afl-clang really don't like incomplete
initialization (see a2a5cd1 "Resolve compiler warnings").
Replace those with plain memset() to avoid warnings.
* Do not use fine suppression with AFL
afl-clang does not seem to support detailed UBSan variants so avoid
using them in ed25519 suppressions when compiling for AFL. However, we
still need the suppression to avoid triggering UBSan: use unqualified
no_sanitize("undefined") for that.
* Use fewer PBKDF2 iterations for AFL fuzzing
Similar to CirclCI (7edf2df "Fuzz passphrase API of Secure Cell").
* Build all eligible Carthage projects
Instead of enumerating the schemes, just run "carthage build" to build
everything that Carthage will build on user machines. By default it will
build only dependencies, pass --no-skip-current so that all projects in
the current directory will also get built.
* Run Carthage tests as well
We have CocoaPods-based tests and Carthage-based tests in different
Xcode projects. Let's test all of them.
* Add missing "import base64" in PyThemis samples
Apparently, it got lost during KDF implementation in PyThemis.
* Use "actions/setup-node" to install Node.js
GitHub Actions really want to force their users to install stuff via
Actions, so they have broken NVM installation. (I'm kidding, of course,
but this *might* be related to recent acquisition of npm, Inc. by GitHub
slash Microsoft).
Anyways, since NVM in unexplicably broken, use more idiomatic way to
install Node.js here. This expands the test matrix enormously, but ship
first, ask questions later. We'll optimize build runs later.
Both JsThemis and WasmThemis need Node.js so update both of them.
Integration tests need it too.
* Install JsThemis without sudo
Using "sudo make jsthemis_install" will result in system Node.js being
used, not the one installed for testing. Run JsThemis installation as
a regular user. Ditto for WasmThemis (though it should not be affected).
The reason it needs to be run with sudo sometimes is that some previous
installer builds Themis as root so the build directory ends up being
owned by root and npm cannot move its stuff there. Apply a quickfix for
that, but if we do it properly, we should not be building stuff as root
in the first place.
I'm really seriously frustrated with this changeset so I don't have
mental capacity to debug this tangle at the moment. I'll leave a FIXME
there and hope to come back at it later.- Loading branch information
