-
Notifications
You must be signed in to change notification settings - Fork 143
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
GitHub Actions automation #600
Merged
Merged
Changes from all commits
Commits
Show all changes
27 commits
Select commit
Hold shift + click to select a range
479e71d
Action: Themis Core
ilammy 3afbb1b
Action: ThemisPP
ilammy 0cf13ce
Action: GoThemis
ilammy ffecb9e
Action: PHPThemis
ilammy 50510ba
Action: JavaThemis
ilammy 6d1c63b
Action: ObjCThemis
ilammy 3d96c5a
Action: JsThemis
ilammy 25e1e95
Action: WasmThemis
ilammy 1cea9c3
Action: PyThemis
ilammy dd33e99
Action: RbThemis
ilammy fb332c3
Action: RustThemis
ilammy 28419e2
Action: Code style
ilammy fbf3337
Action: Integration testing
ilammy b1a892e
CHANGELOG
ilammy 20cc25b
Merge branch 'master' into github-actions-v2
ilammy 84fbeb1
Run AFL fuzzers on CI
ilammy f92ab0b
Check C compiler flags with AFL_CC when available
ilammy 3cba46b
Zero-initialize structures with memset
ilammy 0fe5f3d
Do not use fine suppression with AFL
ilammy 8602449
Merge branch 'master' into github-actions-v2
ilammy 8bee13d
Use fewer PBKDF2 iterations for AFL fuzzing
ilammy 9886723
Merge branch 'master' into github-actions-v2
ilammy d300710
Build all eligible Carthage projects
ilammy 335abc8
Run Carthage tests as well
ilammy a6b61f3
Add missing "import base64" in PyThemis samples
ilammy 86c71d5
Use "actions/setup-node" to install Node.js
ilammy 37db66c
Install JsThemis without sudo
ilammy File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,61 @@ | ||
name: Code style | ||
|
||
on: | ||
pull_request: | ||
paths: | ||
- '.github/workflows/code-style.yaml' | ||
- 'docs/examples/c*/**' | ||
- 'jni/**' | ||
- 'src/soter/**' | ||
- 'src/themis/**' | ||
- 'src/wrappers/themis/jsthemis/**' | ||
- 'src/wrappers/themis/themispp/**' | ||
- 'tests/common/**' | ||
- 'tests/soter/**' | ||
- 'tests/themis/**' | ||
- 'tests/themispp/**' | ||
- 'tools/afl/**' | ||
- '**/*.mk' | ||
- 'Makefile' | ||
- '!**/README*' | ||
push: | ||
branches: | ||
- master | ||
- stable | ||
- release/* | ||
schedule: | ||
- cron: '0 6 * * *' # every day at 6:00 UTC | ||
|
||
env: | ||
WITH_FATAL_WARNINGS: yes | ||
|
||
jobs: | ||
check-formatting: | ||
name: Check formatting | ||
runs-on: ubuntu-latest | ||
# GitHub's host contains way too much crap in /etc/apt/sources.list | ||
# which causes package conflicts in clang-format-8 and clang-tidy-8 | ||
# installation. Run this job in a pristine Ubuntu 18.04 container. | ||
container: ubuntu:18.04 | ||
steps: | ||
- name: Install system dependencies | ||
run: | | ||
export DEBIAN_FRONTEND=noninteractive | ||
apt update | ||
# System nodejs requires old OpenSSL libraries, not modern ones :( | ||
apt install --yes make clang-8 clang-format-8 clang-tidy-8 libstdc++-8-dev \ | ||
nodejs npm libssl1.0-dev \ | ||
default-jdk | ||
- name: Check out code | ||
uses: actions/checkout@v1 | ||
with: | ||
submodules: true | ||
- name: Check code formatting | ||
env: | ||
CC: clang-8 | ||
CXX: clang++-8 | ||
CLANG_FORMAT: clang-format-8 | ||
CLANG_TIDY: clang-tidy-8 | ||
run: | | ||
make fmt_check ENGINE=boringssl | ||
make fmt_check ENGINE=openssl |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,157 @@ | ||
name: Integration testing | ||
|
||
on: | ||
pull_request: | ||
paths: | ||
- '.github/workflows/integration.yaml' | ||
- 'gothemis/**' | ||
- 'src/soter/**' | ||
- 'src/themis/**' | ||
- 'src/wrappers/themis/jsthemis/**' | ||
- 'src/wrappers/themis/php/**' | ||
- 'src/wrappers/themis/php7/**' | ||
- 'src/wrappers/themis/python/**' | ||
- 'src/wrappers/themis/ruby/**' | ||
- 'src/wrappers/themis/rust/**' | ||
- 'src/wrappers/themis/wasm/**' | ||
- 'third_party/boringssl/src/**' | ||
- 'tools/_integration/**' | ||
- 'Cargo.toml' | ||
- '**/*.mk' | ||
- 'Makefile' | ||
- '!**/README*' | ||
push: | ||
branches: | ||
- master | ||
- stable | ||
- release/* | ||
schedule: | ||
- cron: '0 6 * * *' # every day at 6:00 UTC | ||
|
||
env: | ||
WITH_FATAL_WARNINGS: yes | ||
|
||
jobs: | ||
cross-language: | ||
name: Cross-language tests | ||
runs-on: ubuntu-latest | ||
env: | ||
GOTHEMIS_IMPORT: github.com/cossacklabs/themis/gothemis | ||
steps: | ||
- name: Install system dependencies | ||
run: | | ||
sudo sh -c 'echo "DEBIAN_FRONTEND=noninteractive" >> /etc/environment' | ||
sudo apt update | ||
sudo apt install --yes gcc g++ make libssl-dev \ | ||
python python-setuptools \ | ||
python3 python3-setuptools \ | ||
ruby ruby-dev \ | ||
pkg-config clang | ||
- name: Install RVM | ||
run: | | ||
sudo apt install --yes software-properties-common | ||
sudo apt-add-repository -y ppa:rael-gc/rvm | ||
sudo apt update | ||
sudo apt install rvm | ||
# Recent versions of RVM do not add us to "rvm" group automatically | ||
# and install their binaries into /usr/share, expecting the PATH | ||
# to be set via shell profile. GitHub Actions do not load profile | ||
# so we have to tweak the path manually here. | ||
sudo usermod -a -G rvm $(id -nu) | ||
echo "::add-path::/usr/share/rvm/bin" | ||
- name: Install stable Rust | ||
uses: actions-rs/toolchain@v1 | ||
with: | ||
toolchain: stable | ||
profile: minimal | ||
- name: Install Node.js 10.x | ||
uses: actions/setup-node@v1 | ||
with: | ||
node-version: 10.x | ||
- name: Install Emscripten | ||
run: | | ||
# Install Emscripten toolchain as described in documentation: | ||
# https://emscripten.org/docs/getting_started/downloads.html | ||
cd $HOME | ||
git clone https://github.com/emscripten-core/emsdk.git | ||
cd $HOME/emsdk | ||
# "upstream" flavor using LLVM compiler is still unstable for us | ||
./emsdk install latest-fastcomp | ||
./emsdk activate latest-fastcomp | ||
- name: Install PHP from PPA | ||
run: | | ||
sudo apt install --yes software-properties-common | ||
sudo add-apt-repository ppa:ondrej/php | ||
sudo apt update | ||
sudo apt install --yes \ | ||
php7.2 php7.2-fpm- php7.2-dev php7.2-xml php7.2-mbstring | ||
sudo update-alternatives --set php /usr/bin/php7.2 | ||
sudo update-alternatives --set php-config /usr/bin/php-config7.2 | ||
sudo update-alternatives --set phpize /usr/bin/phpize7.2 | ||
- name: Check out code | ||
uses: actions/checkout@v1 | ||
with: | ||
submodules: true | ||
- name: Install Themis Core | ||
run: sudo make install | ||
- name: Install ThemisPP | ||
run: sudo make themispp_install | ||
- name: Install PyThemis | ||
run: sudo make pythemis_install | ||
- name: Install RubyThemis | ||
run: sudo make rbthemis_install | ||
- name: Install GoThemis | ||
run: | | ||
mkdir -p $HOME/go/src/$GOTHEMIS_IMPORT | ||
rsync -auv gothemis/ $HOME/go/src/$GOTHEMIS_IMPORT | ||
# Cargo pulls in quite a few stuff from the Internet and Rust always | ||
# (slowly) recompiles dependencies, so make heavy use of caching | ||
- name: Cache Cargo registry | ||
uses: actions/cache@v1 | ||
with: | ||
path: ~/.cargo/registry | ||
key: ${{ runner.os }}-cargo-registry-${{ hashFiles('**/Cargo.toml') }} | ||
restore-keys: ${{ runner.os }}-cargo-registry- | ||
- name: Cache Cargo index | ||
uses: actions/cache@v1 | ||
with: | ||
path: ~/.cargo/git | ||
key: ${{ runner.os }}-cargo-index-${{ hashFiles('**/Cargo.toml') }} | ||
restore-keys: ${{ runner.os }}-cargo-index- | ||
- name: Cache Cargo build | ||
uses: actions/cache@v1 | ||
with: | ||
path: target | ||
key: ${{ runner.os }}-cargo-build-target-tools-${{ hashFiles('**/Cargo.toml') }} | ||
restore-keys: | | ||
${{ runner.os }}-cargo-build-target-tools- | ||
${{ runner.os }}-cargo-build-target- | ||
- name: Install RustThemis (test tools) | ||
run: make rustthemis_integration_tools | ||
- name: Install JsThemis | ||
run: | | ||
echo Node.js: $(node --version) | ||
echo npm: $(npm --version) | ||
# FIXME(ilammy, 2020-03-20): don't run previous installers as root | ||
# This makes "build" owned by root and JsThemis cannot move there. | ||
# We should not have a reason to build stuff as root. | ||
sudo chown $(id -u):$(id -g) build | ||
make jsthemis_install | ||
- name: Install WasmThemis | ||
run: | | ||
source "$HOME/emsdk/emsdk_env.sh" | ||
emmake make wasmthemis BUILD_PATH=build-wasm | ||
make wasmthemis_install BUILD_PATH=build-wasm | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. oh this js world 🤦♀️ |
||
- name: Install PHPThemis | ||
run: | | ||
sudo make phpthemis_install | ||
sudo bash -c 'echo "extension=phpthemis.so" > /etc/php/7.2/cli/conf.d/20-phpthemis.ini' | ||
- name: Run integration tests | ||
run: | | ||
python tests/_integration/tests_generator.py | ||
echo "Integration tests..." | ||
bash tests/_integration/integration_total.sh | ||
echo | ||
echo "Key generation tests..." | ||
bash tests/tools/check_keygen.sh | ||
echo |
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
for what this command?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This adds
rvm
command into PATH for all future actions. For some weird reason RVM is really unfriendly to automated installation. I’ve copied this from CircleCI scripts, but I wonder if it can be simplified. I guess we could avoid using RVM entirely and install Ruby via some GitHub Action way. But in this PR I’d avoid tweaking it.There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
actually I dont understand how
echo "::add-path::/path/"
adds path to $PATH. I knowexport PATH=/new/path:$PATH
way. And didn't found anything in google how to add a new path withecho "..."
command without append to some profile filep.s. as I remember, was some bug or we need some feature in ruby which was fixed/added in new minor version of ruby but debian hasn't it and used some own patches. and the simplest method to install supported ruby version was rvm.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Oh... I'm sorry. This is some GitHub Actions magic, here are the docs. It's one of the special “workflow commands”. Each action executes in a separate shell session so
export PATH
works only for next command in an action but not for the following actions.echo ::add-path
adds the path for next actions (but not for the current one).Currently even CircleCI does not test any particular Ruby, we're testing against the system one only. I'd imagine that BuildBot uses it though.
I've looked through our internal task tracker but that did not reveal anything relevant.
Whatever. I'll look into properly installed Ruby later. We'd better test all supported versions, which currently are 2.4 — 2.7.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
holy shit, one more tool specific language ))
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
How else are you expected to differentiate yourself from other free* CI services? Obviously not by providing better computing power or more stable platform, but by locking your users in with yet another YAML syntax.
For that reason I don’t really like all those special actions to install things. However, they do tend to integrate better.