GitHub Actions automation #600
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,61 @@ | ||
| name: Code style | ||
|
|
||
| on: | ||
| pull_request: | ||
| paths: | ||
| - '.github/workflows/code-style.yaml' | ||
| - 'docs/examples/c*/**' | ||
| - 'jni/**' | ||
| - 'src/soter/**' | ||
| - 'src/themis/**' | ||
| - 'src/wrappers/themis/jsthemis/**' | ||
| - 'src/wrappers/themis/themispp/**' | ||
| - 'tests/common/**' | ||
| - 'tests/soter/**' | ||
| - 'tests/themis/**' | ||
| - 'tests/themispp/**' | ||
| - 'tools/afl/**' | ||
| - '**/*.mk' | ||
| - 'Makefile' | ||
| - '!**/README*' | ||
| push: | ||
| branches: | ||
| - master | ||
| - stable | ||
| - release/* | ||
| schedule: | ||
| - cron: '0 6 * * *' # every day at 6:00 UTC | ||
|
|
||
| env: | ||
| WITH_FATAL_WARNINGS: yes | ||
|
|
||
| jobs: | ||
| check-formatting: | ||
| name: Check formatting | ||
| runs-on: ubuntu-latest | ||
| # GitHub's host contains way too much crap in /etc/apt/sources.list | ||
| # which causes package conflicts in clang-format-8 and clang-tidy-8 | ||
| # installation. Run this job in a pristine Ubuntu 18.04 container. | ||
| container: ubuntu:18.04 | ||
| steps: | ||
| - name: Install system dependencies | ||
| run: | | ||
| export DEBIAN_FRONTEND=noninteractive | ||
| apt update | ||
| # System nodejs requires old OpenSSL libraries, not modern ones :( | ||
| apt install --yes make clang-8 clang-format-8 clang-tidy-8 libstdc++-8-dev \ | ||
| nodejs npm libssl1.0-dev \ | ||
| default-jdk | ||
| - name: Check out code | ||
| uses: actions/checkout@v1 | ||
| with: | ||
| submodules: true | ||
| - name: Check code formatting | ||
| env: | ||
| CC: clang-8 | ||
| CXX: clang++-8 | ||
| CLANG_FORMAT: clang-format-8 | ||
| CLANG_TIDY: clang-tidy-8 | ||
| run: | | ||
| make fmt_check ENGINE=boringssl | ||
| make fmt_check ENGINE=openssl |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,157 @@ | ||
| name: Integration testing | ||
|
|
||
| on: | ||
| pull_request: | ||
| paths: | ||
| - '.github/workflows/integration.yaml' | ||
| - 'gothemis/**' | ||
| - 'src/soter/**' | ||
| - 'src/themis/**' | ||
| - 'src/wrappers/themis/jsthemis/**' | ||
| - 'src/wrappers/themis/php/**' | ||
| - 'src/wrappers/themis/php7/**' | ||
| - 'src/wrappers/themis/python/**' | ||
| - 'src/wrappers/themis/ruby/**' | ||
| - 'src/wrappers/themis/rust/**' | ||
| - 'src/wrappers/themis/wasm/**' | ||
| - 'third_party/boringssl/src/**' | ||
| - 'tools/_integration/**' | ||
| - 'Cargo.toml' | ||
| - '**/*.mk' | ||
| - 'Makefile' | ||
| - '!**/README*' | ||
| push: | ||
| branches: | ||
| - master | ||
| - stable | ||
| - release/* | ||
| schedule: | ||
| - cron: '0 6 * * *' # every day at 6:00 UTC | ||
|
|
||
| env: | ||
| WITH_FATAL_WARNINGS: yes | ||
|
|
||
| jobs: | ||
| cross-language: | ||
| name: Cross-language tests | ||
| runs-on: ubuntu-latest | ||
| env: | ||
| GOTHEMIS_IMPORT: github.com/cossacklabs/themis/gothemis | ||
| steps: | ||
| - name: Install system dependencies | ||
| run: | | ||
| sudo sh -c 'echo "DEBIAN_FRONTEND=noninteractive" >> /etc/environment' | ||
| sudo apt update | ||
| sudo apt install --yes gcc g++ make libssl-dev \ | ||
| python python-setuptools \ | ||
| python3 python3-setuptools \ | ||
| ruby ruby-dev \ | ||
| pkg-config clang | ||
| - name: Install RVM | ||
| run: | | ||
| sudo apt install --yes software-properties-common | ||
| sudo apt-add-repository -y ppa:rael-gc/rvm | ||
| sudo apt update | ||
| sudo apt install rvm | ||
| # Recent versions of RVM do not add us to "rvm" group automatically | ||
| # and install their binaries into /usr/share, expecting the PATH | ||
| # to be set via shell profile. GitHub Actions do not load profile | ||
| # so we have to tweak the path manually here. | ||
| sudo usermod -a -G rvm $(id -nu) | ||
| echo "::add-path::/usr/share/rvm/bin" | ||
| - name: Install stable Rust | ||
| uses: actions-rs/toolchain@v1 | ||
| with: | ||
| toolchain: stable | ||
| profile: minimal | ||
| - name: Install Node.js 10.x | ||
| uses: actions/setup-node@v1 | ||
| with: | ||
| node-version: 10.x | ||
| - name: Install Emscripten | ||
| run: | | ||
| # Install Emscripten toolchain as described in documentation: | ||
| # https://emscripten.org/docs/getting_started/downloads.html | ||
| cd $HOME | ||
| git clone https://github.com/emscripten-core/emsdk.git | ||
| cd $HOME/emsdk | ||
| # "upstream" flavor using LLVM compiler is still unstable for us | ||
| ./emsdk install latest-fastcomp | ||
| ./emsdk activate latest-fastcomp | ||
| - name: Install PHP from PPA | ||
| run: | | ||
| sudo apt install --yes software-properties-common | ||
| sudo add-apt-repository ppa:ondrej/php | ||
| sudo apt update | ||
| sudo apt install --yes \ | ||
| php7.2 php7.2-fpm- php7.2-dev php7.2-xml php7.2-mbstring | ||
| sudo update-alternatives --set php /usr/bin/php7.2 | ||
| sudo update-alternatives --set php-config /usr/bin/php-config7.2 | ||
| sudo update-alternatives --set phpize /usr/bin/phpize7.2 | ||
| - name: Check out code | ||
| uses: actions/checkout@v1 | ||
| with: | ||
| submodules: true | ||
| - name: Install Themis Core | ||
| run: sudo make install | ||
| - name: Install ThemisPP | ||
| run: sudo make themispp_install | ||
| - name: Install PyThemis | ||
| run: sudo make pythemis_install | ||
| - name: Install RubyThemis | ||
| run: sudo make rbthemis_install | ||
| - name: Install GoThemis | ||
| run: | | ||
| mkdir -p $HOME/go/src/$GOTHEMIS_IMPORT | ||
| rsync -auv gothemis/ $HOME/go/src/$GOTHEMIS_IMPORT | ||
| # Cargo pulls in quite a few stuff from the Internet and Rust always | ||
| # (slowly) recompiles dependencies, so make heavy use of caching | ||
| - name: Cache Cargo registry | ||
| uses: actions/cache@v1 | ||
| with: | ||
| path: ~/.cargo/registry | ||
| key: ${{ runner.os }}-cargo-registry-${{ hashFiles('**/Cargo.toml') }} | ||
| restore-keys: ${{ runner.os }}-cargo-registry- | ||
| - name: Cache Cargo index | ||
| uses: actions/cache@v1 | ||
| with: | ||
| path: ~/.cargo/git | ||
| key: ${{ runner.os }}-cargo-index-${{ hashFiles('**/Cargo.toml') }} | ||
| restore-keys: ${{ runner.os }}-cargo-index- | ||
| - name: Cache Cargo build | ||
| uses: actions/cache@v1 | ||
| with: | ||
| path: target | ||
| key: ${{ runner.os }}-cargo-build-target-tools-${{ hashFiles('**/Cargo.toml') }} | ||
| restore-keys: | | ||
| ${{ runner.os }}-cargo-build-target-tools- | ||
| ${{ runner.os }}-cargo-build-target- | ||
| - name: Install RustThemis (test tools) | ||
| run: make rustthemis_integration_tools | ||
| - name: Install JsThemis | ||
| run: | | ||
| echo Node.js: $(node --version) | ||
| echo npm: $(npm --version) | ||
| # FIXME(ilammy, 2020-03-20): don't run previous installers as root | ||
| # This makes "build" owned by root and JsThemis cannot move there. | ||
| # We should not have a reason to build stuff as root. | ||
| sudo chown $(id -u):$(id -g) build | ||
| make jsthemis_install | ||
| - name: Install WasmThemis | ||
| run: | | ||
| source "$HOME/emsdk/emsdk_env.sh" | ||
| emmake make wasmthemis BUILD_PATH=build-wasm | ||
| make wasmthemis_install BUILD_PATH=build-wasm | ||
|
|
||
| - name: Install PHPThemis | ||
| run: | | ||
| sudo make phpthemis_install | ||
| sudo bash -c 'echo "extension=phpthemis.so" > /etc/php/7.2/cli/conf.d/20-phpthemis.ini' | ||
| - name: Run integration tests | ||
| run: | | ||
| python tests/_integration/tests_generator.py | ||
| echo "Integration tests..." | ||
| bash tests/_integration/integration_total.sh | ||
| echo | ||
| echo "Key generation tests..." | ||
| bash tests/tools/check_keygen.sh | ||
| echo | ||
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
for what this command?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This adds
rvmcommand into PATH for all future actions. For some weird reason RVM is really unfriendly to automated installation. I’ve copied this from CircleCI scripts, but I wonder if it can be simplified. I guess we could avoid using RVM entirely and install Ruby via some GitHub Action way. But in this PR I’d avoid tweaking it.There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
actually I dont understand how
echo "::add-path::/path/"adds path to $PATH. I knowexport PATH=/new/path:$PATHway. And didn't found anything in google how to add a new path withecho "..."command without append to some profile filep.s. as I remember, was some bug or we need some feature in ruby which was fixed/added in new minor version of ruby but debian hasn't it and used some own patches. and the simplest method to install supported ruby version was rvm.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Oh... I'm sorry. This is some GitHub Actions magic, here are the docs. It's one of the special “workflow commands”. Each action executes in a separate shell session so
export PATHworks only for next command in an action but not for the following actions.echo ::add-pathadds the path for next actions (but not for the current one).Currently even CircleCI does not test any particular Ruby, we're testing against the system one only. I'd imagine that BuildBot uses it though.
I've looked through our internal task tracker but that did not reveal anything relevant.
Whatever. I'll look into properly installed Ruby later. We'd better test all supported versions, which currently are 2.4 — 2.7.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
holy shit, one more tool specific language ))
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
How else are you expected to differentiate yourself from other free* CI services? Obviously not by providing better computing power or more stable platform, but by locking your users in with yet another YAML syntax.
For that reason I don’t really like all those special actions to install things. However, they do tend to integrate better.