Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Try to ONLY build rust once per platform #270

Merged
merged 9 commits into from
Mar 18, 2021
Merged

Try to ONLY build rust once per platform #270

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
9 commits
Select commit Hold shift + click to select a range
4bd105f
build rust once/platform, share with artifacts
bfjelds Mar 9, 2021
2b70368
timeout => timeout-minutes
bfjelds Mar 9, 2021
f9cf90d
trigger when build-rust-code changes
bfjelds Mar 9, 2021
9618ce1
only run container builds after successful rust build
bfjelds Mar 9, 2021
99ef190
install rust toolchain to build rust
bfjelds Mar 9, 2021
a6db8cd
add crossbuild qemu
bfjelds Mar 9, 2021
54cfdf5
reduce retention to minimum
bfjelds Mar 9, 2021
f1c7a26
make test timeout longer
bfjelds Mar 10, 2021
045ff3e
update version
bfjelds Mar 11, 2021
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
57 changes: 22 additions & 35 deletions .github/workflows/build-agent-container.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,35 +1,10 @@
name: Build Agent

on:
push:
branches: [ main ]
paths:
- .github/actions/build-component-per-arch/**
- .github/actions/build-component-multi-arch/**
- .github/workflows/build-agent-container.yml
- build/containers/Dockerfile.agent
- agent/**
- shared/**
- version.txt
- build/akri-containers.mk
- build/akri-rust-containers.mk
- Makefile
pull_request:
branches: [ main ]
paths:
- .github/actions/build-component-per-arch/**
- .github/actions/build-component-multi-arch/**
- .github/workflows/build-agent-container.yml
- build/containers/Dockerfile.agent
- agent/**
- shared/**
- version.txt
- build/akri-containers.mk
- build/akri-rust-containers.mk
- Makefile
release:
types:
- published
workflow_run:
workflows: ["Build Production Rust Code"]
types:
- completed

env:
AKRI_COMPONENT: agent
Expand All @@ -38,6 +13,7 @@ env:
jobs:

per-arch:
if: ${{ github.event.workflow_run.conclusion == 'success' }}
runs-on: ubuntu-latest
timeout-minutes: 40
strategy:
Expand All @@ -62,24 +38,35 @@ jobs:
yarn install
yarn add @actions/core @actions/github @actions/exec fs

- name: Download rust build artifacts
uses: dawidd6/action-download-artifact@v2
with:
workflow: ${{ github.event.workflow_run.workflow_id }}
workflow_conclusion: success
commit: ${{ github.event.workflow_run.head_sha }}
name: rust-${{ matrix.arch.label }}-binaries
path: /tmp

- name: Unpack Rust binaries
run: |
tar -xvf /tmp/rust-${{ matrix.arch.label }}-binaries.tar

- name: Run Per-Arch component build for ${{ env.AKRI_COMPONENT }}
uses: ./.github/actions/build-component-per-arch
with:
github_event_name: ${{ github.event_name }}
github_event_name: ${{ github.event.workflow_run.event }}
github_ref: ${{ github.ref }}
github_event_action: ${{ github.event.action }}
github_merged: ${{ github.event.pull_request.merged }}
container_name: ${{ env.AKRI_COMPONENT }}
container_prefix: ghcr.io/deislabs/akri
container_registry_base_url: ghcr.io
container_registry_username: ${{ secrets.crUsername }}
container_registry_password: ${{ secrets.crPassword }}
makefile_component_name: ${{ env.MAKEFILE_COMPONENT }}
platform: ${{ matrix.arch }}
build_rust: "1"
build_rust: "0"

multi-arch:
if: (github.event_name == 'release') || (github.event_name == 'push' && github.ref == 'refs/heads/main') || (startsWith(github.event_name, 'pull_request') && github.event.action == 'closed' && github.event.pull_request.merged == true && github.ref != 'refs/heads/main')
if: ${{ github.event.workflow_run.conclusion == 'success' }} && ((github.event.workflow_run.event == 'release') || (github.event.workflow_run.event == 'push' && github.ref == 'refs/heads/main'))
needs: per-arch
runs-on: ubuntu-latest
timeout-minutes: 5
Expand All @@ -102,7 +89,7 @@ jobs:
- name: Run Multi-Arch component build for ${{ env.AKRI_COMPONENT }}
uses: ./.github/actions/build-component-multi-arch
with:
github_event_name: ${{ github.event_name }}
github_event_name: ${{ github.event.workflow_run.event }}
container_name: ${{ env.AKRI_COMPONENT }}
container_prefix: ghcr.io/deislabs/akri
container_registry_base_url: ghcr.io
Expand Down
57 changes: 22 additions & 35 deletions .github/workflows/build-controller-container.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,35 +1,10 @@
name: Build Controller

on:
push:
branches: [ main ]
paths:
- .github/actions/build-component-per-arch/**
- .github/actions/build-component-multi-arch/**
- .github/workflows/build-controller-container.yml
- build/containers/Dockerfile.controller
- controller/**
- shared/**
- version.txt
- build/akri-containers.mk
- build/akri-rust-containers.mk
- Makefile
pull_request:
branches: [ main ]
paths:
- .github/actions/build-component-per-arch/**
- .github/actions/build-component-multi-arch/**
- .github/workflows/build-controller-container.yml
- build/containers/Dockerfile.controller
- controller/**
- shared/**
- version.txt
- build/akri-containers.mk
- build/akri-rust-containers.mk
- Makefile
release:
types:
- published
workflow_run:
workflows: ["Build Production Rust Code"]
types:
- completed

env:
AKRI_COMPONENT: controller
Expand All @@ -38,6 +13,7 @@ env:
jobs:

per-arch:
if: ${{ github.event.workflow_run.conclusion == 'success' }}
runs-on: ubuntu-latest
timeout-minutes: 40
strategy:
Expand All @@ -61,24 +37,35 @@ jobs:
yarn install
yarn add @actions/core @actions/github @actions/exec fs

- name: Download rust build artifacts
uses: dawidd6/action-download-artifact@v2
with:
workflow: ${{ github.event.workflow_run.workflow_id }}
workflow_conclusion: success
commit: ${{ github.event.workflow_run.head_sha }}
name: rust-${{ matrix.arch.label }}-binaries
path: /tmp

- name: Unpack Rust binaries
run: |
tar -xvf /tmp/rust-${{ matrix.arch.label }}-binaries.tar

- name: Run Per-Arch component build for ${{ env.AKRI_COMPONENT }}
uses: ./.github/actions/build-component-per-arch
with:
github_event_name: ${{ github.event_name }}
github_event_name: ${{ github.event.workflow_run.event }}
github_ref: ${{ github.ref }}
github_event_action: ${{ github.event.action }}
github_merged: ${{ github.event.pull_request.merged }}
container_name: ${{ env.AKRI_COMPONENT }}
container_prefix: ghcr.io/deislabs/akri
container_registry_base_url: ghcr.io
container_registry_username: ${{ secrets.crUsername }}
container_registry_password: ${{ secrets.crPassword }}
makefile_component_name: ${{ env.MAKEFILE_COMPONENT }}
platform: ${{ matrix.arch }}
build_rust: "1"
build_rust: "0"

multi-arch:
if: (github.event_name == 'release') || (github.event_name == 'push' && github.ref == 'refs/heads/main') || (startsWith(github.event_name, 'pull_request') && github.event.action == 'closed' && github.event.pull_request.merged == true && github.ref != 'refs/heads/main')
if: ${{ github.event.workflow_run.conclusion == 'success' }} && ((github.event.workflow_run.event == 'release') || (github.event.workflow_run.event == 'push' && github.ref == 'refs/heads/main'))
needs: per-arch
runs-on: ubuntu-latest
timeout-minutes: 5
Expand All @@ -101,7 +88,7 @@ jobs:
- name: Run Multi-Arch component build for ${{ env.AKRI_COMPONENT }}
uses: ./.github/actions/build-component-multi-arch
with:
github_event_name: ${{ github.event_name }}
github_event_name: ${{ github.event.workflow_run.event }}
container_name: ${{ env.AKRI_COMPONENT }}
container_prefix: ghcr.io/deislabs/akri
container_registry_base_url: ghcr.io
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/build-opcua-monitoring-broker-container.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -74,7 +74,7 @@ jobs:
container_registry_password: ${{ secrets.crPassword }}
makefile_component_name: ${{ env.MAKEFILE_COMPONENT }}
platform: ${{ matrix.arch }}
build_rust: "1"
build_rust: "0"

multi-arch:
if: (github.event_name == 'release') || (github.event_name == 'push' && github.ref == 'refs/heads/main') || (startsWith(github.event_name, 'pull_request') && github.event.action == 'closed' && github.event.pull_request.merged == true && github.ref != 'refs/heads/main')
Expand Down
101 changes: 101 additions & 0 deletions .github/workflows/build-rust-code.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,101 @@
name: Build Production Rust Code
kate-goldenring marked this conversation as resolved.
Show resolved Hide resolved

on:
push:
branches: [ main ]
paths:
- .github/actions/build-component-per-arch/**
- .github/actions/build-component-multi-arch/**
- .github/workflows/build-rust-code.yml
- .github/workflows/build-agent-container.yml
- .github/workflows/build-controller-container.yml
- .github/workflows/build-udev-video-broker-container.yml
- .github/workflows/build-webhook-configuration-container.yml
- build/containers/Dockerfile.agent
- build/containers/Dockerfile.controller
- build/containers/Dockerfile.udev-video-broker
- build/containers/Dockerfile.webhook-configuration
- '**.rs'
- '**/Cargo.toml'
- '**/Cargo.lock'
- version.txt
- build/akri-containers.mk
- build/akri-rust-containers.mk
- Makefile
pull_request:
branches: [ main ]
paths:
- .github/actions/build-component-per-arch/**
- .github/actions/build-component-multi-arch/**
- .github/workflows/build-rust-code.yml
- .github/workflows/build-agent-container.yml
- .github/workflows/build-controller-container.yml
- .github/workflows/build-udev-video-broker-container.yml
- .github/workflows/build-webhook-configuration-container.yml
- build/containers/Dockerfile.agent
- build/containers/Dockerfile.controller
- build/containers/Dockerfile.udev-video-broker
- build/containers/Dockerfile.webhook-configuration
- '**.rs'
- '**/Cargo.toml'
- '**/Cargo.lock'
- version.txt
- build/akri-containers.mk
- build/akri-rust-containers.mk
- Makefile
release:
types:
- published

jobs:

build-rust:
runs-on: ubuntu-latest
timeout-minutes: 40
strategy:
matrix:
arch:
- label: arm64v8
make-target: arm64
rust-target-path: aarch64-unknown-linux-gnu
- label: arm32v7
make-target: arm32
rust-target-path: armv7-unknown-linux-gnueabihf
- label: amd64
make-target: amd64
rust-target-path: x86_64-unknown-linux-gnu

steps:
- name: Checkout the head commit of the branch
uses: actions/checkout@v2
with:
persist-credentials: false

- name: Rust install
uses: actions-rs/toolchain@v1
with:
toolchain: 1.49.0
override: true
components: clippy, rustfmt

- name: Build production rust for ${{ matrix.arch.label }}
run: |
docker run --rm --privileged multiarch/qemu-user-static --reset -p yes
make install-cross
cross --version
make akri-cross-build-${{ matrix.arch.make-target }}

- name: Package build binaries
run: |
tar_manifest='/tmp/tar-contents.txt'
> $tar_manifest
for f in target/${{ matrix.arch.rust-target-path }}/release/*; do filetype=$( file "$f" ); case "$filetype" in *ELF*) echo "$f" >> $tar_manifest ;; esac; done
tar -cvf /tmp/rust-${{ matrix.arch.label }}-binaries.tar `cat $tar_manifest`

romoh marked this conversation as resolved.
Show resolved Hide resolved
- name: Upload target binaries as artifact
uses: actions/upload-artifact@v2
romoh marked this conversation as resolved.
Show resolved Hide resolved
with:
name: rust-${{ matrix.arch.label }}-binaries
path: /tmp/rust-${{ matrix.arch.label }}-binaries.tar
retention-days: 1

Loading