Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Migrate asv workflow to new runner #1530

Merged
merged 1 commit into from
Jan 5, 2021
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
85 changes: 14 additions & 71 deletions .github/workflows/asv.yml
Original file line number Diff line number Diff line change
@@ -1,13 +1,12 @@
# Runner information:
# Standard F8s_v2 (8 vcpus, 16 GiB memory)
# Intel Xeon® Platinum 8168 (SkyLake)
# CPU: Intel(R) Core(TM) i7-6700K CPU @ 4.00GHz
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nitpicking: add core/thread count ?

# GPU: NVIDIA GeForce RTX 2060

name: asv-benchmarks

env:
OUTPUT_PATH: ${{ github.workspace }}
RESOURCE_GROUP: devito-regression
VM_NAME: asv-runner-00
ENVHOME: "/home/devito/environments"

on:
# Trigger the workflow on push to the master branch
Expand All @@ -16,49 +15,10 @@ on:
- master

jobs:
# Boot the self-hosted runner and start runner app
start-runner:
name: start-runner
runs-on: ubuntu-latest

steps:
- name: checkout repo
uses: actions/[email protected]

- name: start VM
env:
SP_APPID: ${{ secrets.SERVICE_PRINCIPAL_APPID }}
SP_SECRET: ${{ secrets.SERVICE_PRINCIPAL_SECRET }}
TENANT_ID: ${{ secrets.SERVICE_PRINCIPAL_TENANTID }}
SUB_ID: ${{ secrets.AZURE_SUBSCRIPTION_ID }}
run: >
pwsh -command "& '${{ env.OUTPUT_PATH }}\.github\azure\startVM.ps1'"
-servicePrincipal $SP_APPID
-servicePrincipalSecret $SP_SECRET
-servicePrincipalTenantId $TENANT_ID
-azureSubscriptionName $SUB_ID
-resourceGroupName $RESOURCE_GROUP
-vmName $VM_NAME
- name: set host
run: echo ::set-output name=action_host::$(az vm show -d -g $RESOURCE_GROUP -n $VM_NAME --query publicIps -o tsv)
id: host

- name: start actions runner app
uses: fifsky/ssh-action@master
with:
command: |
#!/bin/bash
nohup actions-runner/run.sh >/dev/null 2>&1 &
host: ${{ steps.host.outputs.action_host }}
user: ${{ secrets.CI_GPU_VM_ADMIN_LOGIN }}
pass: ${{ secrets.CI_GPU_VM_ADMIN_PASSWORD }}
args: "-tt"

# Run the asv benchmarks on the self-hosted runner
benchmarks:
name: benchmarks
needs: start-runner
runs-on: [self-hosted, asv]

env:
Expand All @@ -74,6 +34,15 @@ jobs:
- name: Checkout devito
uses: actions/[email protected]

- name: Set VIRTUAL_ENV
run: |
echo "VIRTUAL_ENV=$ENVHOME/asv" >> $GITHUB_ENV
echo "PATH=$VIRTUAL_ENV/bin:$PATH" >> $GITHUB_ENV
- name: Set PATH
run: |
echo "PATH=$VIRTUAL_ENV/bin:$PATH" >> $GITHUB_ENV
- name: Install dependencies
run: |
pip install --upgrade pip
Expand All @@ -82,11 +51,11 @@ jobs:
- name: Setup asv
run: |
asv machine --config benchmarks/regression/asv.conf.json --machine F8s_v2 --os ubuntu-18.04 --arch x86-64 --cpu IntelXeon8168 --num_cpu 8 --ram 16GB
asv machine --config benchmarks/regression/asv.conf.json --machine i7-6700K --os ubuntu-20.10 --arch x86-64 --cpu i7-6700K --num_cpu 8 --ram 16GB
- name: Run benchmarks
run: |
asv run -v --strict --show-stderr --config benchmarks/regression/asv.conf.json --cpu-affinity 0-7 --machine F8s_v2
asv run -v --strict --show-stderr --config benchmarks/regression/asv.conf.json --cpu-affinity 0-7 --machine i7-6700K
- name: Checkout asv-results branch
uses: actions/[email protected]
Expand Down Expand Up @@ -120,29 +89,3 @@ jobs:
EXTERNAL_REPOSITORY: devitocodes/devito-performance
PUBLISH_BRANCH: gh-pages
PUBLISH_DIR: ./benchmarks/regression/.asv/html

# Stop the runner
stop-runner:
name: stop-runner
if: ${{ always() }}
needs: benchmarks
runs-on: ubuntu-latest

steps:
- name: checkout repo
uses: actions/[email protected]

- name: stop VM
env:
SP_APPID: ${{ secrets.SERVICE_PRINCIPAL_APPID }}
SP_SECRET: ${{ secrets.SERVICE_PRINCIPAL_SECRET }}
TENANT_ID: ${{ secrets.SERVICE_PRINCIPAL_TENANTID }}
SUB_ID: ${{ secrets.AZURE_SUBSCRIPTION_ID }}
run: >
pwsh -command "& '${{ env.OUTPUT_PATH }}\.github\azure\stopVM.ps1'"
-servicePrincipal $SP_APPID
-servicePrincipalSecret $SP_SECRET
-servicePrincipalTenantId $TENANT_ID
-azureSubscriptionName $SUB_ID
-resourceGroupName $RESOURCE_GROUP
-vmName $VM_NAME
14 changes: 5 additions & 9 deletions benchmarks/regression/asv.conf.json
Original file line number Diff line number Diff line change
Expand Up @@ -32,7 +32,7 @@
// If missing or the empty string, the tool will be automatically
// determined by looking for tools on the PATH environment
// variable.
"environment_type": "conda",
"environment_type": "virtualenv",

// timeout in seconds for installing any dependencies in environment
// defaults to 10 min
Expand All @@ -43,11 +43,7 @@

// The Pythons you'd like to test against. If not provided, defaults
// to the current version of Python used to run `asv`.
// "pythons": ["3.6"],

// The list of conda channel names to be searched for benchmark
// dependency packages in the specified order
// "conda_channels": ["conda-forge", "defaults"],
// "pythons": ["3.8"],

// The matrix of dependencies to test. Each key is the name of a
// package (in PyPI) and the values are version numbers. An empty
Expand Down Expand Up @@ -92,10 +88,10 @@
// ],
//
// "include": [
// // additional env for python3.6
// {"python": "3.6", "numpy": "1.8"},
// // additional env for python3.8
// {"python": "3.8", "numpy": "1.8"},
// // additional env if run on windows+conda
// {"platform": "win32", "environment_type": "conda", "python": "3.6", "libpython": ""},
// {"platform": "win32", "environment_type": "conda", "python": "3.8", "libpython": ""},
// ],

// The directory (relative to the current directory) that benchmarks are
Expand Down