Skip to content
Merged
Show file tree
Hide file tree
Changes from 3 commits
Commits
Show all changes
33 commits
Select commit Hold shift + click to select a range
2095a70
Support test data in benchmark workflows
Nov 8, 2021
02f59c5
Use existing test data for now to test
Nov 8, 2021
fb26b03
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Nov 8, 2021
2d1b780
Set OVERRIDE_TEST_DATA_REPOSITORY properly
Nov 8, 2021
760fff1
Merge branch 'wjbenfold-ci-benchmarks-test-data' of github.com:wjbenf…
Nov 8, 2021
1acbea7
Manually set version number to mirror cirrus.yml
Nov 8, 2021
90cf420
Add licence to top of file
Nov 8, 2021
2d9b8f9
Specify absolute path of test data
Nov 8, 2021
b944fa7
Changes to regridding to remove unnecessary print statements
Nov 8, 2021
4f7462e
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Nov 8, 2021
30bd14a
Can't refer to environment variables in env block
Nov 9, 2021
b78ab47
Merge branch 'wjbenfold-ci-benchmarks-test-data' of github.com:wjbenf…
Nov 9, 2021
8bb1b46
Precommit checks benchmarks too
Nov 9, 2021
b0c856f
Set the env var for data location
Nov 9, 2021
a40a88c
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Nov 9, 2021
62b03f2
Add some logging to noxfile for now
Nov 9, 2021
c0a5a57
Merge branch 'wjbenfold-ci-benchmarks-test-data' of github.com:wjbenf…
Nov 9, 2021
0ab3125
Do we have the test data path set right?
Nov 9, 2021
0bd99af
Rearrange imports and fix logging
Nov 9, 2021
6e76a92
More logging because iris.config.TEST_DATA_DIR was None
Nov 9, 2021
4f55a5f
More logging and changed mv
Nov 9, 2021
f28cdd0
Revert logging changes
Nov 9, 2021
8f7032f
Enable caching
Nov 9, 2021
25f7d76
Update to use new test data
Nov 9, 2021
04ec65c
Trim out last bit of logging
Nov 9, 2021
dc666e6
Update cache test data directory
Nov 9, 2021
15521eb
Reprompt ci
Nov 9, 2021
c3ba131
Fix cache location
Nov 9, 2021
568bdff
Bit of logging to reprompt CI
Nov 9, 2021
f57768c
Remove that logging
Nov 9, 2021
43000d9
Added type hinting
Nov 10, 2021
abfd06a
Review changes
Nov 11, 2021
fdfba71
Prevent piping hiding benchmark failures
wjbenfold Nov 11, 2021
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
25 changes: 25 additions & 0 deletions .github/workflows/benchmark.yml
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,10 @@ jobs:
benchmark:
runs-on: ubuntu-latest

env:
IRIS_TEST_DATA_DIR: benchmarks/iris-test-data
OVERRIDE_TEST_DATA_REPOSITORY: ${IRIS_TEST_DATA_DIR}/test_data

steps:
# Checks-out your repository under $GITHUB_WORKSPACE, so your job can access it
- uses: actions/checkout@v2
Expand Down Expand Up @@ -38,6 +42,27 @@ jobs:
key: ${{ runner.os }}-${{ github.sha }}
restore-keys: ${{ runner.os }}

- name: Get test data version
run: |
echo "IRIS_TEST_DATA_VERSION=$(wget --quiet -O - https://raw.githubusercontent.com/SciTools/iris-test-data/master/version.txt)" >> $GITHUB_ENV
Comment thread
trexfeathers marked this conversation as resolved.
Outdated

# - name: Cache test data directory
# id: cache-test-data
# uses: actions/cache@v2
# with:
# path: |
# ${{ env.IRIS_TEST_DATA_DIR }}
# key:
# test-data-${{ env.IRIS_TEST_DATA_VERSION }}

- name: Fetch the test data
# if: steps.cache-test-data.outputs.cache-hit != 'true'
run: |
wget --quiet https://github.com/SciTools/iris-test-data/archive/v${IRIS_TEST_DATA_VERSION}.zip -O iris-test-data.zip
unzip -q iris-test-data.zip
mkdir --parents ${IRIS_TEST_DATA_DIR}
mv iris-test-data-${IRIS_TEST_DATA_VERSION} ${IRIS_TEST_DATA_DIR}

- name: Run CI benchmarks
run: |
mkdir --parents benchmarks/.asv
Comment thread
wjbenfold marked this conversation as resolved.
Expand Down
42 changes: 42 additions & 0 deletions benchmarks/benchmarks/regridding.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,42 @@
"""
This file currently exists as a simple demonstration of Airspeed Velocity
performance testing. it is expected to be expanded/replaced in future.

"""

# import iris tests first so that some things can be initialised before
# importing anything else.
from iris import tests

import iris
from iris.analysis import AreaWeighted


@tests.skip_data
Comment thread
wjbenfold marked this conversation as resolved.
Outdated
class RegriddingTests:
Comment thread
wjbenfold marked this conversation as resolved.
Outdated
def setup(self):
# Prepare a cube and a regridding scheme.
cube_file_path = tests.get_data_path(
["NetCDF", "global", "xyt", "SMALL_hires_wind_u_for_ipcc4.nc"]
)
# cube_file_path = tests.get_data_path(
# ["NetCDF", "GloSea", "glosea_bm.nc"]
# )
self.cube = iris.load_cube(cube_file_path)

template_file_path = tests.get_data_path(
["NetCDF", "global", "xyt", "SMALL_hires_wind_u_for_ipcc4.nc"]
)
# template_file_path = tests.get_data_path(
# ["NetCDF", "GloSea", "template_regrid.nc"]
# )
self.template_cube = iris.load_cube(template_file_path)

# Chunked data makes the regridder run repeatedly
self.cube.data = self.cube.lazy_data().rechunk((1, -1, -1))

def time_regrid_area_w(self):
# Regrid the cube onto the template.
out = self.cube.regrid(self.template_cube, AreaWeighted())
# Realise the data
out.data