Skip to content
Merged
Show file tree
Hide file tree
Changes from 3 commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 4 additions & 4 deletions .github/workflows/benchmarks_run.yml
Original file line number Diff line number Diff line change
Expand Up @@ -71,9 +71,9 @@ jobs:
with:
fetch-depth: 0

- name: Install ASV & Nox
- name: Install Nox
run: |
pip install asv nox
pip install nox

- name: Cache environment directories
id: cache-env-dir
Expand Down Expand Up @@ -112,7 +112,7 @@ jobs:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
PR_NUMBER: ${{ github.event.number }}
run: |
python benchmarks/bm_runner.py branch origin/${{ github.base_ref }}
nox -s benchmarks -- branch origin/${{ github.base_ref }}

- name: Run overnight benchmarks
id: overnight
Expand All @@ -128,7 +128,7 @@ jobs:

if [ "$first_commit" != "" ]
then
python benchmarks/bm_runner.py overnight $first_commit
nox -s benchmarks -- overnight $first_commit
fi

- name: Warn of failure
Expand Down
12 changes: 6 additions & 6 deletions benchmarks/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -20,13 +20,13 @@ the PR's base branch, thus showing performance differences introduced
by the PR. (This run is managed by
[the aforementioned GitHub Action](../.github/workflows/benchmark.yml)).

`asv ...` commands must be run from this directory. You will need to have ASV
installed, as well as Nox (see
[Benchmark environments](#benchmark-environments)).

The benchmark runner ([bm_runner.py](./bm_runner.py)) provides conveniences for
To run locally: the **benchmark runner** provides conveniences for
common benchmark setup and run tasks, including replicating the automated
overnight run locally. See `python bm_runner.py --help` for detail.
overnight run locally. This is accessed via the Nox `benchmarks` session - see
`nox -s benchmarks -- --help` for detail (_see also:
[bm_runner.py](./bm_runner.py)_). Alternatively you can directly run `asv ...`
commands from this directory (you will still need Nox installed - see
[Benchmark environments](#benchmark-environments)).

A significant portion of benchmark run time is environment management. Run-time
can be reduced by placing the benchmark environment on the same file system as
Expand Down
3 changes: 3 additions & 0 deletions docs/src/whatsnew/latest.rst
Original file line number Diff line number Diff line change
Expand Up @@ -102,6 +102,9 @@ This document explains the changes made to Iris for this release
benchmark runs, and introduced larger more 'real world' benchmarks where
coverage was needed. (:pull:`5949`).

#. `@trexfeathers`_ made a Nox `benchmarks` session as the recommended entry
point for running benchmarks. (:pull:`5951`)


.. comment
Whatsnew author names (@github name) in alphabetical order. Note that,
Expand Down
24 changes: 24 additions & 0 deletions noxfile.py
Original file line number Diff line number Diff line change
Expand Up @@ -291,3 +291,27 @@ def wheel(session: nox.sessions.Session):
"import iris; print(f'{iris.__version__=}')",
external=True,
)


@nox.session
def benchmarks(session: nox.sessions.Session):
"""Run the Iris benchmark runner. Run session with `-- -h` for help.

Parameters
----------
session : object
A `nox.sessions.Session` object.

"""
if len(session.posargs) == 0:
message = (
"This session MUST be run with at least one argument. The "
"arguments are passed down to the benchmark runner script. E.g:\n"
"nox -s benchmarks -- --help\n"
"nox -s benchmarks -- something --help\n"
"nox -s benchmarks -- something\n"
)
session.error(message)
session.install("asv", "nox")
with session.chdir(Path(__file__).parent / "benchmarks"):
session.run("python", "bm_runner.py", *session.posargs)