Add load/save benchmarks#132
Conversation
for more information, see https://pre-commit.ci
Codecov Report
@@ Coverage Diff @@
## unstructured_scheme #132 +/- ##
=======================================================
- Coverage 99.53% 99.53% -0.01%
=======================================================
Files 27 27
Lines 1717 1715 -2
=======================================================
- Hits 1709 1707 -2
Misses 8 8
Continue to review full report at Codecov.
|
for more information, see https://pre-commit.ci
* origin/io_benchmarks: [pre-commit.ci] auto fixes from pre-commit.com hooks # Conflicts: # benchmarks/benchmarks/ci/esmf_regridder.py
for more information, see https://pre-commit.ci
Synthetic file generation - re-use files and ensure uniqueness.
for more information, see https://pre-commit.ci
* origin/io_benchmarks: [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* origin/io_benchmarks: [pre-commit.ci] auto fixes from pre-commit.com hooks
|
For the record, when the scalability benchmarks are run locally I get the following results: In the process, about 195MB of files are generated. |
trexfeathers
left a comment
There was a problem hiding this comment.
Thanks @stephenworsley.
I'm pretty concerned about benchmark run time now - 14mins is a long time. My suggestion about parameterisation might help somewhat. Given other demands I think we're going to be forced to roll with it, but we shouldn't add any further benchmarks after this without reconsidering the efficiency of what we have (either by making existing ones smaller, or perhaps looking at GitHub Actions/caching).
The meat of this PR makes sense. My suggestions are mainly to improve efficiency/readability.
trexfeathers
left a comment
There was a problem hiding this comment.
Top work @stephenworsley 🥇
* unstructured_scheme: (22 commits) Check mesh equality on MeshToGridESMFRegridder call (SciTools#138) Formalise regridder file format (SciTools#137) Fix issue 135 (mesh to grid chunking problems) (SciTools#136) Add load/save benchmarks (SciTools#132) Regridder load/saving (SciTools#130) Update dependencies (SciTools#128) Perform scalability for larger grids (SciTools#122) Add performance tests (SciTools#117) change iris source (SciTools#115) Unstructured scheme lazy regridding (with performance tests) (SciTools#111) Updating feature branch unstructured_scheme from 44d6048 to head of main, e528cbf Add grid to mesh scheme (SciTools#96) Unstructured scheme integration test (SciTools#66) Updated lockfiles according to branch requirement spec. Update Version to v0.1.dev2 (SciTools#59) Update version to v0.1.dev1 (SciTools#58) Unstructured Scheme - Extra Dims (SciTools#55) add __init__ to tests (SciTools#56) Unstructured Scheme - Additional Polish (docstrings and test coverage) (SciTools#53) Unstructured Scheme - Cube Creation 3D (SciTools#47) ... # Conflicts: # benchmarks/benchmarks/ci/esmf_regridder.py # benchmarks/benchmarks/generate_data.py # requirements/nox.lock/py36-linux-64.lock # requirements/nox.lock/py37-linux-64.lock # requirements/nox.lock/py38-linux-64.lock # requirements/py36.yml
* unstructured_scheme: (24 commits) reset flag (SciTools#145) Use more stable version of connectivity indices (SciTools#144) Check mesh equality on MeshToGridESMFRegridder call (SciTools#138) Formalise regridder file format (SciTools#137) Fix issue 135 (mesh to grid chunking problems) (SciTools#136) Add load/save benchmarks (SciTools#132) Regridder load/saving (SciTools#130) Update dependencies (SciTools#128) Perform scalability for larger grids (SciTools#122) Add performance tests (SciTools#117) change iris source (SciTools#115) Unstructured scheme lazy regridding (with performance tests) (SciTools#111) Updating feature branch unstructured_scheme from 44d6048 to head of main, e528cbf Add grid to mesh scheme (SciTools#96) Unstructured scheme integration test (SciTools#66) Updated lockfiles according to branch requirement spec. Update Version to v0.1.dev2 (SciTools#59) Update version to v0.1.dev1 (SciTools#58) Unstructured Scheme - Extra Dims (SciTools#55) add __init__ to tests (SciTools#56) ... # Conflicts: # benchmarks/benchmarks/ci/esmf_regridder.py # benchmarks/benchmarks/generate_data.py # requirements/nox.lock/py36-linux-64.lock # requirements/nox.lock/py37-linux-64.lock # requirements/nox.lock/py38-linux-64.lock # requirements/py36.yml # requirements/py37.yml
Add benchmarks for the functions
esmf_regrid.experimental.io.save_redridderandesmf_regrid.experimental.io.l_roadedridder.