Skip to content

Add load/save benchmarks#132

Merged
trexfeathers merged 39 commits intoSciTools:unstructured_schemefrom
stephenworsley:io_benchmarks
Nov 29, 2021
Merged

Add load/save benchmarks#132
trexfeathers merged 39 commits intoSciTools:unstructured_schemefrom
stephenworsley:io_benchmarks

Conversation

@stephenworsley
Copy link
Contributor

Add benchmarks for the functions esmf_regrid.experimental.io.save_redridder and esmf_regrid.experimental.io.l_roadedridder.

@codecov
Copy link

codecov bot commented Nov 18, 2021

Codecov Report

Merging #132 (b4763b8) into unstructured_scheme (0484f41) will decrease coverage by 0.00%.
The diff coverage is 100.00%.

Impacted file tree graph

@@                   Coverage Diff                   @@
##           unstructured_scheme     #132      +/-   ##
=======================================================
- Coverage                99.53%   99.53%   -0.01%     
=======================================================
  Files                       27       27              
  Lines                     1717     1715       -2     
=======================================================
- Hits                      1709     1707       -2     
  Misses                       8        8              
Impacted Files Coverage Δ
.../tests/unit/experimental/io/test_round_tripping.py 100.00% <100.00%> (ø)
...ntal/unstructured_scheme/test__mesh_to_MeshInfo.py 100.00% <100.00%> (ø)

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 0484f41...b4763b8. Read the comment docs.

@stephenworsley
Copy link
Contributor Author

stephenworsley commented Nov 25, 2021

For the record, when the scalability benchmarks are run locally I get the following results:

[ 85.29%] ··· long.esmf_regridder.PrepareScalabilityGridToMesh.time_load                                                                        ok
[ 85.29%] ··· ============ ==========
               grid width            
              ------------ ----------
                   50       89.7±4ms 
                  100       98.3±7ms 
                  200       101±5ms  
                  400       119±5ms  
                  600       158±7ms  
                  800       192±10ms 
              ============ ==========

[ 88.24%] ··· long.esmf_regridder.PrepareScalabilityGridToMesh.time_prepare                                                                     ok
[ 88.24%] ··· ============ ============
               grid width              
              ------------ ------------
                   50        166±5ms   
                  100        593±30ms  
                  200       2.69±0.07s 
                  400       11.8±0.09s 
                  600       28.1±0.6s  
                  800       50.2±0.2s  
              ============ ============

[ 91.18%] ··· long.esmf_regridder.PrepareScalabilityGridToMesh.time_save                                                                        ok
[ 91.18%] ··· ============ ==========
               grid width            
              ------------ ----------
                   50       40.3±5ms 
                  100       45.1±5ms 
                  200       71.6±4ms 
                  400       140±4ms  
                  600       250±7ms  
                  800       447±10ms 
              ============ ==========

[ 94.12%] ··· long.esmf_regridder.PrepareScalabilityMeshToGrid.time_load                                                                        ok
[ 94.12%] ··· ============ ==========
               grid width            
              ------------ ----------
                   50       103±10ms 
                  100       102±10ms 
                  200       102±4ms  
                  400       122±8ms  
                  600       152±6ms  
                  800       212±10ms 
              ============ ==========

[ 97.06%] ··· long.esmf_regridder.PrepareScalabilityMeshToGrid.time_prepare                                                                     ok
[ 97.06%] ··· ============ ============
               grid width              
              ------------ ------------
                   50        217±10ms  
                  100        725±50ms  
                  200       3.14±0.2s  
                  400        14.4±1s   
                  600        30.7±1s   
                  800       1.06±0.01m 
              ============ ============

[100.00%] ··· long.esmf_regridder.PrepareScalabilityMeshToGrid.time_save                                                                        ok
[100.00%] ··· ============ ==========
               grid width            
              ------------ ----------
                   50       44.8±3ms 
                  100       56.9±4ms 
                  200       78.8±5ms 
                  400       160±10ms 
                  600       278±10ms 
                  800       484±30ms 
              ============ ==========

In the process, about 195MB of files are generated.

Copy link
Contributor

@trexfeathers trexfeathers left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @stephenworsley.

I'm pretty concerned about benchmark run time now - 14mins is a long time. My suggestion about parameterisation might help somewhat. Given other demands I think we're going to be forced to roll with it, but we shouldn't add any further benchmarks after this without reconsidering the efficiency of what we have (either by making existing ones smaller, or perhaps looking at GitHub Actions/caching).

The meat of this PR makes sense. My suggestions are mainly to improve efficiency/readability.

Copy link
Contributor

@trexfeathers trexfeathers left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Top work @stephenworsley 🥇

@trexfeathers trexfeathers merged commit 52a1bee into SciTools:unstructured_scheme Nov 29, 2021
stephenworsley added a commit to stephenworsley/iris-esmf-regrid that referenced this pull request Jan 10, 2022
* unstructured_scheme: (22 commits)
  Check mesh equality on MeshToGridESMFRegridder call (SciTools#138)
  Formalise regridder file format (SciTools#137)
  Fix issue 135 (mesh to grid chunking problems) (SciTools#136)
  Add load/save benchmarks (SciTools#132)
  Regridder load/saving (SciTools#130)
  Update dependencies (SciTools#128)
  Perform scalability for larger grids (SciTools#122)
  Add performance tests (SciTools#117)
  change iris source (SciTools#115)
  Unstructured scheme lazy regridding (with performance tests) (SciTools#111)
  Updating feature branch unstructured_scheme from 44d6048 to head of main, e528cbf
  Add grid to mesh scheme (SciTools#96)
  Unstructured scheme integration test (SciTools#66)
  Updated lockfiles according to branch requirement spec.
  Update Version to v0.1.dev2 (SciTools#59)
  Update version to v0.1.dev1 (SciTools#58)
  Unstructured Scheme - Extra Dims (SciTools#55)
  add __init__ to tests (SciTools#56)
  Unstructured Scheme - Additional Polish (docstrings and test coverage) (SciTools#53)
  Unstructured Scheme - Cube Creation 3D (SciTools#47)
  ...

# Conflicts:
#	benchmarks/benchmarks/ci/esmf_regridder.py
#	benchmarks/benchmarks/generate_data.py
#	requirements/nox.lock/py36-linux-64.lock
#	requirements/nox.lock/py37-linux-64.lock
#	requirements/nox.lock/py38-linux-64.lock
#	requirements/py36.yml
stephenworsley added a commit to stephenworsley/iris-esmf-regrid that referenced this pull request Jan 31, 2022
* unstructured_scheme: (24 commits)
  reset flag (SciTools#145)
  Use more stable version of connectivity indices (SciTools#144)
  Check mesh equality on MeshToGridESMFRegridder call (SciTools#138)
  Formalise regridder file format (SciTools#137)
  Fix issue 135 (mesh to grid chunking problems) (SciTools#136)
  Add load/save benchmarks (SciTools#132)
  Regridder load/saving (SciTools#130)
  Update dependencies (SciTools#128)
  Perform scalability for larger grids (SciTools#122)
  Add performance tests (SciTools#117)
  change iris source (SciTools#115)
  Unstructured scheme lazy regridding (with performance tests) (SciTools#111)
  Updating feature branch unstructured_scheme from 44d6048 to head of main, e528cbf
  Add grid to mesh scheme (SciTools#96)
  Unstructured scheme integration test (SciTools#66)
  Updated lockfiles according to branch requirement spec.
  Update Version to v0.1.dev2 (SciTools#59)
  Update version to v0.1.dev1 (SciTools#58)
  Unstructured Scheme - Extra Dims (SciTools#55)
  add __init__ to tests (SciTools#56)
  ...

# Conflicts:
#	benchmarks/benchmarks/ci/esmf_regridder.py
#	benchmarks/benchmarks/generate_data.py
#	requirements/nox.lock/py36-linux-64.lock
#	requirements/nox.lock/py37-linux-64.lock
#	requirements/nox.lock/py38-linux-64.lock
#	requirements/py36.yml
#	requirements/py37.yml
@trexfeathers trexfeathers mentioned this pull request Feb 21, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants