Skip to content

Use iris' regridder caching for faster regridding? #2341

@schlunma

Description

@schlunma

Some regridding schemes of iris allow reusing the regridding weights (see here for an overview of them), which might significantly speed up our regrid preprocessor if multiple variables of the same dataset are regridded. This is described here and basically boils down to using the following pattern

regridder = iris.analysis.Nearest().regridder(source_cube, target_cube)
for cube in list_of_cubes_on_source_grid:
    result = regridder(cube)

instead of

for cube in list_of_cubes_on_source_grid:
    result = cube.regrid(target_cube, iris.analysis.Nearest())

This should be fairly easy to implement with a dict cache that uses the source and target latitude and longitude dimensions as keys (we cannot use lru_cache since the hash of a coordinate is simply its id, therefore this would never use cached results for different cubes).

One drawback here is that we have an additional comparison of the coordinates for each regrid call, which makes it slightly slower if just a single variable for many data sets is analyzed.

@ESMValGroup/esmvaltool-coreteam is this something we want? I can open a draft PR so that the discussion is easier.

Metadata

Metadata

Labels

enhancementNew feature or request

Type

No type

Projects

No projects

Milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions