Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
27 commits
Select commit Hold shift + click to select a range
d36a492
Cleaned extract_shape
schlunma Apr 19, 2023
d9d6a8f
Raise proper error when cropping fails due to invalid IDs
schlunma Apr 19, 2023
758691b
Allowed dict as ids for extract shape
schlunma Apr 19, 2023
bb981d3
Added doc
schlunma Apr 19, 2023
e9b8573
Fixed existing tests
schlunma Apr 19, 2023
cdafd5c
Simplified existing tests
schlunma Apr 19, 2023
51e547e
Fixed existing tests that tested the wrong thing
schlunma Apr 19, 2023
87d4ef7
Added AR6 shapefiles
schlunma Apr 19, 2023
7485676
Allowed specifying shapefile relative to esmvalcore/preprocessor/shap…
schlunma Apr 19, 2023
7182085
Added license for AR6 region shapefiles
schlunma Apr 19, 2023
489afa1
Added tests for _update_extract_shape
schlunma Apr 19, 2023
b0240a9
Fixed tests
schlunma Apr 20, 2023
0afcb64
Optimized doc
schlunma Apr 20, 2023
e6a3f7c
Fixed doc
schlunma Apr 20, 2023
79f07a9
Added further tests
schlunma Apr 20, 2023
1556c0f
Fixed Codacy issues
schlunma Apr 20, 2023
b091dd7
Merge remote-tracking branch 'origin/main' into refactor_extract_shape
schlunma May 2, 2023
fb1dd77
Added debug messages for shapefile locations
schlunma May 2, 2023
da8bef8
Merge branch 'main' into refactor_extract_shape
schlunma May 31, 2023
6f15947
Merge branch 'main' into refactor_extract_shape
schlunma Jun 1, 2023
c313a6a
Update doc/recipe/preprocessor.rst
schlunma Jun 1, 2023
8847b95
Code review suggestions by @valeriupredoi
schlunma Jun 1, 2023
cd10461
Merge remote-tracking branch 'origin/main' into refactor_extract_shape
schlunma Jun 1, 2023
a02d289
Merge remote-tracking branch 'origin/main' into refactor_extract_shape
schlunma Jun 5, 2023
0fc0f32
Restored usage of os.path.exists
schlunma Jun 5, 2023
cdd5beb
Allow usage of special shapefiles in extract_shape()
schlunma Jun 5, 2023
f50a1ee
Fixed doc
schlunma Jun 5, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
96 changes: 68 additions & 28 deletions doc/recipe/preprocessor.rst
Original file line number Diff line number Diff line change
Expand Up @@ -1687,40 +1687,67 @@ See also :func:`esmvalcore.preprocessor.extract_named_regions`.


``extract_shape``
-------------------------
-----------------

Extract a shape or a representative point for this shape from
the data.
Extract a shape or a representative point for this shape from the data.

Parameters:
* ``shapefile``: path to the shapefile containing the geometry of the
region to be extracted. If the file contains multiple shapes behaviour
depends on the decomposed parameter. This path can be relative to
``auxiliary_data_dir`` defined in the :ref:`user configuration file`.
region to be extracted.
If the file contains multiple shapes behaviour depends on the
``decomposed`` parameter.
This path can be relative to ``auxiliary_data_dir`` defined in the
:ref:`user configuration file` or relative to
``esmvalcore/preprocessor/shapefiles`` (in that priority order).
Alternatively, a string (see "Shapefile name" below) can be given to load
one of the following shapefiles that are shipped with ESMValCore:

=============== ===================== ==========================================
Shapefile name Description Reference
=============== ===================== ==========================================
ar6 IPCC WG1 reference https://doi.org/10.5281/zenodo.5176260
regions (v4) used in
Assessment Report 6
=============== ===================== ==========================================

* ``method``: the method to select the region, selecting either all points
contained by the shape or a single representative point. Choose either
'contains' or 'representative'. If not a single grid point is contained
in the shape, a representative point will be selected.
contained by the shape or a single representative point.
Choose either `'contains'` or `'representative'`.
If not a single grid point is contained in the shape, a representative
point will be selected.
* ``crop``: by default extract_region_ will be used to crop the data to a
minimal rectangular region containing the shape. Set to ``false`` to only
mask data outside the shape. Data on irregular grids will not be cropped.
* ``decomposed``: by default ``false``, in this case the union of all the
regions in the shape file is masked out. If ``true``, the regions in the
shapefiles are masked out separately, generating an auxiliary dimension
for the cube for this.
* ``ids``: by default, ``[]``, in this case all the shapes in the file will
be used. If a list of IDs is provided, only the shapes matching them will
be used. The IDs are assigned from the ``name`` or ``id`` attributes (in
that order of priority) if present in the file or from the reading order
if otherwise not present. So, for example, if a file has both ```name``
and ``id`` attributes, the ids will be assigned from ``name``. If the file
only has the ``id`` attribute, it will be taken from it and if no ``name``
nor ``id`` attributes are present, an integer id starting from 1 will be
assigned automatically when reading the shapes. We discourage to rely on
this last behaviour as we can not assure that the reading order will be the
same in different platforms, so we encourage you to modify the file to add
a proper id attribute. If the file has an id attribute with a name that is
not supported, please open an issue so we can add support for it.
minimal rectangular region containing the shape.
Set to ``false`` to only mask data outside the shape.
Data on irregular grids will not be cropped.
* ``decomposed``: by default ``false``; in this case the union of all the
regions in the shapefile is masked out.
If set to ``true``, the regions in the shapefiles are masked out separately
and the output cube will have an additional dimension ``shape_id``
describing the requested regions.
* ``ids``: Shapes to be read from the shapefile.
Can be given as:

* :obj:`list`: IDs are assigned from the attributes ``name``, ``NAME``,
``Name``, ``id``, or ``ID`` (in that priority order; the first one
available is used).
If none of these attributes are available in the shapefile,
assume that the given `ids` correspond to the reading order of the
individual shapes.
So, for example, if a file has both ``name`` and ``id`` attributes, the
ids will be assigned from ``name``.
If the file only has the ``id`` attribute, it will be taken from it and
if no ``name`` nor ``id`` attributes are present, an integer ID starting
from 0 will be assigned automatically when reading the shapes.
We discourage to rely on this last behaviour as we can not assure that
the reading order will be the same on different platforms, so we
encourage you to specify a custom attribute using a :obj:`dict` (see
below) instead.
Note: An empty list is interpreted as ``ids=None`` (see below).
* :obj:`dict`: IDs (dictionary value; :obj:`list` of :obj:`str`) are
assigned from attribute given as dictionary key (:obj:`str`).
Only dictionaries with length 1 are supported.
Example: ``ids={'Acronym': ['GIC', 'WNA']}``.
* `None`: select all available regions from the shapefile.

Examples:
* Extract the shape of the river Elbe from a shapefile:
Expand All @@ -1746,6 +1773,19 @@ Examples:
- United Kingdom
- Taiwan

* Extract European AR6 regions:

.. code-block:: yaml

extract_shape:
shapefile: ar6
method: contains
ids:
Acronym:
- NEU
- WCE
- MED

See also :func:`esmvalcore.preprocessor.extract_shape`.


Expand Down
9 changes: 3 additions & 6 deletions esmvalcore/_recipe/recipe.py
Original file line number Diff line number Diff line change
Expand Up @@ -42,6 +42,7 @@
PreprocessingTask,
PreprocessorFile,
)
from esmvalcore.preprocessor._area import _update_shapefile_path
from esmvalcore.preprocessor._other import _group_products
from esmvalcore.preprocessor._regrid import (
_spec_to_latlonvals,
Expand Down Expand Up @@ -631,12 +632,8 @@ def _update_extract_shape(settings, session):
if 'extract_shape' in settings:
shapefile = settings['extract_shape'].get('shapefile')
if shapefile:
if not os.path.exists(shapefile):
shapefile = os.path.join(
session['auxiliary_data_dir'],
shapefile,
)
settings['extract_shape']['shapefile'] = shapefile
shapefile = _update_shapefile_path(shapefile, session=session)
settings['extract_shape']['shapefile'] = shapefile
check.extract_shape(settings['extract_shape'])


Expand Down
Loading