Skip to content
Merged
Changes from 1 commit
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
39 changes: 31 additions & 8 deletions docs/src/whatsnew/3.6.rst
Original file line number Diff line number Diff line change
Expand Up @@ -13,14 +13,37 @@ This document explains the changes made to Iris for this release
:animate: fade-in
:open:

We're so excited about our recent support for **delayed saving of lazy data
to netCDF** (:pull:`5191`) that we're celebrating this important step change
in behavour with its very own dedicated release 🥳

We're super keen for the community to leverage the benefit of this new
feature within Iris that we've brought this release forward several months.
As a result, this minor release of Iris is intentionally light in content.
However, there are some other goodies available for you to enjoy, such as:
We're so excited about our recent support for **delayed saving of lazy data
to netCDF** (:pull:`5191`) that we're celebrating this important step change
in behavour with its very own dedicated release 🥳

By using ``iris.save(..., compute=False)`` you can now save to multiple netcdf files
in parallel. See the new ``compute`` keyword in :func:`iris.fileformats.netcdf.save`.
This can share and re-use any common (lazy) result computations, and it makes much
better use of resources during any file-system waiting (i.e. it can use such periods to
progress the *other* saves).

Usage example::

# Create output files with delayed data saving.
delayeds = [
iris.save(cubes, filepath, compute=False)
for cubes, filepath in zip(output_cubesets, output_filepaths)
]
# Complete saves in parallel.
dask.compute(delayeds)
Comment thread
bjlittle marked this conversation as resolved.
Outdated

This advance also includes **another substantial benefit**, because netcdf saves can
now use a
`Dask.distributed scheduler <https://docs.dask.org/en/stable/scheduling.html>`_.
With `Distributed <https://distributed.dask.org/en/stable/>`_ you can parallelise the
saves across a whole cluster. Whereas previously, the netcdf saving *only* worked with
a "threaded" scheduler, limiting it to a single CPU.

We're so super keen for the community to leverage the benefit of this new
feature within Iris that we've brought this release forward several months.
As a result, this minor release of Iris is intentionally light in content.
However, there are some other goodies available for you to enjoy, such as:
Comment thread
bjlittle marked this conversation as resolved.
Outdated

* Performing lazy arithmetic with an Iris :class:`~iris.cube.Cube` and a
:class:`dask.array.Array`, and
Expand Down