Avoid mutating the input cubes when building the combined cube in preprocessor function multi_model_statistics

[bouweandela]

Description

Avoid mutating the input cubes when building the combined cube in preprocessor function multi_model_statistics.

Issue was introduced in #2518 and is not present in any released version of the tool.

Closes #2559

---

Before you get started

• ☝ Create an issue to discuss what you are going to do

Checklist

It is the responsibility of the author to make sure the pull request is ready to review. The icons indicate whether the item will be subject to the 🛠 Technical or 🧪 Scientific review.

• 🧪 The new functionality is relevant and scientifically sound
• 🛠 This pull request has a descriptive title and labels
• 🛠 Code is written according to the code quality guidelines
• 🧪 and 🛠 Documentation is available
• 🛠 Unit tests have been added
• 🛠 Changes are backward compatible
• 🛠 The list of authors is up to date
• 🛠 All checks below this pull request were successful

---

To help with the number pull requests:

• 🙏 We kindly ask you to review two other open pull requests in this repository

[codecov]

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 94.97%. Comparing base (865ea7d) to head (b42f07d).
Report is 54 commits behind head on main.

Additional details and impacted files

@@           Coverage Diff           @@
##             main    #2564   +/-   ##
=======================================
  Coverage   94.97%   94.97%           
=======================================
  Files         254      254           
  Lines       14747    14748    +1     
=======================================
+ Hits        14006    14007    +1     
  Misses        741      741           

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

[bettina-gier]

Just commenting to say that this fixes the issue, the new portrait plot recipe runs successfully with this Pr as tested by @lukruh and me.

[bouweandela]

It looks like non-lazy statistics operators will soon be a thing of the past (last one in-progress in SciTools/iris#6167), so maybe it's not worth it to spend time on writing tests for this case.

[schlunma]

I tested this last Friday with

esmvaltool run ~/ESMValTool/esmvaltool/recipes/recipe_perfmetrics_CMIP5.yml --search_esgf=when_missing --max-parallel-tasks=4

The good news is that the recipe runs successfully now. The bad news is that it takes more than 6 hours to run (with the threaded scheduler); in the previous recipe testing, it finished in ~45min. The main reason seems to be diagnostics with the variable ta:

...
2025-01-17 23:50:48,725 UTC [1608452] INFO    Successfully completed task ta5/ta (priority 37) in 1:51:59.521698
...
2025-01-18 02:59:26,616 UTC [1608450] INFO    Successfully completed task taZONAL/ta (priority 3) in 5:56:36.845380
...

Here is an example log: job_14827486.out.log

I also did a test with the distributed scheduler, but that failed (my guess would be that something is not fully lazy, but I didn't investigate this further).

Probably also not related to this PR.

[schlunma]

Tested this again with the distributed scheduler, this time with 8GB of memory per worker instead of 4GB. The recipe now ran successfully in 45min. Merging this now 🚀

[bouweandela]

That seems slow indeed, it used to run in about 15 minutes for v2.8.