Automatically install iris-test-data for benchmark data generation

benchmarks/README.md

@@ -41,13 +41,13 @@ if it is not already. You can achieve this by either:
 
 ### Environment variables
 
-* `OVERRIDE_TEST_DATA_REPOSITORY` - required - some benchmarks use
-`iris-test-data` content, and your local `site.cfg` is not available for
-benchmark scripts.
 * `DATA_GEN_PYTHON` - required - path to a Python executable that can be
 used to generate benchmark test objects/files; see
 [Data generation](#data-generation). The benchmark runner sets this 
-automatically, but will defer to any value already set in the shell.
+automatically, but will defer to any value already set in the shell. Note that
+[Mule](https://github.com/metomi/mule) and 
+[iris-test-data](https://github.com/SciTools/iris-test-data) will be 
+automatically installed into this environment.
 * `BENCHMARK_DATA` - optional - path to a directory for benchmark synthetic
 test data, which the benchmark scripts will create if it doesn't already
 exist. Defaults to `<root>/benchmarks/.data/` if not set. Note that some of

benchmarks/bm_runner.py

@@ -91,17 +91,16 @@ def _prep_data_gen_env() -> None:
         ).resolve()
         environ[data_gen_var] = str(data_gen_python)
 
+        def clone_resource(name: str, clone_source: str) -> Path:
+            resource_dir = data_gen_python.parents[1] / "resources"
+            resource_dir.mkdir(exist_ok=True)
+            clone_dir = resource_dir / name
+            if not clone_dir.is_dir():
+                _subprocess_runner(["git", "clone", clone_source, str(clone_dir)])
+            return clone_dir
+
         echo("Installing Mule into data generation environment ...")
-        mule_dir = data_gen_python.parents[1] / "resources" / "mule"
-        if not mule_dir.is_dir():
-            _subprocess_runner(
-                [
-                    "git",
-                    "clone",
-                    "https://github.com/metomi/mule.git",
-                    str(mule_dir),
-                ]
-            )
+        mule_dir = clone_resource("mule", "https://github.com/metomi/mule.git")
         _subprocess_runner(
             [
                 str(data_gen_python),
@@ -112,6 +111,12 @@ def _prep_data_gen_env() -> None:
             ]
         )
 
+        echo("Installing iris-test-data into data generation environment ...")
+        test_data_dir = clone_resource(
+            "iris-test-data", "https://github.com/SciTools/iris-test-data.git"
+        )
+        environ["OVERRIDE_TEST_DATA_REPOSITORY"] = str(test_data_dir / "test_data")
+
         echo("Data generation environment ready.")
 
 

docs/src/whatsnew/latest.rst

@@ -108,6 +108,9 @@ This document explains the changes made to Iris for this release
 #. `@ESadek-MO`_ added further `benchmarks` for aggregation and collapse.
    (:pull:`5954`)
 
+#. `@trexfeathers`_ set the benchmark data generation environment to
+   automatically install iris-test-data during setup. (:pull:`5958`)
+
 
 .. comment
     Whatsnew author names (@github name) in alphabetical order. Note that,