test(tensilelite): run the unit tests' ParallelMap calls in-process

projects/hipblaslt/tensilelite/TESTING.md

@@ -52,6 +52,14 @@ question that this section returns to below. It matters more than the headline c
 | How to run | `cd tensilelite && tox -e unit` (skips the client build), or `tox -e rocisa` for the extension only |
 | Coverage | `tox -e coverage-unit` measures; `tox -e coverage-gate` enforces the floors against what it wrote |
 
+**Worker pools are not exercised here.** The unit suite pins `CpuThreads` to 1 (a session fixture in
+[`Tests/unit/conftest.py`](tensilelite/Tests/unit/conftest.py)), so its `ParallelMap` and `ParallelMap2`
+calls run in the test process and nothing is pickled or sent to a worker. The real pool path is
+covered by the device-library builds (`TensileCreateLibrary` with the default `CpuThreads` of -1), by
+the subprocess-based `Tests/common/test_config.py` runs, and by one small test in
+[`characterization/Parallel`](tensilelite/Tests/unit/characterization/Parallel/test_parallel_char.py)
+that runs both helpers with `CpuThreads=2`.
+
 A significant part of that suite is a **characterization suite** under
 [`tensilelite/Tests/unit/characterization/`](tensilelite/Tests/unit/characterization/), established in
 [PR #7989](https://github.com/ROCm/rocm-libraries/pull/7989) and grown considerably since. It now

projects/hipblaslt/tensilelite/tensilelite/Tests/unit/characterization/Parallel/test_parallel_char.py

@@ -7,7 +7,10 @@
 global-parameter overwrite, the pcall wrappers, ``apply_print_exception``, and
 the single-threaded / n_jobs=1 (in-process) paths of ``ParallelMap`` /
 ``ParallelMap2``. The real fork/spawn process-pool paths are not unit-driven
-(flaky/slow) — see the suite's target.md."""
+(flaky/slow) — see the suite's target.md — apart from one small tripwire test
+(``test_parallel_maps_use_worker_processes_with_two_threads``). The unit
+conftest pins ``CpuThreads`` to 1, so nothing else in the unit lane sends work
+to a worker process."""
 
 import contextlib
 import importlib
@@ -43,6 +46,13 @@ def _add(a, b):
     return a + b
 
 
+def _where_am_i(_):
+    """Which process ran this, and the ``CpuThreads`` that process sees."""
+    import os
+
+    return os.getpid(), globalParameters["CpuThreads"]
+
+
 # ---------------------------------------------------------------------------
 
 def test_joblib_supports_generator_returns_bool():
@@ -63,6 +73,22 @@ def test_cpu_thread_count_capped():
         assert P.CPUThreadCount(enable=True) == 1
 
 
+def test_parallel_maps_use_worker_processes_with_two_threads():
+    # Tripwire: the unit conftest runs every other ParallelMap call in-process,
+    # so an unpicklable payload or a worker that misses parent state would no
+    # longer fail anywhere in the unit lane. With CpuThreads=2 both helpers
+    # must hand the work to other processes, with the parent's global
+    # parameters visible there.
+    import os
+
+    with _gp(CpuThreads=2, ShowProgressBar=False):
+        pool_rv = P.ParallelMap(_where_am_i, [0, 1])
+        joblib_rv = P.ParallelMap2(_where_am_i, [0, 1], multiArg=False)
+    for pid, cpu_threads in pool_rv + list(joblib_rv):
+        assert pid != os.getpid()
+        assert cpu_threads == 2
+
+
 def test_overwrite_global_parameters():
     from tensilelite.Common import GlobalParameters as GPmod
     saved = dict(GPmod.globalParameters)

projects/hipblaslt/tensilelite/tensilelite/Tests/unit/conftest.py

@@ -22,6 +22,43 @@ def _isolate_rocisa_state():
         yield
 
 
+@pytest.fixture(scope="session", autouse=True)
+def _single_threaded_parallel_map():
+    """Run the unit tests' ``ParallelMap``/``ParallelMap2`` calls in-process.
+
+    Why here: ``CPUThreadCount()`` sizes its pool from ``globalParameters["CpuThreads"]``
+    (default -1, meaning every CPU the process can see). That is the machine's CPU count,
+    not the CI job's CPU allocation, so each pytest/xdist process launches a pool of
+    roughly that many workers (96 on a 96-CPU host) for tests that map over one or a few
+    items. The pool never speeds those tests up, and it multiplies CPU use by the number
+    of xdist workers. A session-scoped autouse fixture in this conftest runs once in every
+    process that runs unit tests, each xdist worker included, so no test or CLI flag has to
+    opt in.
+
+    Why a fixture and not a ``pytest_configure`` hook: pytest runs a nested conftest's
+    ``pytest_configure`` as soon as it imports that conftest during collection, so a run
+    that collects all of ``Tests`` (``tox -e py3 -- tensilelite/Tests -m common``) would
+    also have set ``CpuThreads`` for the non-unit tests that call ``Tensile`` in-process.
+    Autouse fixtures only reach tests under ``unit/``, and the patches are undone when the
+    session ends.
+
+    Why both values: ``restoreDefaultGlobalParameters()``, called by the ``Tensile``,
+    ``TensileUpdateLibrary`` and ``TensileRetuneLibrary`` entry points the tests exercise,
+    clears ``globalParameters`` and refills it from ``defaultGlobalParameters``. Setting
+    only the live value is undone by the first such call and the pools come back.
+    A test that needs a different value sets it for itself, as several already do. The
+    real worker-pool path stays covered by the device-library builds, by the
+    subprocess-based ``Tests/common/test_config.py`` runs, and by one small test in
+    ``characterization/Parallel``.
+    """
+    from tensilelite.Common import GlobalParameters
+
+    with pytest.MonkeyPatch.context() as patch:
+        patch.setitem(GlobalParameters.globalParameters, "CpuThreads", 1)
+        patch.setitem(GlobalParameters.defaultGlobalParameters, "CpuThreads", 1)
+        yield
+
+
 @pytest.fixture(autouse=True)
 def skip_parametrized_if_cli(request):
     """Skip all tests except test_storeD_cli when CLI options (--mn) are provided."""