From 2ef146588a0bb002b38e93d950964a2b80580190 Mon Sep 17 00:00:00 2001 From: Adam Dickin Date: Thu, 5 Nov 2026 14:01:45 -0600 Subject: [PATCH] [hipDNN] Document removing reference-disable flags once supported (ALMIOPEN-3138) Ask authors to record the reason for no_cpu_reference/no_gpu_reference in the metadata notes field and to remove the flag once the reference supports the bundle. Add a matching hint to the 'add the op to a reference executor' option. --- dnn-providers/integration-tests/docs/adding-tests.md | 12 +++++++++++- 1 file changed, 11 insertions(+), 1 deletion(-) diff --git a/dnn-providers/integration-tests/docs/adding-tests.md b/dnn-providers/integration-tests/docs/adding-tests.md index 2bb77f393cba..a6d1fccdd15d 100644 --- a/dnn-providers/integration-tests/docs/adding-tests.md +++ b/dnn-providers/integration-tests/docs/adding-tests.md @@ -185,7 +185,10 @@ one of these: - **add golden data** produced by an independent implementation (see [Golden data with DVC](#golden-data-with-dvc)), or -- **add the op to a reference executor**, or +- **add the op to a reference executor**. Then search the bundle metadata + (`*.meta.json`, `cases[].metadata`) for the matching `no_cpu_reference` or + `no_gpu_reference` flag, and try removing it for the ops and shapes the + executor now handles ([Disable a reference for a bundle](#disable-a-reference-for-a-bundle)), or - **set `enforcement_level`** in the metadata to `applicability` (the engine accepts the graph) or `buildable` (its plans compile), so that the bundle claims only what the harness can check today, or @@ -204,6 +207,13 @@ it. The harness never uses a disabled reference as an oracle { "format_version": 1, "generator": "…", "reference_source": "…", "no_cpu_reference": true } ``` +- **Record why in `notes`.** Put the reason the reference is disabled (wrong + result, too slow, op or shape it does not support) in the `notes` metadata + field, so a later reader can tell whether it still applies. +- **Remove the flag once the reference supports the bundle.** A disabled + reference never runs the bundle, so nothing notices when it gains the op or + shape. When you add an op or shape to a reference executor, search the bundle + metadata for the flag and try removing it. - **Disabling both requires golden data.** Add it first ([Golden data with DVC](#golden-data-with-dvc)). A bundle with both flags and no declared golden data fails to load. `verify_golden_bundles.py` reports it