#170: unique_combination 7th test primitive

.claude/rules/business-rule-tests.md

@@ -1,8 +1,8 @@
-# Custom business-rule tests + variant-extension pattern (`custom_sql` 5th; `row_count_between` 6th)
+# Custom business-rule tests + variant-extension pattern (`custom_sql` 5th; `row_count_between` 6th; `unique_combination` 7th)
 
-Established by issue #116 (`custom_sql`) and generalised by issue #169 (`row_count_between`) — this is now a **2-instance precedent** for adding a new variant to the `CandidateTest` discriminated union. Apply to any code touching either variant — drafting, prompt-rendering, resolving Jinja, compiling/pruning, grading, diffing, writing to disk, or ingesting from `tests/*.sql` / `dbt_expectations.*` macros.
+Established by issue #116 (`custom_sql`), generalised by issue #169 (`row_count_between`), and extended by issue #170 (`unique_combination`) — this is now a **3-instance precedent** for adding a new variant to the `CandidateTest` discriminated union. Apply to any code touching any of the three variants — drafting, prompt-rendering, resolving Jinja, compiling/pruning, grading, diffing, writing to disk, or ingesting from `tests/*.sql` / `dbt_expectations.*` / `dbt_utils.*` macros.
 
-`custom_sql` (#116) is the first test type that is a **full singular-test SELECT** (returns failing rows) rather than a parameterless/parameterised dbt schema test. `row_count_between` (#169) is the first **model-level-only** variant with a structured args contract (`minimum` / `maximum` / optional `where`). Both encode Architectural Commitment #1 for business rules an operator can't express with the four original built-ins: SignalForge drafts them (from `meta.signalforge.business_rules`, LLM inference, or — for `row_count_between` — `dbt_expectations.expect_table_row_count_to_be_between` ingest), then **prunes and grades** them like everything else — an always-pass business rule is dropped, not shipped.
+`custom_sql` (#116) is the first test type that is a **full singular-test SELECT** (returns failing rows) rather than a parameterless/parameterised dbt schema test. `row_count_between` (#169) is the first **model-level-only** variant with a structured args contract (`minimum` / `maximum` / optional `where`). `unique_combination` (#170) is the **second model-level-only** variant — composite-key uniqueness over a `tuple[str, ...]` of columns plus optional `where`; emits as `dbt_utils.unique_combination_of_columns` YAML. All three encode Architectural Commitment #1 for business rules an operator can't express with the four original built-ins: SignalForge drafts them (from `meta.signalforge.business_rules`, LLM inference, or — for `row_count_between` and `unique_combination` — `dbt_expectations.*` / `dbt_utils.*` ingest recognition), then **prunes and grades** them like everything else — an always-pass business rule is dropped, not shipped.
 
 ## The variants (#116 DEC-002; #169 DEC-001)
 
@@ -121,4 +121,18 @@ No logging, no audit writer in either path (stage-0; `ingest-layer.md`).
 
 ## Reference
 
-`plans/super/116-business-rule-tests.md` — DEC-001 … DEC-015 + the QG fixes (the 1-instance precedent: `custom_sql`). `plans/super/169-row-count-between.md` — DEC-001 … DEC-015 + the US-007a CTE-shape correction + the US-012 ingest anchor exemption (the 2-instance precedent: `row_count_between`). See-Also: `llm-drafter.md` (prompt + drafter `_PROMPT_VERSION`, sqlglot type-coherence from #159), `grade-layer.md` (grade `_PROMPT_VERSION`, rubric calibration), `prune-engine.md` (DropReason lock, conservative-bias routing, materialised sample + per-test override), `diff-renderer.md` (proposed artifacts, fail-closed writer, audit_schema_version), `ingest-layer.md` (stage-0 reader, closed SkipReason, anchor-contract collect-all), `cli-layer.md` (exit-code taxonomy, 5-surface parity), `manifest-readers.md` (resolver + source registry, drift detectors), `warehouse-adapters.md` (`_sql_safety`, `maximum_bytes_billed`).
+`plans/super/116-business-rule-tests.md` — DEC-001 … DEC-015 + the QG fixes (the 1-instance precedent: `custom_sql`). `plans/super/169-row-count-between.md` — DEC-001 … DEC-015 + the US-007a CTE-shape correction + the US-012 ingest anchor exemption (the 2-instance precedent: `row_count_between`). `plans/super/170-unique-combination.md` — DEC-001 … DEC-017 + the QG findings (the 3-instance precedent: `unique_combination`; covers SORTED `columns` canonical hash form DEC-011, grade-side `_PROMPT_VERSION` snapshot surface DEC-012, custom `__repr__` redaction retroactive on existing variants DEC-013, the two-conditional engine routing pattern in `prune.engine` — `all_bypass_to_source` short-circuit AND per-test `per_test_table_ref` override). See-Also: `llm-drafter.md` (prompt + drafter `_PROMPT_VERSION`, sqlglot type-coherence from #159), `grade-layer.md` (grade `_PROMPT_VERSION`, rubric calibration), `prune-engine.md` (DropReason lock, conservative-bias routing, materialised sample + per-test override), `diff-renderer.md` (proposed artifacts, fail-closed writer, audit_schema_version), `ingest-layer.md` (stage-0 reader, closed SkipReason, anchor-contract collect-all), `cli-layer.md` (exit-code taxonomy, 5-surface parity), `manifest-readers.md` (resolver + source registry, drift detectors), `warehouse-adapters.md` (`_sql_safety`, `maximum_bytes_billed`).
+
+## #170 lessons worth carrying forward
+
+Five durable conventions surfaced during #170's QG that the next variant-extension should pre-empt:
+
+1. **Two engine conditionals, not one** — sample-mode source override has TWO sites: `all_bypass_to_source` (short-circuit when EVERY candidate bypasses) AND `per_test_table_ref` (per-test override inside the loop). The US-005b worker discovered this; both sites grow in lockstep when adding a metadata-bypass variant. Single-variant tests only exercise the short-circuit; a mixed-candidate test (1× bypassing + 1× not-bypassing) is **load-bearing** for the per-test arm (see `tests/prune/test_engine.py::test_prune_tests_mixed_candidates_per_test_override_routes_unique_combination_to_source` — the routing pin Pass 3 of #170's QG flagged was missing).
+
+2. **`__repr__` redaction is not enough; Pydantic v2 has `__repr_args__` / `__rich_repr__` / `__pretty__`** — DEC-013 of #170 added `__repr__` redaction for 3 variants carrying LLM-emitted text (`where` / `sql` / `rationale`). QG Pass 1 empirically verified that `rich.print()` / `devtools.pretty()` / `pprint` reach through `__repr_args__()` rather than `repr()` and would otherwise still leak. Add `__repr_args__` overrides in lockstep so all three structured-debug hooks inherit the filter. Tests must exercise `instance.__repr_args__()` directly (the `repr()`-only test misses the leak).
+
+3. **SORT a tuple-shaped arg in the canonical hash, never preserve order** — DEC-011 of #170 made the `columns` tuple sort before hashing in `_common.artifact_id.model_test_args_hash`. Pinned by `tests/diff/test_artifact_id.py::test_unique_combination_columns_sorted_before_hashing` + the three-column permutation test. Rationale: `(a, b)` and `(b, a)` describe the same constraint (GROUP BY result-row identity). Mirrors the `accepted_values.values` precedent. The diff EMITTER preserves declared order; only the hash sorts (split contract).
+
+4. **mkdocs anchor corruption from ATX `##` inside a fenced code block** — the `## BUSINESS RULES` literal inside a ` ```text ` fence in `docs/draft-ops.md` (originally added by #163) silently broke 14 of 20 H2 anchors downstream of the block. mkdocs's anchor generator scans for ATX headings without respecting code-fence boundaries; the fix is to use an indented (4-space) code block, which IS respected. Worth pre-empting in any new ops doc that needs to show a literal `##` heading in an example.
+
+5. **Drive-by formatting reveals a pre-merge gap** — three of #170's workers (US-005a, US-007, US-002) all independently "drive-by formatted" `tests/draft/test_parser.py` despite US-004's own validation having passed `ruff format --check .`. Pre-merge `ruff format --check` against the LATEST `dev` HEAD (not just the worker's worktree HEAD) would catch this class of merge-induced format drift. A maintainer note in the closeout flow: re-run `ruff format --check .` after merging each worker branch, not just at the worker's own commit time.

.claude/rules/grade-layer.md

@@ -68,6 +68,8 @@ Every `GradeEvent` carries five 16-hex blake2b-8 fingerprints:
 
 The four default criterion texts (DEC-016) are locked verbatim and tested for stability via a pinned golden hash. Changing the text is a reproducibility break — bump `audit_schema_version` if it happens.
 
+**Grade-side `_PROMPT_VERSION` snapshot surface (#170 DEC-012).** Issue #170 closed an asymmetry that had drifted into `business-rule-tests.md`: the rule file historically claimed "two `_PROMPT_VERSION` constants in the pipeline, each with its own cache-stability snapshot" — but only the drafter side had a snapshot. The grade side carried `rubric_hash` / `prompt_version_template` / `criterion_prompt_hash` dynamically per-event but had no module-level constant and no pinning test. #170 added `signalforge.grade.prompts._PROMPT_VERSION` (typed `Final[str]`, computed at import via `prompt_version_template(DEFAULT_RUBRIC)`) plus `tests/grade/test_prompt_cache_stability.py` mirroring the drafter snapshot shape — pins both the `_PROMPT_VERSION` hex AND a rendered rubric-block golden with `difflib.unified_diff` on mismatch. **Rotation policy: rotate the snapshot when the grade `_SYSTEM_PROMPT` text changes OR any of the four `DEFAULT_RUBRIC` criterion texts change.** Mirrors the drafter-side `_TEST_CATALOGUE_LINES` rotation contract (`llm-drafter.md`). A new variant's grade-rubric extension (e.g. #170's `no-redundant` extension for grain-meaningfulness) rotates the constant value naturally; pre-existing pinned tests fail loud and the new value is computed and pinned in the same commit (`tests/grade/test_rubric.py` + `tests/grade/test_prompts.py` rotate alongside).
+
 ## `_artifact_id_for` canonical dotted-path format (DEC-009, issue #42 hoist)
 
 Six shapes the formatter emits:

CHANGELOG.md

@@ -6,6 +6,24 @@ All notable changes to SignalForge are documented here. The format is loosely ba
 
 ### Added
 
+- **`unique_combination` as the 7th first-class `CandidateTest` variant (#170).** End-to-end coverage across drafter + prune + grade + diff + ingest. The drafter proposes it as a structured candidate for models whose natural grain is a composite key — `(order_id, line_item_id)` on an order-line table, `(user_id, day)` on a daily activity rollup, `(start_station_id, end_station_id, trip_date)` on a trip-pairs aggregate — with the no-`where` and with-`where` JSON shapes both illustrated in the prompt catalogue and a cautionary `_UNIQUE_COMBINATION_SCOPE_INSTRUCTION` block steering the drafter away from vacuously-unique tuples like `(primary_key, anything)`; the prune compiler emits a multi-column `GROUP BY <cols> HAVING COUNT(*) > 1` (dialect-driven, BigQuery + Snowflake snapshots green) routed past the materialised-sample substitution to the source table (sample-mode false-negative bypass mirroring `row_count_between`); the grade rubric's existing `no-redundant` criterion is extended to score grain-meaningfulness for composite-key tuples (`(pk, anything)` shapes route to `flagged`, not a new degrade slot); the diff emitter renders kept artifacts as `{dbt_utils.unique_combination_of_columns: {combination_of_columns: [...], where: "..."}}` (null fields omitted); `prune-existing` ingest now recognises `dbt_utils.unique_combination_of_columns` from external `schema.yml` and promotes it to the typed variant — same shape as `expect_table_row_count_to_be_between` promotion from #169 (other `dbt_utils.*` macros continue to skip-record). Operators can suppress via `exclude_tests=["unique_combination"]` in `signalforge.yml`. The `args_hash` canonical form sorts the `columns` tuple before hashing (DEC-011) so `(a, b)` and `(b, a)` share one cache key — the GROUP BY result-row identity is column-order-invariant.
+- **`dbt_utils.unique_combination_of_columns` ingest recognition (#170).** New parser arm in `signalforge.ingest.parser._parse_named_test` promotes the macro from `SkippedTest(reason="custom-or-generic-test")` to the typed `CandidateTestUniqueCombination` variant on `prune-existing` runs. Inbound mapping: `combination_of_columns` → `columns`, `where` → `where` (inverted by the diff emitter on outbound — round-trips to the same dbt-utils YAML). Malformed shapes (missing key, empty / `< 2` columns, duplicate items, non-string items, column-scoped usage, non-string `where`) route to `SkippedTest(reason="malformed-supported-test")`; the closed 3-value `SkipReason` literal stays locked. Operators without `dbt-utils` installed in their project will see a clear `dbt parse` error when applying the diff — there is no `.sql` fallback in v1, mirroring the `dbt-expectations` posture established in #169 (operators install the package or remove the test).
+
+### Docs
+
+- **New `docs/drafter-catalogue.md` — SSOT for the test catalogue (#170).** Single-source-of-truth ops doc enumerating the seven first-class primitives (`not_null`, `unique`, `accepted_values`, `relationships`, `custom_sql`, `row_count_between`, `unique_combination`) with YAML examples, structural slots, ingest signatures, and rubric coverage; a "`custom_sql` is the catch-all" sub-section clarifying that it is graded ad-hoc rather than against per-primitive calibration; and a "What we do NOT generate today" boundary section naming the test classes SignalForge explicitly does not propose (column value range / statistical thresholds, conditional uniqueness beyond a simple `where`, statistical / distributional anomalies, cross-table reconciliation, time-series anomaly detection). Linked from README, `business-rule-tests.md`, `docs/draft-ops.md`, `docs/ingest-ops.md`, `docs/prune-ops.md`, and `docs/grade-ops.md`. Wired into the mkdocs site nav between "Claude Code Skill" and "Pipeline Stages."
+- **README "What tests SignalForge generates" section (#170).** New compact 7-row table sitting between "What it does" and "How it works" so prospective evaluators can decide at a glance whether SignalForge's catalogue covers their patterns; links to the SSOT for example YAML and the full boundary.
+- **`docs/draft-ops.md` § Composite uniqueness (`unique_combination`) (#170).** New subsection mirroring the `row_count_between` shape — what the variant is, when the drafter proposes it, a worked example for `fct_order_line_items`, and the `exclude_tests` short-circuit.
+- **`docs/ingest-ops.md` § Recognition of `dbt_utils.unique_combination_of_columns` (#170).** New subsection mirroring the `expect_table_row_count_to_be_between` recognition section — inbound field mapping, skip-recorded shapes, the unchanged-other-`dbt_utils.*`-macros boundary.
+- **`docs/grade-ops.md` § Composite-key calibration (`unique_combination`) (#170).** Paragraph extension of the `no-redundant` criterion narrative naming the `unique_combination` calibration and the vacuously-unique-tuple shape; the criterion was extended rather than added as a fifth.
+- **`docs/prune-ops.md` § `unique_combination` (#170).** Callout in the row-count cost model section documenting the engine's source-vs-temp routing override (composite uniqueness on a sample is semantically approximate; engine always full-scans the source).
+
+### Changed
+
+- **Drafter `_PROMPT_VERSION` rotated for the 7th catalogue entry (#170).** `77e9ee8a6ae7d875` → `389c8aa970df86cc`. The new entry illustrates BOTH the no-`where` and with-`where` shapes plus a `_UNIQUE_COMBINATION_SCOPE_INSTRUCTION` cautionary block. Only the system prompt changed; the cached-block golden (manifest summary) is unchanged. Operators upgrading from a cached prior version pay a one-time Anthropic prompt-cache miss on their next run before the new prompt warms — expected and inexpensive (the cached block dwarfs the system prompt by a wide margin).
+- **Grade rubric `no-redundant` criterion extended for `unique_combination` grain-meaningfulness (#170).** The criterion text grew sibling prose covering composite-key tuples (`unique_combination`) alongside the existing numeric-bound prose (`row_count_between`) — vacuously-unique `(primary_key, anything)` tuples score low and route to `flagged`. Rubric stays at four criteria.
+- **Grade-side `_PROMPT_VERSION` snapshot surface established (#170).** New `signalforge.grade.prompts._PROMPT_VERSION` constant + `tests/grade/test_prompt_cache_stability.py` pinning the rubric hash, mirroring the drafter side. Closes the asymmetry where the rule file claimed a grade-side rotation contract that did not previously exist. Current value: `4dae4421972e9c2d`. As with the drafter rotation above, operators upgrading from a cached prior version pay a one-time prompt-cache miss on their next grade run before the new rubric warms.
+
 - **`row_count_between` as a 6th first-class `CandidateTest` variant (#169).** End-to-end coverage across drafter + prune + grade + diff + ingest. The drafter proposes it as a structured candidate against bounded aggregations (model SQL with a `GROUP BY` + date-window, daily/weekly rollups, monitoring-shaped reports) with the no-`where` and with-`where` JSON shapes both illustrated in the prompt catalogue; prune evaluates the candidate against the warehouse via a failing-rows CTE wrapping a single `COUNT(*)` (sample-mode deliberately bypassed — the engine's per-test loop routes `row_count_between` past the materialised-sample substitution back to the source table because a COUNT against a sample returns the sample size, not the model's true row count); the grade rubric's existing `no-redundant` criterion is extended with calibration prose scoring whether the bound is a meaningful guardrail vs. trivially satisfiable (vacuous `minimum=0`/no-`maximum` bounds score low → `passed=False` → ship as `flagged`, not `kept-uncertain`, and the 3-trigger degrade taxonomy stays locked); the diff emitter renders kept artifacts as `{dbt_expectations.expect_table_row_count_to_be_between: {min_value: N, max_value: M, where: "..."}}` (null fields omitted, no `.sql` fallback in v1 — operators without `dbt-expectations` installed see a clear `dbt parse` error); `prune-existing` ingest now recognises `dbt_expectations.expect_table_row_count_to_be_between` from external `schema.yml` and promotes it to the typed variant (closes AC-5 — other `dbt_expectations.*` macros continue to skip-record). Operators can suppress via `exclude_tests=["row_count_between"]` in `signalforge.yml`. Empty-table → `kept` is the intended behaviour (catching "the upstream produced nothing" is what the test exists to do — documented in `prune-ops.md` so the routing isn't surprising).
 
 ### Docs