#196 — subquery-aware decomposition v1: NOT EXISTS / NOT IN anti-join arms

book/src/checks/join.anti-join.md

@@ -14,14 +14,19 @@
 ## Conditions
 
 - the model LEFT-JOINs a relation and filters `WHERE <right>.<key> IS NULL` in a top-level AND conjunct, where `<key>` is one of the join's ON equi-key right columns — the anti-join idiom: the join deliberately keeps the UNMATCHED left rows
+- OR the model filters `WHERE NOT EXISTS (SELECT … FROM <inner> WHERE …)` in a top-level AND conjunct, the inner being a single plain named relation whose WHERE carries a correlated reference to the outer query — the resolvable outer↔inner equi-conjuncts are the anti-join keys (cute-dbt#196)
+- OR the model filters `WHERE <col> NOT IN (SELECT <col> FROM <inner>)` in a top-level AND conjunct, the inner being a single plain named relation projecting exactly one column — the membership pair (outer column ↔ inner projected column) is the anti-join key (cute-dbt#196). SQL honesty note: a NULL in the inner column makes NOT IN yield NO rows at all; detection still treats the construct as the anti-join idiom (that is how it is authored), and the matched-row fixture this check recommends is exactly what surfaces the NULL trap
 - the recommendation INVERTS join.left-null-propagation's: the anti-join's risk is the matched class leaking through, so the missing fixture is a left row that DOES match a right row, with `expect` proving it is excluded
-- satisfaction: some unit test's literal givens carry a left row whose ON equi-key matches a right given row (both cells non-NULL, equal on the value-normalized key)
-- supersedes join.left-null-propagation on the same construct: NULL right-side columns are the anti-join's working mechanism, not an untested gap
+- satisfaction (all arms): some unit test's literal givens carry a left/outer row whose key matches an inner/right given row (both cells non-NULL, equal on the value-normalized key)
+- supersedes join.left-null-propagation on the same construct: NULL right-side columns are the anti-join's working mechanism, not an untested gap (the subquery constructs are never enumerated by left-null-propagation at all)
 - given binding follows the union.arm-coverage premise: only `given` entries carry statically-visible rows — an ungiven non-seed input is an empty mock; join sides bind directly by external leaf name or through a single-external simple-FROM closure
 
 ## Exclusions
 
-- the NOT EXISTS / NOT IN anti-join equivalents are NOT detected in v1 — only the LEFT JOIN + IS NULL form is recognized (a declared gap: the construct is silent, never misclassified)
+- negated subqueries anywhere but a top-level AND conjunct of a SELECT's WHERE (OR branches, HAVING, JOIN ON, projections) are NOT detected — different semantics or position, silent, never misclassified
+- non-negated EXISTS / IN subqueries (semi-join and membership inclusion — future evidence consumers) and scalar subqueries are NOT detected
+- a NOT EXISTS / NOT IN whose inner is a derived table, joins or reads several relations, or carries its own WITH clause is NOT detected; an uncorrelated NOT EXISTS (zero outer references) is not a keyed anti-join — silent
+- expression (non-column) correlations and projections are not key material: a correlated NOT EXISTS with no resolvable equi pair, and a NOT IN whose outer column does not resolve to a single relation, degrade to UNKNOWN, never UNCOVERED
 - an IS NULL on a non-key right column is a data filter, not the anti-join idiom — join.left-null-propagation governs that construct
 - an IS NULL inside an OR branch has different semantics and is never treated as the anti-join filter
 - unrecoverable join keys, unresolvable side bindings, external `fixture:` files, non-literal `format: sql` givens, and ungiven seed inputs degrade to UNKNOWN, never UNCOVERED

dbt-project/models/marts/_marts__models.yml

@@ -49,6 +49,24 @@ models:
           The right-side join key — always NULL on kept rows; projected so
           the general left-null check provably fires and is superseded.
 
+  # cute-dbt#196 dogfood — the NOT IN membership anti-join. Carries a
+  # unit test whose givens include a MATCHING outer↔inner pair, so the
+  # PR preview renders join.anti-join COVERED on the not_in construct
+  # (the satisfied twin of customers_with_no_orders' UNCOVERED arm).
+  - name: orders_never_refunded
+    description: >
+      Orders that were never refunded — the <col> NOT IN (SELECT …)
+      membership anti-join idiom (cute-dbt#196 subquery evidence family).
+    columns:
+      - name: order_id
+        description: Order identifier (natural key of the kept rows).
+      - name: customer_id
+        description: Customer who placed the order.
+      - name: order_date
+        description: Date the order was placed.
+      - name: status
+        description: Order status.
+
 unit_tests:
   # DICT format (given + expect). dict only compares listed columns, so
   # the derived `amount_share` / `distinct_status_count` are asserted while
@@ -175,6 +193,44 @@ unit_tests:
           number_of_orders: 1
           customer_lifetime_value: 10
 
+  # cute-dbt#196 dogfood — the anti-join SATISFACTION case: the givens
+  # carry a matching pair (order 1 IS refunded) and expect proves the
+  # matched row is excluded. This is exactly the inverted fixture the
+  # join.anti-join recommendation asks for, so the live preview shows
+  # the not_in construct COVERED with attribution to this test.
+  - name: test_orders_never_refunded_excludes_refunded_orders
+    description: |
+      orders_never_refunded keeps only orders absent from the refunded
+      set. Order 1 is refunded (its id appears in stg_refunds) and order
+      2 is clean, so the output carries order 2 alone — the matched row
+      is provably excluded (the anti-join's risk class, exercised).
+    model: orders_never_refunded
+    given:
+      - input: ref('stg_orders')
+        format: dict
+        rows:
+          - order_id: 1
+            customer_id: 1
+            order_date: '2024-01-01'
+            status: 'completed'
+          - order_id: 2
+            customer_id: 2
+            order_date: '2024-01-02'
+            status: 'completed'
+      - input: ref('stg_refunds')
+        format: dict
+        rows:
+          - refund_id: 1
+            order_id: 1
+            amount: -10
+    expect:
+      format: dict
+      rows:
+        - order_id: 2
+          customer_id: 2
+          order_date: '2024-01-02'
+          status: 'completed'
+
   # EXTERNAL FIXTURE format (cute-dbt#126). Same shape as
   # test_orders_sums_payment_methods, but the stg_payments GIVEN is sourced
   # from an external csv fixture file (tests/fixtures/stg_payments_credit_card.csv)

dbt-project/models/marts/orders_never_refunded.sql

@@ -0,0 +1,22 @@
+-- orders_never_refunded — the cute-dbt#196 NOT IN anti-join dogfood.
+--
+-- The membership anti-join idiom: deliberately keep the orders whose
+-- order_id is NOT IN the refunded set. cute-dbt#173 shipped this form
+-- as a declared exclusion (silent); cute-dbt#196's correlated-subquery
+-- evidence family lifts it — join.anti-join now fires on the
+-- not_in[…] construct. Unlike customers_with_no_orders (the UNCOVERED
+-- showcase), this model carries a unit test whose givens include a
+-- MATCHING pair (order 1 is refunded) with expect proving the matched
+-- row is excluded — the live PR preview renders the arm COVERED with
+-- attribution.
+
+select
+    o.order_id,
+    o.customer_id,
+    o.order_date,
+    o.status
+from {{ ref('stg_orders') }} o
+where o.order_id not in (
+    select r.order_id
+    from {{ ref('stg_refunds') }} r
+)

features/coverage_checks.feature

@@ -102,6 +102,54 @@ Feature: cute-dbt surfaces unique-key coverage findings at the payload level
     And the payload carries no "join.anti-join" finding for "order_emails"
     And the "join.left-null-propagation" finding for "order_emails" suggests a no-match given row
 
+  # cute-dbt#196 — the correlated-subquery evidence family lifts the
+  # declared cute-dbt#173 NOT EXISTS / NOT IN anti-join exclusions: both
+  # forms now fire join.anti-join with the SAME inverted recommendation
+  # (a given row that DOES match, with expect proving the exclusion),
+  # and the same matched-pair satisfaction. left-null-propagation never
+  # enumerates subquery constructs at all.
+  Scenario: A correlated NOT EXISTS anti-join is detected with the inverted recommendation
+    Given the modified coverage model "customers_without_orders" compiles to:
+      """
+      select c.customer_id, c.email
+      from "db"."main"."stg_customers" c
+      where not exists (
+          select 1 from "db"."main"."stg_orders" o
+          where o.customer_id = c.customer_id
+      )
+      """
+    When I render the coverage report
+    Then the payload carries a "join.anti-join" finding for "customers_without_orders" with verdict "uncovered"
+    And the payload carries no "join.left-null-propagation" finding for "customers_without_orders"
+    And the "join.anti-join" finding for "customers_without_orders" suggests a given row that matches
+
+  Scenario: A NOT IN membership anti-join is detected with the inverted recommendation
+    Given the modified coverage model "orders_never_refunded" compiles to:
+      """
+      select o.order_id
+      from "db"."main"."stg_orders" o
+      where o.order_id not in (
+          select r.order_id from "db"."main"."stg_refunds" r
+      )
+      """
+    When I render the coverage report
+    Then the payload carries a "join.anti-join" finding for "orders_never_refunded" with verdict "uncovered"
+    And the "join.anti-join" finding for "orders_never_refunded" suggests a given row that matches
+
+  Scenario: A matching given pair satisfies the NOT EXISTS anti-join
+    Given the modified coverage model "customers_without_orders" compiles to:
+      """
+      select c.customer_id, c.email
+      from "db"."main"."stg_customers" c
+      where not exists (
+          select 1 from "db"."main"."stg_orders" o
+          where o.customer_id = c.customer_id
+      )
+      """
+    And a unit test "test_exclusion_proven" on "customers_without_orders" giving "ref('stg_customers')" rows [{"customer_id": 1}] and "ref('stg_orders')" rows [{"customer_id": 1}]
+    When I render the coverage report
+    Then the payload carries a "join.anti-join" finding for "customers_without_orders" with verdict "covered"
+
   # cute-dbt#164 — incremental.branch-coverage (coverage-intelligence
   # rule #1): an incremental model's unit tests must exercise BOTH
   # is_incremental() branches. A test overriding is_incremental to true

heuristics/registry.toml

@@ -87,18 +87,24 @@ supersedes = [
 ]
 evidence = [
   "cte-graph.left-join-facts",
+  "cte-graph.subquery-facts",
   "cte-graph.body-leaf-table-refs",
   "manifest.unit-test-givens",
 ]
 conditions = [
   "the model LEFT-JOINs a relation and filters `WHERE <right>.<key> IS NULL` in a top-level AND conjunct, where `<key>` is one of the join's ON equi-key right columns — the anti-join idiom: the join deliberately keeps the UNMATCHED left rows",
+  "OR the model filters `WHERE NOT EXISTS (SELECT … FROM <inner> WHERE …)` in a top-level AND conjunct, the inner being a single plain named relation whose WHERE carries a correlated reference to the outer query — the resolvable outer↔inner equi-conjuncts are the anti-join keys (cute-dbt#196)",
+  "OR the model filters `WHERE <col> NOT IN (SELECT <col> FROM <inner>)` in a top-level AND conjunct, the inner being a single plain named relation projecting exactly one column — the membership pair (outer column ↔ inner projected column) is the anti-join key (cute-dbt#196). SQL honesty note: a NULL in the inner column makes NOT IN yield NO rows at all; detection still treats the construct as the anti-join idiom (that is how it is authored), and the matched-row fixture this check recommends is exactly what surfaces the NULL trap",
   "the recommendation INVERTS join.left-null-propagation's: the anti-join's risk is the matched class leaking through, so the missing fixture is a left row that DOES match a right row, with `expect` proving it is excluded",
-  "satisfaction: some unit test's literal givens carry a left row whose ON equi-key matches a right given row (both cells non-NULL, equal on the value-normalized key)",
-  "supersedes join.left-null-propagation on the same construct: NULL right-side columns are the anti-join's working mechanism, not an untested gap",
+  "satisfaction (all arms): some unit test's literal givens carry a left/outer row whose key matches an inner/right given row (both cells non-NULL, equal on the value-normalized key)",
+  "supersedes join.left-null-propagation on the same construct: NULL right-side columns are the anti-join's working mechanism, not an untested gap (the subquery constructs are never enumerated by left-null-propagation at all)",
   "given binding follows the union.arm-coverage premise: only `given` entries carry statically-visible rows — an ungiven non-seed input is an empty mock; join sides bind directly by external leaf name or through a single-external simple-FROM closure",
 ]
 exclusions = [
-  "the NOT EXISTS / NOT IN anti-join equivalents are NOT detected in v1 — only the LEFT JOIN + IS NULL form is recognized (a declared gap: the construct is silent, never misclassified)",
+  "negated subqueries anywhere but a top-level AND conjunct of a SELECT's WHERE (OR branches, HAVING, JOIN ON, projections) are NOT detected — different semantics or position, silent, never misclassified",
+  "non-negated EXISTS / IN subqueries (semi-join and membership inclusion — future evidence consumers) and scalar subqueries are NOT detected",
+  "a NOT EXISTS / NOT IN whose inner is a derived table, joins or reads several relations, or carries its own WITH clause is NOT detected; an uncorrelated NOT EXISTS (zero outer references) is not a keyed anti-join — silent",
+  "expression (non-column) correlations and projections are not key material: a correlated NOT EXISTS with no resolvable equi pair, and a NOT IN whose outer column does not resolve to a single relation, degrade to UNKNOWN, never UNCOVERED",
   "an IS NULL on a non-key right column is a data filter, not the anti-join idiom — join.left-null-propagation governs that construct",
   "an IS NULL inside an OR branch has different semantics and is never treated as the anti-join filter",
   "unrecoverable join keys, unresolvable side bindings, external `fixture:` files, non-literal `format: sql` givens, and ungiven seed inputs degrade to UNKNOWN, never UNCOVERED",