Repository navigation
#196 — subquery-aware decomposition v1: NOT EXISTS / NOT IN anti-join arms - #229
Conversation
…IN anti-join arms (#196 v1) The check engine was blind to logic living in subqueries: #173 shipped join.anti-join detecting only the LEFT JOIN + IS NULL idiom, with the NOT EXISTS / NOT IN equivalents as declared exclusions. This lands the cute-dbt#196 evidence family (v1) and its first consumers. Domain (src/domain/cte.rs): SubqueryFact { kind: SubqueryKind {NotExists, NotIn} (#[non_exhaustive]), consumer, inner_leaf, equi_keys: Vec<JoinKeyPair> } — the LeftJoinFact sibling, riding CteGraph behind #[serde(skip)] (wire-isolation test pins it; the embedded payload shape is byte-identical to pre-#196). Engine (src/adapters/cte_engine.rs): collect_subquery_facts rides the SAME single parse at the same call sites as the LEFT JOIN collection. From top-level WHERE AND conjuncts only: correlated NOT EXISTS over a single plain-table inner (equi pairs normalized OUTER-side-left; a correlated-but-keyless form emits empty keys), and single-column NOT IN (the membership pair; an unresolvable outer side emits empty keys). OR-branch / non-negated / derived-or-multi-table-inner / uncorrelated forms stay silent, each pinned by a negative test. Detector (src/domain/checks.rs): bind_keys / key_match_verdict generalize over an internal KeyedJoinView constructed from both fact families — LEFT JOIN behavior byte-identical (existing tests pin it). detect_join_anti_join gains not_exists[...] / not_in[...] sites with ordinal dedup, form-specific evidence, the same inverted-recommendation sketch, and the same Covered/Uncovered/UNKNOWN ladder. join.left-null-propagation never enumerates subquery constructs (verified by test, not assumed). Spec + ledger: conditions gain both forms (incl. the NOT IN NULL-trap honesty note); the lifted v1 exclusion line is replaced by the residual exclusions; heuristics/registry.toml + book/src/checks/join.anti-join.md regenerated (byte-gated). Tests: the #173 silence pin not_exists_anti_join_is_silent_through_the_ real_engine FLIPS into positive detection; NOT IN positive twin added; covered/uncovered/UNKNOWN through the real engine for both arms; fresh residual-exclusion pins; 14 engine inline tests; 7 detector unit tests; 3 BDD scenarios join coverage_checks.feature (feature count stays 22). Dogfood (the #164 precedent): the synthetic playground-current.json gains int_patients__never_admitted (correlated NOT EXISTS, no unit test) — the golden playground report now renders the not_exists arm UNCOVERED with the inverted sketch (programmatic single-line JSON splice; MANIFEST.toml sha updated; real-fixture pin added). dbt-project/ gains orders_never_refunded (NOT IN) + a unit test whose givens carry a matching pair — verified Covered with attribution against a real fusion-compiled manifest; the live PR preview renders the satisfied twin. jaffle-shop + diff-showcase goldens are byte-identical (no anti-join spec embeds there); playground + explore regenerated and audited delta-by-delta. Closes #196 Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The --document-private-items doc gate reads [NOT] as an intra-doc link. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
|
Warning Review limit reached
More reviews will be available in 25 minutes and 59 seconds. Learn how PR review limits work. Your organization has run out of usage credits. Purchase more credits in the billing tab to continue. ⌛ How to resolve this issue?After more reviews become available, a review can be triggered using the We recommend that you space out your commits to avoid hitting the rate limit. 🚦 How do rate limits work?CodeRabbit enforces hourly rate limits for each developer per organization. Our paid plans include higher PR review limits than trial, open-source, and free plans. In all cases, reviews become available again over time. During sustained high-volume PR review activity, CodeRabbit may temporarily slow when the next review becomes available. Please see our Fair Usage Limits Policy for further information. ℹ️ Review info⚙️ Run configurationConfiguration used: defaults Review profile: CHILL Plan: Pro Run ID: 📒 Files selected for processing (18)
✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
📄 Rendered report previewAll golden examples regenerated cleanly. 🟡 Golden examplesCommitted to
🐶 Live dogfood previewThis PR's own
▶ Open ↗ opens the report in your browser in one click — The Pages preview may take ~1 min to update after this comment Alternative: GitHub CLI# gh CLI >= 2.63 extracts into ./report-preview-playground/.
gh run download 27362224840 -R breezy-bays-labs/cute-dbt -n report-preview-playground
open report-preview-playground/playground-report.htmlPosted by |
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
There was a problem hiding this comment.
Code Review
This pull request implements support for detecting anti-joins authored as NOT EXISTS or NOT IN subqueries. It updates the CTE engine to parse and collect these subquery facts, integrates them into the join.anti-join heuristic check, and updates the documentation and test suites accordingly. The review feedback suggests a minor refactoring in src/adapters/cte_engine.rs to use subquery.body.as_ref() instead of &*subquery.body for more idiomatic Rust code.
Important
The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.
Gemini review on PR #229: `subquery.body.as_ref()` over `&*subquery.body` in conforming_inner's let-else. House-style check confirms it: the `&*` form was the only occurrence in all of src/, while .as_ref() is the file's established explicit-reference idiom. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…nti-join-evidence
1 participant
Design summary
The correlated-subquery evidence family (v1) + its first consumers:
join.anti-joingains NOT EXISTS and NOT IN detection arms with the same supersedes + inverted-recommendation + covered/uncovered/UNKNOWN semantics as the shipped LEFT JOIN + IS NULL form.subquery_factsfamily, never a join-fact lie:SubqueryFact { kind ∈ {NotExists, NotIn} (#[non_exhaustive]), consumer, inner_leaf, equi_keys: Vec<JoinKeyPair> }ridesCteGraphbehind#[serde(skip)]— the #172 — check union.arm-coverage: unexercised UNION ALL arms #191/renderer hygiene: fold post-merge bot findings on PR #38 into the next render-layer touch #40 additive pattern. The embedded report payload is byte-identical to pre-capability: subquery-aware model decomposition — correlated-subquery evidence family for the check engine #196 (wire-isolation test mirrorsleft_join_facts_never_reach_the_wire). Normalizing subqueries intoLeftJoinFactwas the rejected alternative.collect_subquery_factsrides the engine's existing single sqlparser pass, invoked from the exact call sites as the LEFT JOIN collection (every CTE body + the terminal body, WITH-less models included). Extraction reads only top-levelWHEREAND conjuncts (collect_and_conjunctsreuse).bind_keys/key_match_verdictgeneralize over an internalKeyedJoinViewconstructed from both&LeftJoinFactand&SubqueryFact(the subquery's inner relation plays the LEFT JOIN's right-leaf role; pairs are normalized OUTER-side-left). The LEFT JOIN path is byte-identical — every pre-existing join test pins it unchanged. The subquery arms inherit closure binding (resolve_side_external), value-normalized key matching, attribution, and the inverted matching-row sketch for free.join.left-null-propagationnever enumerates subquery constructs (it consumesleft_join_factsonly) — verified by test, not assumed — so per-construct supersedes stays trivially correct.Tier honesty (HIGH, cues never assertions)
not_exists[{consumer}:{leaf}], inverted sketch, matched-pair satisfactionnot_exists_anti_join_fires_through_the_real_engine(the FLIPPED #173 pin) + BDDnot_in[{consumer}:{leaf}], membership pair as the keynot_in_anti_join_fires_through_the_real_engine(new specimen) + BDDcorrelated_but_non_equi_not_exists_yields_a_fact_with_empty_keys+ real-engine verdict testnot_in_unresolvable_outer_column_yields_empty_keys+ real-engine verdict testuncorrelated_not_exists_emits_no_fact+ real-engine residual pinnon_negated_exists_and_in_emit_no_facts+ real-engine residual pinor_branch_not_exists_emits_no_fact+ real-engine residual pinderived_table_inner_emits_no_fact,multi_table_inner_emits_no_factnot_in_multi_column_or_expression_projection_emits_no_factNOT IN NULL-trap honesty: the spec prose states that a NULL in the inner column makes
NOT INyield no rows; detection still treats the construct as the anti-join idiom (that is how it is authored), and the matched-row fixture the check recommends is exactly what surfaces the trap. Stated, never silently ignored.Candidate further consumers (enumerated per the issue AC)
col IN (SELECT …)) — semi-join inclusion; satisfaction needs both a member and a non-member given row.(SELECT max(…) …)in projection/predicate) — pairs with the framing doc's literal-predicate evaluator zone.All arrive as additive
SubqueryKindvariants + consumers — never an engine rewrite.Restoration-sites ledger (the greppable-exclusion discipline)
tests/check_engine.rs—not_exists_anti_join_is_silent_through_the_real_engineflipped intonot_exists_anti_join_fires_through_the_real_engine(comment trail notes the feature: check pair join.left-null-propagation + join.anti-join — supersedes showcase (catalog C4) #173 exclusion is lifted by capability: subquery-aware model decomposition — correlated-subquery evidence family for the check engine #196); fresh residual pins replace it.src/domain/checks.rsJoinAntiJoin spec — the v1 exclusion line ("the NOT EXISTS / NOT IN anti-join equivalents are NOT detected in v1…") replaced by the residual exclusions;evidencegainscte-graph.subquery-facts.heuristics/registry.toml+book/src/checks/join.anti-join.md— regenerated (GEN_HEURISTICS_LEDGER=1), byte-gate green.src/adapters/cte_engine.rs—not_exists_anti_join_emits_no_left_join_factre-commented as the family-separation pin (still true: a subquery never becomes aLeftJoinFact).Dogfood (the #164 precedent)
tests/fixtures/playground-current.jsongainsint_patients__never_admitted— a correlated NOT EXISTS overstg_synthea__encounters, no unit test (programmatic single-line JSON splice; ensure-ascii round-trip verified lossless before splicing;MANIFEST.tomlsha + provenance prose updated). The golden playground report now rendersnot_exists[(final select):stg_synthea__encounters]UNCOVERED with the inverted sketch; a real-fixture test pins it.jaffle-shop+diff-showcasegoldens are byte-identical (no anti-join spec embeds there);playground+exploreregenerated with the exact CI commands and audited at the JSON level — every delta traces to the new model card, thejoin.anti-joincheck_specsentry, ormanifest_nodes.dbt-project/models/marts/orders_never_refunded.sql(NOT IN overstg_refunds) + a unit test whose givens carry a matching pair (order 1 IS refunded) withexpectproving the exclusion. Verified end-to-end against a real fusion-compiled manifest (dbt-fusion 2.0.0-preview.177): the payload carriesjoin.anti-join | not_in[(final select):stg_refunds] | coveredattributed to the new test. The sticky PR preview'sdbt-projectrow renders the satisfied twin live.Gates (run directly — lefthook skips in fresh worktrees)
fmt ✓ · clippy
--all-targets --locked -D warningsexit 0 ✓ · nextest 1222/1222 ✓ · BDD 160 scenarios / 1032 steps ✓ (feature count stays 22) · heuristics_ledger byte-gate ✓ · headless zero-egress (--ignored) ✓ · headless toggle (--ignored) ✓ ·cargo doc-D warnings(incl.--document-private-items, the pre-push mirror) ✓ ·cargo deny check✓ · domain purity (clean-arch test) ✓.Closes #196
🤖 Generated with Claude Code