Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions book/src/SUMMARY.md
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,7 @@
- [join.left-null-propagation](./checks/join.left-null-propagation.md)
- [join.anti-join](./checks/join.anti-join.md)
- [incremental.branch-coverage](./checks/incremental.branch-coverage.md)
- [enforcement.constraint-unbacked](./checks/enforcement.constraint-unbacked.md)
- [Selecting & suppressing checks](./check-selection.md)
- [Experimental features](./experimental.md)
- [Examples](./examples.md)
Expand Down
32 changes: 32 additions & 0 deletions book/src/checks/enforcement.constraint-unbacked.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,32 @@
<!-- GENERATED — do not edit. Source of truth: the `heuristics!` block in src/domain/checks.rs. Regenerate: GEN_HEURISTICS_LEDGER=1 cargo test --test heuristics_ledger -->

# enforcement.constraint-unbacked

**Declared constraint without a backing test**

| | |
|---|---|
| Group | `enforcement` |
| Tier | `total` |
| Instrument | `data-test` |

## Conditions

- the model declares a primary_key or unique constraint (model-level constraints[] or a column-level constraint) whose enforcement on the manifest's adapter is metadata-only (NotEnforced) — the warehouse accepts the declaration but does not enforce uniqueness at write time
- no enabled generic uniqueness data test (unique, attached to the model) backs the constrained column — the constraint→test edge is INFERRED by column + test-name match, since the manifest never links a constraint to its test
- the verdict is UNCOVERED only when the constraint is declared, metadata-only, AND has no inferred backing test; a backing test (even one cute-dbt could miss) keeps it silent

## Exclusions

- a constraint the adapter ENFORCES at write time (e.g. not_null / foreign_key on Postgres/DuckDB) is never a gap — the warehouse guarantees it
- a constraint kind with no column-level generic-test backing (check / custom / foreign_key) is out of this inference and never reported here
- the inferred edge can MISS a renamed test or a singular/custom test asserting the same uniqueness — the copy says "backing test" (an authoring-discipline cue), never that the warehouse lacks an index; columns are authored-YAML-only, so this is never a warehouse-truth claim
- the whole `enforcement` group is gated behind the governance experiment — off by default, it never fires on a non-governance report

## Recommendation

Add a uniqueness data test on the declared-but-unenforced constraint column (`unique` for a single column), so the grain the contract DECLARES is actually verified by a test on every run. The warehouse will not enforce it for you on this adapter.

## Rationale

A primary-key / unique constraint that the warehouse treats as metadata-only is a DECLARED guarantee with nothing checking it: duplicate rows load silently, and any downstream join or incremental merge that trusts the declared grain corrupts. A backing data test is the only thing that actually verifies the declared uniqueness on this adapter.
1 change: 1 addition & 0 deletions book/src/checks/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,3 +11,4 @@ The coverage-intelligence check registry. Each check pairs a construct trigger w
| [`join.left-null-propagation`](./join.left-null-propagation.md) | LEFT JOIN null propagation untested | `high` | `both` |
| [`join.anti-join`](./join.anti-join.md) | Anti-join exclusion untested | `high` | `unit-test` |
| [`incremental.branch-coverage`](./incremental.branch-coverage.md) | Unexercised is_incremental() branch | `high` | `unit-test` |
| [`enforcement.constraint-unbacked`](./enforcement.constraint-unbacked.md) | Declared constraint without a backing test | `total` | `data-test` |
2 changes: 1 addition & 1 deletion examples/diff-showcase-report.html

Large diffs are not rendered by default.

25 changes: 25 additions & 0 deletions heuristics/registry.toml
Original file line number Diff line number Diff line change
Expand Up @@ -139,3 +139,28 @@ exclusions = [
]
recommendation = "Add a unit test for each missing is_incremental() branch: one with `overrides: macros: is_incremental: true` (mock the prior model state with a `given: - input: this` entry) to exercise the incremental branch, and one without the override for the initial full build (dbt compiles is_incremental() as false by default). This finding's evidence carries a copy-pasteable unit-test sketch per missing branch."
rationale = "An incremental model is two programs in one body: the initial full build, and the incremental run that filters on the high-water mark and merges by key. Each unit test compiles only one of them, so a suite living entirely on one branch ships the other untested — exactly where incremental models silently drop, duplicate, or re-process rows."

[[heuristic]]
id = "enforcement.constraint-unbacked"
name = "Declared constraint without a backing test"
group = "enforcement"
tier = "total"
instrument = "data-test"
evidence = [
"manifest.constraints",
"manifest.metadata.adapter-type",
"manifest.test-nodes",
]
conditions = [
"the model declares a primary_key or unique constraint (model-level constraints[] or a column-level constraint) whose enforcement on the manifest's adapter is metadata-only (NotEnforced) — the warehouse accepts the declaration but does not enforce uniqueness at write time",
"no enabled generic uniqueness data test (unique, attached to the model) backs the constrained column — the constraint→test edge is INFERRED by column + test-name match, since the manifest never links a constraint to its test",
"the verdict is UNCOVERED only when the constraint is declared, metadata-only, AND has no inferred backing test; a backing test (even one cute-dbt could miss) keeps it silent",
]
exclusions = [
"a constraint the adapter ENFORCES at write time (e.g. not_null / foreign_key on Postgres/DuckDB) is never a gap — the warehouse guarantees it",
"a constraint kind with no column-level generic-test backing (check / custom / foreign_key) is out of this inference and never reported here",
"the inferred edge can MISS a renamed test or a singular/custom test asserting the same uniqueness — the copy says \"backing test\" (an authoring-discipline cue), never that the warehouse lacks an index; columns are authored-YAML-only, so this is never a warehouse-truth claim",
"the whole `enforcement` group is gated behind the governance experiment — off by default, it never fires on a non-governance report",
]
recommendation = "Add a uniqueness data test on the declared-but-unenforced constraint column (`unique` for a single column), so the grain the contract DECLARES is actually verified by a test on every run. The warehouse will not enforce it for you on this adapter."
rationale = "A primary-key / unique constraint that the warehouse treats as metadata-only is a DECLARED guarantee with nothing checking it: duplicate rows load silently, and any downstream join or incremental merge that trusts the declared grain corrupts. A backing data test is the only thing that actually verifies the declared uniqueness on this adapter."
131 changes: 119 additions & 12 deletions src/cli/mod.rs
Original file line number Diff line number Diff line change
Expand Up @@ -371,7 +371,7 @@ fn execute_report(args: &ReportArgs) -> Result<(), RunError> {
// policy plus inline SQL pragmas scanned from each in-scope model's
// manifest `raw_code`. Display-layer only — applied inside payload
// assembly strictly after supersedes resolution.
let check_policy = build_check_policy(args, &current, &models_in_scope);
let check_policy = build_check_policy(args, &current, &models_in_scope, &experiments);
render(
&args.out,
&current,
Expand Down Expand Up @@ -1118,11 +1118,23 @@ fn build_check_policy(
args: &ReportArgs,
current: &Manifest,
models_in_scope: &ModelInScopeSet,
experiments: &EnabledExperiments,
) -> CheckPolicy<HeuristicId> {
let mut policy = args.config.as_ref().map_or_else(CheckPolicy::default, |c| {
resolve_check_policy::<HeuristicId>(&c.checks)
.expect("[checks] was validated by the --config value-parser at parse time")
});
// cute-dbt#260 Slice 3 — reconcile the experiment-gated `enforcement`
// group with the governance flag, AFTER either policy arm (the
// `CheckPolicy::default()` arm already filters experimental checks;
// the `[checks]` config arm resolves from `Id::ALL` and does not). So
// this single post-step is the one authority: governance OFF removes
// every experimental check from the display set (byte-identical to
// pre-#260 output, and the gate-free `explore` page already uses the
// filtered default); governance ON re-adds them in registry order.
// The detector still EVALUATES regardless (the suppression-hierarchy
// invariant) — this is a display filter only.
reconcile_experimental_checks(&mut policy.displayed, experiments);
// Model order is deterministic (the scope set iterates in node-id
// order), so pragma rule order — and warning order — is stable.
for model_id in models_in_scope.iter() {
Expand All @@ -1144,6 +1156,37 @@ fn build_check_policy(
policy
}

/// Reconcile the experiment-gated checks (the `enforcement` group, gated
/// behind `Experiment::Governance`) in `displayed` with the active
/// experiment set (cute-dbt#260 Slice 3). Governance OFF ⇒ drop every
/// experimental check; governance ON ⇒ ensure every experimental check is
/// present, re-inserted in registry (`HeuristicId::ALL`) order so the
/// display set stays deterministic. The single authority over both
/// policy arms (default + `[checks]` config).
fn reconcile_experimental_checks(
displayed: &mut Vec<HeuristicId>,
experiments: &EnabledExperiments,
) {
use crate::domain::CheckId as _;
// For Slice 3 the only experiment-gated group is `enforcement`,
// gated behind `Experiment::Governance`. Future gated groups join
// this predicate.
let gated_enabled = experiments.is_enabled(Experiment::Governance);
if gated_enabled {
// Rebuild from the registry, keeping declaration order: a
// non-experimental check stays iff it was displayed; an
// experimental check is shown.
let keep: std::collections::BTreeSet<HeuristicId> = displayed.iter().copied().collect();
*displayed = HeuristicId::ALL
.iter()
.copied()
.filter(|id| id.is_experimental() || keep.contains(id))
.collect();
} else {
displayed.retain(|id| !id.is_experimental());
}
}

/// Resolve the experimental opt-in set (cute-dbt#289, epic #288):
/// enabled = `[experimental]` TOML set ∪ `CUTE_DBT_EXPERIMENTAL` env
/// set.
Expand Down Expand Up @@ -1464,21 +1507,80 @@ mod tests {
}

#[test]
fn build_check_policy_without_config_or_pragmas_is_the_default() {
fn build_check_policy_without_config_or_pragmas_is_the_default_minus_enforcement() {
// cute-dbt#260 Slice 3 — governance OFF (the default): the
// `enforcement` group is filtered out of the displayed set, so the
// policy is the default minus the enforcement check(s). Everything
// else (grain/union/join/incremental) stays displayed; no
// suppressions.
use crate::domain::CheckId as _;
let manifest = manifest_of_models(vec![model_with_raw("model.shop.orders", None)]);
let policy = build_check_policy(
&cli("report.html"),
&manifest,
&scope_of(&["model.shop.orders"]),
&EnabledExperiments::default(),
);
let expected: Vec<HeuristicId> = CheckPolicy::default()
.displayed
.into_iter()
.filter(|id: &HeuristicId| id.spec().group != "enforcement")
.collect();
assert_eq!(policy.displayed, expected);
assert!(policy.suppressions.is_empty());
assert!(
!policy
.displayed
.contains(&HeuristicId::EnforcementConstraintUnbacked),
"enforcement is gated off by default",
);
}

/// Governance-enabled experiment set (so the enforcement group is
/// not filtered out — for the registry-shape policy tests).
fn governance_on() -> EnabledExperiments {
EnabledExperiments::from_union(
&std::collections::BTreeSet::from([Experiment::Governance]),
&std::collections::BTreeSet::new(),
)
}

#[test]
fn build_check_policy_with_governance_keeps_the_enforcement_group() {
// Governance ON ⇒ EVERY registered check displays, including the
// experiment-gated `enforcement` group (which `CheckPolicy::default`
// excludes). No config/pragmas ⇒ no suppressions.
use crate::domain::CheckId as _;
let manifest = manifest_of_models(vec![model_with_raw("model.shop.orders", None)]);
let policy = build_check_policy(
&cli("report.html"),
&manifest,
&scope_of(&["model.shop.orders"]),
&governance_on(),
);
assert_eq!(policy.displayed, HeuristicId::ALL.to_vec());
assert!(policy.suppressions.is_empty());
assert!(
policy
.displayed
.contains(&HeuristicId::EnforcementConstraintUnbacked),
);
// And the default policy (governance off) does NOT carry it.
assert!(
!CheckPolicy::<HeuristicId>::default()
.displayed
.contains(&HeuristicId::EnforcementConstraintUnbacked),
"the experiment-gated check is off in the default policy",
);
assert_eq!(policy, CheckPolicy::default());
}

#[test]
fn build_check_policy_resolves_the_config_selection() {
use crate::domain::CheckId as _;
// Registry-shape-robust: `grain.*` removes exactly the grain
// group; every other registered check (e.g. union.arm-coverage,
// cute-dbt#172) stays displayed.
// cute-dbt#172) stays displayed. Governance ON so the enforcement
// group is not also filtered (Slice 3 gating tested separately).
let manifest = manifest_of_models(vec![model_with_raw("model.shop.orders", None)]);
let policy = build_check_policy(
&cli_with_checks(crate::domain::ChecksConfig {
Expand All @@ -1487,14 +1589,14 @@ mod tests {
}),
&manifest,
&scope_of(&["model.shop.orders"]),
&governance_on(),
);
let expected: Vec<HeuristicId> = CheckPolicy::default()
.displayed
.into_iter()
.filter(|id: &HeuristicId| {
use crate::domain::CheckId as _;
id.spec().group != "grain"
})
// Governance ON ⇒ start from the FULL registry (enforcement
// included); `grain.*` removes only the grain group.
let expected: Vec<HeuristicId> = HeuristicId::ALL
.iter()
.copied()
.filter(|id: &HeuristicId| id.spec().group != "grain")
.collect();
assert_eq!(policy.displayed, expected, "grain.* removes only grain");
assert!(
Expand All @@ -1513,6 +1615,7 @@ mod tests {
&cli("report.html"),
&manifest,
&scope_of(&["model.shop.orders"]),
&EnabledExperiments::default(),
);
assert_eq!(
policy.suppressions,
Expand All @@ -1527,6 +1630,7 @@ mod tests {

#[test]
fn build_check_policy_skips_unknown_pragma_ids_and_out_of_scope_models() {
use crate::domain::CheckId as _;
let manifest = manifest_of_models(vec![
model_with_raw(
"model.shop.orders",
Expand All @@ -1542,13 +1646,16 @@ mod tests {
&cli("report.html"),
&manifest,
&scope_of(&["model.shop.orders"]),
&governance_on(),
);
assert!(
policy.suppressions.is_empty(),
"unknown id warns + stays inert; out-of-scope models are not scanned: {:?}",
policy.suppressions
);
assert_eq!(policy.displayed, CheckPolicy::default().displayed);
// Governance ON ⇒ the full registry displays (enforcement
// included), unaffected by the inert unknown pragma.
assert_eq!(policy.displayed, HeuristicId::ALL.to_vec());
}

// -----------------------------------------------------------------
Expand Down
13 changes: 12 additions & 1 deletion src/domain/check_config.rs
Original file line number Diff line number Diff line change
Expand Up @@ -298,8 +298,19 @@ pub struct CheckPolicy<Id: CheckId> {

impl<Id: CheckId> Default for CheckPolicy<Id> {
fn default() -> Self {
// cute-dbt#260 Slice 3 — experiment-gated checks (the
// `enforcement` group, gated behind `Experiment::Governance`) are
// OFF in the default display set, so the gate-free `explore` page
// and every non-governance render never surface them. The report
// run loop's `build_check_policy` re-adds them when governance is
// enabled. Gated checks still EVALUATE — this is a display filter,
// not a registry exclusion (the suppression-hierarchy invariant).
Self {
displayed: Id::ALL.to_vec(),
displayed: Id::ALL
.iter()
.copied()
.filter(|id| !id.is_experimental())
.collect(),
suppressions: Vec::new(),
}
}
Expand Down
Loading
Loading