Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
33 changes: 24 additions & 9 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -126,15 +126,13 @@ narrow.
over-report relative to dbt's unrendered-config diff (it never
under-reports), and dbt's `persisted_descriptions` sub-selector has no
cute-dbt modifier yet.
- **`source()` references are not bound to fixtures.** dbt resolves
`source('package', 'name')` to a relation name at `dbt compile`
time, so the compiled SQL the renderer sees no longer carries the
`source()` form — only the resolved relation. Binding a unit-test
`given: source(...)` to its import-CTE node requires reading the
manifest's `sources` block and re-resolving on the renderer side.
Tracked:
[`cute-dbt#57`](https://github.com/breezy-bays-labs/cute-dbt/issues/57)
→ v0.2 sources widening.
- **`source()` givens bind via the dominant authored form only.** The
given-input parsers accept dbt's serialized single-quoted positional
form (`source('a', 'b')`, `ref('name')`, whitespace/keyword-case
tolerant). The engine-valid but rare double-quoted and
`name=`/`table_name=` keyword-argument variants are deliberately not
parsed — an unparsed given simply stays unbound (the empty-state
copy), never an error.

## Import-CTE binding

Expand Down Expand Up @@ -165,6 +163,23 @@ case-insensitive:
single CTE node. Tracked:
[`cute-dbt#34`](https://github.com/breezy-bays-labs/cute-dbt/issues/34).

A `given: source('a', 'b')` input takes one extra resolution hop before
the same two-pass match
([`cute-dbt#57`](https://github.com/breezy-bays-labs/cute-dbt/issues/57)):
dbt resolves `{{ source('a', 'b') }}` to the physical relation at
`dbt compile` time, so the compiled SQL never carries the literal
`source(...)` form. cute-dbt therefore resolves the authored
`(source_name, table)` pair against the manifest's top-level `sources`
block, takes the entry's physical `identifier` (falling back to the
last segment of `relation_name`, then to `name` — dbt's identifier
default), strips identifier quoting, and feeds that token through the
identical two-pass match above. The lookup runs on the YAML
`source_name` + `name` (the authored arguments), never on `identifier`
— an overridden identifier still binds. A pair missing from the
`sources` block leaves the given unbound — fail-open, same as an
unresolvable `ref()`; sources need no preflight (they are referenced by
models, never analyzed themselves).

When neither pass matches — and the node is an `Import` CTE — the
node-detail panel surfaces *"no fixture provided — dbt treats
unspecified inputs as empty"*. The same empty-state is the
Expand Down
2 changes: 1 addition & 1 deletion examples/playground-report.html

Large diffs are not rendered by default.

143 changes: 141 additions & 2 deletions src/adapters/manifest.rs
Original file line number Diff line number Diff line change
Expand Up @@ -44,7 +44,7 @@ use serde::Deserialize;

use crate::domain::{
Checksum, DependsOn, Manifest, ManifestMetadata, Node, NodeConfig, NodeId, PreflightError,
TestMetadata, UnitTest, UnitTestExpect, UnitTestGiven,
SourceNode, TestMetadata, UnitTest, UnitTestExpect, UnitTestGiven,
};
use crate::ports::ManifestSource;
use serde_json::Value;
Expand Down Expand Up @@ -78,6 +78,8 @@ struct WireManifest {
unit_tests: HashMap<String, WireUnitTest>,
#[serde(default)]
macros: HashMap<String, WireMacro>,
#[serde(default)]
sources: HashMap<String, WireSource>,
}

/// Wire projection of one `nodes` entry.
Expand Down Expand Up @@ -184,6 +186,53 @@ struct WireMacro {
macro_sql: String,
}

/// Wire projection of one top-level `sources` entry (cute-dbt#57).
///
/// Like [`WireNode`], no `id` field — dbt keys the `sources` map by
/// `unique_id` (`source.<package>.<source_name>.<name>`) and the map key
/// is folded into the domain [`SourceNode`] during translation.
///
/// **Every** field is `#[serde(default)] Option<…>` — the cute-dbt#145
/// engine-divergence rule applied verbatim: dbt-core emits explicit
/// `null` for unset fields (an `Option` is required; a bare `String`
/// rejects `null`), while fusion's `#[skip_serializing_none]` omits the
/// keys entirely (`default` covers absence). One malformed source entry
/// must never fail the whole manifest parse (ADR-5); a source whose
/// `source_name` / `name` defaults to `""` simply never matches a parsed
/// `source('a', 'b')` given — the same fail-open posture as an
/// unresolvable `ref(...)`.
#[derive(Debug, Deserialize)]
struct WireSource {
#[serde(default)]
source_name: Option<String>,
#[serde(default)]
name: Option<String>,
#[serde(default)]
identifier: Option<String>,
#[serde(default)]
schema: Option<String>,
#[serde(default)]
database: Option<String>,
#[serde(default)]
relation_name: Option<String>,
}

impl WireSource {
/// Translate into the domain [`SourceNode`], folding the
/// authoritative map `key` into the id (the [`WireNode`] precedent).
fn into_domain(self, key: String) -> SourceNode {
SourceNode::new(
NodeId::new(key),
self.source_name.unwrap_or_default(),
self.name.unwrap_or_default(),
self.identifier,
self.schema.unwrap_or_default(),
self.database,
self.relation_name,
)
}
}

/// Wire projection of one `unit_tests` map entry.
///
/// The domain [`UnitTest`] type stores `tags` and `meta` flat, but the
Expand Down Expand Up @@ -360,7 +409,12 @@ impl WireManifest {
.into_iter()
.map(|(key, wire)| (key, wire.macro_sql))
.collect();
Manifest::new(self.metadata, nodes, unit_tests, macros)
let sources = self
.sources
.into_iter()
.map(|(key, wire)| (NodeId::new(key.clone()), wire.into_domain(key)))
.collect();
Manifest::new(self.metadata, nodes, unit_tests, macros).with_sources(sources)
}
}

Expand Down Expand Up @@ -783,6 +837,91 @@ mod tests {
assert!(manifest.nodes().is_empty());
assert!(manifest.unit_tests().is_empty());
assert!(manifest.macros().is_empty());
assert!(manifest.sources().is_empty());
}

// ----- cute-dbt#57: top-level `sources` block ---------------------

#[test]
fn parse_manifest_translates_a_core_style_source_entry() {
// dbt-core 1.11 dialect: unset Option fields serialize as
// explicit `null` (here `database`), and the entry carries
// sibling keys cute-dbt does not consume (ADR-5 tolerance).
let json = format!(
r#"{{
"metadata": {{ "dbt_schema_version": "{V12_URL}" }},
"sources": {{
"source.shop.synthea_raw.patients": {{
"database": null,
"schema": "main",
"name": "patients",
"resource_type": "source",
"source_name": "synthea_raw",
"identifier": "patients",
"relation_name": "\"memory\".\"main\".\"patients\"",
"loaded_at_field": null,
"freshness": null,
"quoting": {{ "database": null }}
}}
}}
}}"#
);
let manifest = parse_manifest(&json).expect("core-style source entry parses");
assert_eq!(manifest.sources().len(), 1);
let source = manifest
.source_by_name("synthea_raw", "patients")
.expect("the (source_name, name) pair resolves");
assert_eq!(source.id().as_str(), "source.shop.synthea_raw.patients");
assert_eq!(source.schema(), "main");
assert_eq!(source.identifier(), Some("patients"));
assert_eq!(source.database(), None, "explicit null → None");
assert_eq!(
source.relation_name(),
Some("\"memory\".\"main\".\"patients\"")
);
}

#[test]
fn parse_manifest_translates_a_fusion_style_source_entry() {
// dbt-fusion dialect: `#[skip_serializing_none]` OMITS unset keys
// entirely (no `identifier`, `database`, `relation_name`) — the
// cute-dbt#145 absent-key half of the engine-divergence rule.
let json = format!(
r#"{{
"metadata": {{ "dbt_schema_version": "{V12_URL}" }},
"sources": {{
"source.shop.synthea_raw.encounters": {{
"schema": "main",
"name": "encounters",
"source_name": "synthea_raw"
}}
}}
}}"#
);
let manifest = parse_manifest(&json).expect("fusion-style source entry parses");
let source = manifest
.source_by_name("synthea_raw", "encounters")
.expect("the (source_name, name) pair resolves");
assert_eq!(source.identifier(), None);
assert_eq!(source.database(), None);
assert_eq!(source.relation_name(), None);
}

#[test]
fn parse_manifest_tolerates_a_degenerate_source_entry() {
// A sources entry with every consumed key absent must not fail
// the whole manifest (ADR-5); it translates to empty-string
// names that can never match a parsed `source('a','b')` given —
// the fail-open posture of an unresolvable ref.
let json = format!(
r#"{{
"metadata": {{ "dbt_schema_version": "{V12_URL}" }},
"sources": {{ "source.shop.broken.entry": {{}} }}
}}"#
);
let manifest = parse_manifest(&json).expect("degenerate source entry tolerated");
assert_eq!(manifest.sources().len(), 1);
assert!(manifest.source_by_name("broken", "entry").is_none());
}

// ----- cute-dbt#165: column descriptions + test attribution ------
Expand Down
Loading
Loading