Skip to content

#253 — snapshots/seeds/sources as typed DAG nodes (explorer lineage no longer severs) - #282

Merged
cmbays merged 2 commits into
mainfrom
render-253-typed-dag-nodes
Jun 12, 2026
Merged

cmbays merged 2 commits into
mainfrom
render-253-typed-dag-nodes

Conversation

@cmbays

@cmbays cmbays commented Jun 12, 2026 •

Copy link
Copy Markdown
Contributor

Summary

The explorer lineage filtered every non-model node out of the DAG, severing stg → snapshot → downstream chains into disconnected components and presenting the downstream model as a false root (proven on the committed playground fixture: stg_synthea__patients → snp_patients → dim_patients split, with dim_patients rendered as a root). Seeds and sources were likewise absent, faking further roots.

build_lineage now renders the typed union: the model set plus every snapshot/seed nodes-map entry, every sources-map entry and every exposures-map entry (the issue's Discovery question — folded in as sink nodes, per the synthesis suggestion), ordered by full node id, with edges filtered to the union.

Closes #253

Acceptance criteria

  • Snapshots render as typed mid-graph nodes; chains stay connected (fixture-proven) — playground_lineage_renders_the_snapshot_mid_chain_with_no_false_roots (tests/render_integration.rs) asserts both snapshot edges by id on the real committed fixture, plus a BDD wire-round-trip scenario and a headless single-component proof.
  • Seeds and sources render as typed root nodes — playground's 16 sources + jaffle-shop's 3 seeds, fixture-proven (typed, outgoing edges only, no incoming).
  • No false roots remain on the committed fixtures' graphs — assert_lineage_complete checks node-set completeness AND full edge completeness (every manifest dependency between renderable ids is a payload edge) on both fixtures.
  • Engine caveat handled honestly — snapshots keep the not_compiled flag: fusion null-fills snapshot compiled_code at parse (dbt-schemas/src/schemas/manifest/manifest_nodes.rs:616-617 @ 9977b6cb) and backfills it on compile (dbt-tasks-sa/src/utils.rs:151-172 — why the committed playground snapshot HAS compiled code). Seeds NEVER flag: fusion null-fills seed compiled_code unconditionally (manifest_nodes.rs:232-233) — nothing to compile, so the dashed dbt-parse treatment would be noise, not honesty. The CTE arm now gates to model-typed highlights (per-model CTE DAGs are model semantics; a source highlight keeps it locked instead of opening a misleading "no CTE structure" state).
  • Universal headless guard — LINEAGE_COMPONENT_FIDELITY_JS (union-find over the payload graph vs the live Cytoscape components()) runs against the committed golden dag.html in the zero-egress suite AND a synthetic all-types chain in a new interaction test: every node id in the manifest's connected component appears in ONE rendered component.

fusion citations (clone @ 9977b6cbb1b761065536300037560d8e3c037011)

  • resource_type wire values: serde tag on DbtNode (model|test|snapshot|seed|operation|analysis|function), dbt-schemas/src/schemas/manifest/manifest.rs:52-64; snapshots/seeds live in the nodes map, sources in the top-level sources map (ManifestSource), exposures in exposures.
  • Snapshot compiled_code null-fill at parse: manifest_nodes.rs:616-617; compile-time backfill: dbt-tasks-sa/src/utils.rs:151-172.
  • Seed compiled_code null-fill (unconditional — the seed loop never backfills): manifest_nodes.rs:232-233.

What changed where

  • Rust (src/adapters/explore.rs) — LineageNodeType wire vocabulary (exhaustive wire_key match = compile-time half of the node-vocab completeness guard, the edge_type_wire_key precedent; a template-grep test is the belt-and-braces half); typed union in build_lineage; type-aware badges (models keep the explicit 0/0 posture; snapshot/seed/source badge data-tests only when non-zero — unit tests cannot target them; exposures never badge); source_detail (column descriptions off the ingested SourceNode); typed header counts. Zero changes to src/adapters/manifest.rs / src/domain/manifest.rs (coordinated with the in-flight ingestion wave; merged over adapters: ingest macro reference family — macros[].depends_on.macros #271).
  • Lineage engine (templates/explore-lineage.js) — per-type Cytoscape styles with redundant shape+color coding (model round-rectangle/blue, snapshot cut-rectangle/purple, seed barrel/teal, source ellipse/green, exposure round-tag/gray), per-type width padding, type-aware tooltip/detail-card facts (grain + files omitted for sources/exposures — not "unknown"), search across all nodes.
  • CTE engine (templates/explore-cte.js) — modelTyped gate: the CTE arm unlocks only on model-typed highlights.
  • Template (templates/explore-dag.html) — canvas-paired legend chips per present type, typed header counts, detail-card type chip.
  • Golden — examples/explore/dag.html regenerated (now 34 models + 1 snapshot + 16 sources + 1 exposure, 72 edges, all in connected lineage); the three report goldens and explore/tests.html are byte-identical (audited with git diff --text -U0 -- examples/).

Discovery answers

  • Exposures as sinks: folded in (typed exposure, sink-only edges) — visible in the committed golden via the playground's provider_quality_dashboard.
  • Report-page DAG severing: not affected — the report page's Mermaid/Cytoscape DAG is the per-model CTE graph (nodes = CTE aliases from one model's compiled SQL); upstream relations are not graph nodes there, so there is nothing to sever.

Gates

cargo fmt --check · cargo clippy --all-targets --locked -- -D warnings (by exit code) · cargo nextest run (1417 passed) · cargo test --test bdd (170 scenarios) · headless pair (headless_zero_egress 11, headless_toggle 76, both -- --ignored with real Chromium) · RUSTDOCFLAGS="-D warnings" cargo doc --no-deps --locked (public + --document-private-items) · cargo deny check — all green over the merged tree (origin/main @ edfdca4).

🤖 Generated with Claude Code


Open in Stage

Summary by CodeRabbit

  • New Features
    • Lineage visualization now includes snapshots, seeds, sources, and exposures as typed nodes alongside models.
    • Legend and search updated with type-specific node chips and styling (labels changed from "models" to "nodes").
    • CTE view restricted to model-type node selection.
    • Type-aware node details with distinct shapes and colors per node type.

github-actions Bot and others added 2 commits June 12, 2026 01:32
The explorer lineage filtered every non-model node out of the DAG,
severing stg -> snapshot -> downstream chains into disconnected
components and presenting downstream models (and every source-fed
staging model) as false roots.

build_lineage now renders the typed union: the model set plus every
snapshot/seed nodes-map entry, every sources-map entry and every
exposures-map entry (the Discovery fold-in), ordered by full node id,
with edges filtered to the union. LineageNodeType is the render-layer
wire vocabulary (model/snapshot/seed/source/exposure — fusion's DbtNode
serde tags + the sources/exposures maps @ 9977b6cb); an exhaustive
wire_key match plus a template-grep test form the node-vocab
completeness guard (the edge-vocab twin).

Engine honesty: snapshots keep the not-compiled flag (fusion null-fills
snapshot compiled_code at parse, manifest_nodes.rs:616-617, and
backfills on compile, dbt-tasks-sa/src/utils.rs:151-172); seeds NEVER
flag (null-filled unconditionally, manifest_nodes.rs:232-233 — no SQL
to compile); sources/exposures carry no code. The CTE arm gates to
model-typed highlights (per-model CTE DAGs are model semantics).

The lineage engine draws each type with redundant shape+color coding
(canvas-paired legend chips), type-aware tooltip/detail facts, and a
per-type width pad; the dag header counts each present type.

Guards: fixture-proven integration tests (playground snapshot mid-chain
+ 16 sources + exposure sink; jaffle-shop seeds as typed roots; full
node/edge completeness = no false roots), two BDD wire-round-trip
scenarios, and the universal headless component-fidelity probe (every
payload component renders as ONE Cytoscape component) on both the
committed golden and a synthetic all-types chain.

Golden regenerated: examples/explore/dag.html (the only example that
changes; report goldens byte-identical).

Closes #253

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…se, auto-deref)

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@coderabbitai

coderabbitai Bot commented Jun 12, 2026 •

Copy link
Copy Markdown

Review Change Stack

Caution

Review failed

Pull request was closed or merged during review

📝 Walkthrough

Walkthrough

The PR upgrades the cute-dbt explore DAG lineage from model-only to a typed full-manifest representation supporting snapshots, seeds, sources, and exposures. It introduces a LineageNodeType enum, reworks lineage graph construction to yield a deterministic typed-node union with forward edges, implements type-aware payload assembly and detail rendering, updates frontend templates with per-type styling, gates CTE view to models only, and validates behavior through feature, integration, and headless browser tests.

Changes

Typed Lineage DAG

Layer / File(s) Summary
Type system and lineage contracts
src/adapters/explore.rs
Adds public LineageNodeType enum (Model, Snapshot, Seed, Source, Exposure) with wire_key() method. Updates LineageNode and LineageNodePayload to carry explicit node_type field for typed node semantics.
Typed lineage graph construction
src/adapters/explore.rs
Replaces model-only build_lineage with a typed union of models, snapshots, seeds, sources, and exposures. Computes forward edges deterministically from each node type's dependencies, filtered to stay within the typed-node set, eliminating false roots when snapshots/seeds sit mid-chain.
Type-aware payload and detail assembly
src/adapters/explore.rs
Reworks build_lineage_payload to switch on node_type when building per-node details and paths: model/snapshot/seed use code detail; sources use column-only detail; exposures use empty detail. Adds typed_badge helper to conditionally render badges per type (models explicit; snapshots/seeds/sources only when non-zero; exposures unbadged).
Frontend template styling and Cytoscape setup
templates/explore-dag.html, templates/explore-lineage.js
Adds CSS for per-type node chips (type-model, type-snapshot, type-seed, type-source, type-exposure). Updates legend to conditionally render typed chips. Updates Cytoscape stylesheet with node[type="..."] selectors for distinct shapes and colors. Changes search placeholder from "models" to "nodes".
Type-aware detail card and tooltip rendering
templates/explore-lineage.js
Makes detail card type-aware: shows type chip for non-models, omits tests badge for non-models, gates grain/files sections to code-bearing types only. Hover tooltip conditionally includes type fact and gates badge/grain to appropriate types.
CTE view gating to model-type nodes
templates/explore-cte.js
Adds modelTyped() helper to classify highlighted nodes as eligible for CTE view (model type or legacy missing type). Gates setView("cte") call and CTE button disabled state to require both highlight and modelTyped classification. Early-returns lineage view when highlight is missing or non-model.
Unit test expansion for typed lineage
src/adapters/explore.rs
Adds helpers to construct typed nodes (snapshot/seed/source/exposure). Validates deterministic node ordering, correct edge formation, per-type not_compiled behavior, per-type badge logic, JSON wire-key serialization, and template/engine completeness across all LineageNodeType variants.
Test helpers and cucumber steps
tests/steps/builders.rs, tests/steps/explore_full_manifest.rs, tests/steps/explore_lineage_dag.rs, tests/steps/world.rs
Generalizes serialize_explore_to_tmp to accept raw_nodes and top_map_entries for injecting sources/exposures into test manifests. Extends ExplorePlan and ExploreModelDecl with typed node collections and raw dependency wiring. Adds cucumber #[given] steps to declare snapshots/seeds/sources/exposures and #[then] step to assert node_type.
Feature scenarios for typed lineage DAG
features/explore_lineage_dag.feature
Adds Gherkin scenarios validating snapshot mid-chain representation (model→snapshot→model edges) and seeds/sources/exposures as typed lineage nodes with correct forward edges and no false roots.
Headless browser fidelity and test rendering
tests/headless_zero_egress.rs
Adds LINEAGE_COMPONENT_FIDELITY_JS probe that validates connected-component parity between payload graph and rendered Cytoscape via union-find. Extends zero-egress test to run fidelity check on dag.html. Introduces render_explore_dag_manifest helper and end-to-end typed-node browser test asserting per-node type, component fidelity, component count, seed compilation state, and CTE view gating.
Integration tests validating typed lineage behavior
tests/render_integration.rs
Adds renderable_ids() helper to compute expected typed-node set from manifest. Adds assert_lineage_complete() to validate node-set and edge-set completeness across all renderable types. Tests playground fixture's mid-chain snapshot (no false roots, edge survival, exposure sink behavior) and jaffle-shop fixture's three seeds as typed roots with no incoming edges and correct compilation flags.

Estimated code review effort

🎯 4 (Complex) | ⏱️ ~45 minutes

Possibly related PRs

  • breezy-bays-labs/cute-dbt#218: Both PRs extend the explore-lineage.js and src/adapters/explore.rs Cytoscape lineage pipeline; PR #218 added interactive highlight/focus behavior while this PR adds typed node vocabulary and type-aware rendering to the same renderer.
  • breezy-bays-labs/cute-dbt#225: Both PRs modify the lineage payload data contract in src/adapters/explore.rs; PR #225 adds external-drive contract and per-node paths fields while this PR adds typed node_type field and type-aware logic to the same structures.

🐰 From snapshots' mid-chain hiding spree,
Now typed nodes flow free,
Sources and seeds emerge bright,
DAG glows complete—no false light! ✨

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title concisely describes the main change: typed DAG nodes for snapshots/seeds/sources with fixed lineage connectivity.
Linked Issues check ✅ Passed All acceptance criteria from issue #253 are met: snapshots render as typed mid-graph nodes, seeds/sources/exposures are typed root nodes, no false roots, snapshot null-filling handled, and headless fidelity guard implemented.
Out of Scope Changes check ✅ Passed All changes are directly scoped to issue #253: typed lineage node implementation, UI/template updates for node types, and comprehensive test coverage for the feature.
Docstring Coverage ✅ Passed Docstring coverage is 100.00% which is sufficient. The required threshold is 80.00%.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch render-253-typed-dag-nodes

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

@ghost

ghost commented Jun 12, 2026

Copy link
Copy Markdown

@github-actions

Copy link
Copy Markdown
Contributor

📄 Rendered report preview

All golden examples regenerated cleanly.

🟡 Golden examples

Committed to examples/ and byte-identity gated — the canonical reports contributors and consumers browse. Stable across PRs.

Report View Download
playground-report.html ▶ Open ↗ ⬇ Download
jaffle-shop-report.html ▶ Open ↗ ⬇ Download
diff-showcase-report.html ▶ Open ↗ ⬇ Download

🐶 Live dogfood preview

This PR doesn't touch dbt-project/, so there's no live dogfood preview.

▶ Open ↗ opens the report in your browser in one click —
published to this repo's GitHub Pages under /pr-282/.
⬇ Download fetches the same self-contained HTML as a workflow
artifact (auth-gated; works fully offline). Either way the report
makes zero external resource requests.

The Pages preview may take ~1 min to update after this comment
posts. On PRs from forks the Open link is unavailable (read-only
token) — use Download.

Alternative: GitHub CLI
# gh CLI >= 2.63 extracts into ./report-preview-playground/.
gh run download 27396916325 -R breezy-bays-labs/cute-dbt -n report-preview-playground
open report-preview-playground/playground-report.html

Posted by report-preview.yml for 17a8e0f151485fc3632a0f3b6ead585959bd5f13. Affordance only — never blocks merge.

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request implements typed lineage nodes (snapshots, seeds, sources, and exposures) in the explore lineage DAG, resolving issue #253 where filtering out non-model nodes severed lineage chains and created false roots. The changes introduce a LineageNodeType enum, update the lineage graph construction to union models with other node types, and adapt the frontend templates (HTML, CSS, JS) to style and legend each node type appropriately. Additionally, extensive unit, integration, and headless browser tests have been added to verify lineage completeness and fidelity. There are no review comments provided, so I have no feedback to provide on the review itself.

Important

The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.

@cmbays
cmbays merged commit 97e8d38 into main Jun 12, 2026
32 of 33 checks passed
@cmbays
cmbays deleted the render-253-typed-dag-nodes branch June 12, 2026 05:43
@cmbays

cmbays commented Jun 12, 2026

Copy link
Copy Markdown
Contributor Author

Overnight orchestration wrap-up

Merged as 97e8d38 (squash) under Christopher's overnight authority — CI fully green, 0 unresolved review threads, push verified (origin == head 17a8e0f). Issue #253 auto-closed ✔.

What shipped: snapshots/seeds/sources/exposures as first-class typed nodes in the explorer lineage (Cytoscape+dagre) with redundant shape+color coding, type-aware tooltips/detail, and a node-vocab completeness guard mirroring the EdgeType CI guard. Two load-bearing fusion-grounded findings in the PR body: the report page's DAG is the per-model CTE graph (so typed-node treatment correctly lands only on the explorer; both report engines untouched), and snapshot compiled_code is backfilled on compile while seed compiled_code is null-filled unconditionally (seeds never trip the dbt-parse dashed treatment). Exposures folded in per the issue's Discovery (playground dashboard visible as a sink in the committed golden).

🤖 Generated with Claude Code

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

adapters: explorer lineage severs at snapshots — seeds/sources/snapshots missing as typed DAG nodes (false roots)

1 participant