Repository navigation
#253 — snapshots/seeds/sources as typed DAG nodes (explorer lineage no longer severs) - #282
Conversation
The explorer lineage filtered every non-model node out of the DAG, severing stg -> snapshot -> downstream chains into disconnected components and presenting downstream models (and every source-fed staging model) as false roots. build_lineage now renders the typed union: the model set plus every snapshot/seed nodes-map entry, every sources-map entry and every exposures-map entry (the Discovery fold-in), ordered by full node id, with edges filtered to the union. LineageNodeType is the render-layer wire vocabulary (model/snapshot/seed/source/exposure — fusion's DbtNode serde tags + the sources/exposures maps @ 9977b6cb); an exhaustive wire_key match plus a template-grep test form the node-vocab completeness guard (the edge-vocab twin). Engine honesty: snapshots keep the not-compiled flag (fusion null-fills snapshot compiled_code at parse, manifest_nodes.rs:616-617, and backfills on compile, dbt-tasks-sa/src/utils.rs:151-172); seeds NEVER flag (null-filled unconditionally, manifest_nodes.rs:232-233 — no SQL to compile); sources/exposures carry no code. The CTE arm gates to model-typed highlights (per-model CTE DAGs are model semantics). The lineage engine draws each type with redundant shape+color coding (canvas-paired legend chips), type-aware tooltip/detail facts, and a per-type width pad; the dag header counts each present type. Guards: fixture-proven integration tests (playground snapshot mid-chain + 16 sources + exposure sink; jaffle-shop seeds as typed roots; full node/edge completeness = no false roots), two BDD wire-round-trip scenarios, and the universal headless component-fidelity probe (every payload component renders as ONE Cytoscape component) on both the committed golden and a synthetic all-types chain. Golden regenerated: examples/explore/dag.html (the only example that changes; report goldens byte-identical). Closes #253 Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…se, auto-deref) Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
|
Caution Review failedPull request was closed or merged during review 📝 WalkthroughWalkthroughThe PR upgrades the ChangesTyped Lineage DAG
Estimated code review effort🎯 4 (Complex) | ⏱️ ~45 minutes Possibly related PRs
🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches📝 Generate docstrings
🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
|
Ready to review this PR? Stage has broken it down into 11 individual chapters for you: Chapters generated by Stage for commit 17a8e0f on Jun 12, 2026 5:38am UTC. |
📄 Rendered report previewAll golden examples regenerated cleanly. 🟡 Golden examplesCommitted to
🐶 Live dogfood previewThis PR doesn't touch ▶ Open ↗ opens the report in your browser in one click — The Pages preview may take ~1 min to update after this comment Alternative: GitHub CLI# gh CLI >= 2.63 extracts into ./report-preview-playground/.
gh run download 27396916325 -R breezy-bays-labs/cute-dbt -n report-preview-playground
open report-preview-playground/playground-report.htmlPosted by |
There was a problem hiding this comment.
Code Review
This pull request implements typed lineage nodes (snapshots, seeds, sources, and exposures) in the explore lineage DAG, resolving issue #253 where filtering out non-model nodes severed lineage chains and created false roots. The changes introduce a LineageNodeType enum, update the lineage graph construction to union models with other node types, and adapt the frontend templates (HTML, CSS, JS) to style and legend each node type appropriately. Additionally, extensive unit, integration, and headless browser tests have been added to verify lineage completeness and fidelity. There are no review comments provided, so I have no feedback to provide on the review itself.
Important
The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.
Overnight orchestration wrap-upMerged as What shipped: snapshots/seeds/sources/exposures as first-class typed nodes in the explorer lineage (Cytoscape+dagre) with redundant shape+color coding, type-aware tooltips/detail, and a node-vocab completeness guard mirroring the EdgeType CI guard. Two load-bearing fusion-grounded findings in the PR body: the report page's DAG is the per-model CTE graph (so typed-node treatment correctly lands only on the explorer; both report engines untouched), and snapshot 🤖 Generated with Claude Code |
Summary
The explorer lineage filtered every non-model node out of the DAG, severing
stg → snapshot → downstreamchains into disconnected components and presenting the downstream model as a false root (proven on the committed playground fixture:stg_synthea__patients → snp_patients → dim_patientssplit, withdim_patientsrendered as a root). Seeds and sources were likewise absent, faking further roots.build_lineagenow renders the typed union: the model set plus everysnapshot/seednodes-map entry, everysources-map entry and everyexposures-map entry (the issue's Discovery question — folded in as sink nodes, per the synthesis suggestion), ordered by full node id, with edges filtered to the union.Closes #253
Acceptance criteria
playground_lineage_renders_the_snapshot_mid_chain_with_no_false_roots(tests/render_integration.rs) asserts both snapshot edges by id on the real committed fixture, plus a BDD wire-round-trip scenario and a headless single-component proof.assert_lineage_completechecks node-set completeness AND full edge completeness (every manifest dependency between renderable ids is a payload edge) on both fixtures.not_compiledflag: fusion null-fills snapshotcompiled_codeat parse (dbt-schemas/src/schemas/manifest/manifest_nodes.rs:616-617@9977b6cb) and backfills it on compile (dbt-tasks-sa/src/utils.rs:151-172— why the committed playground snapshot HAS compiled code). Seeds NEVER flag: fusion null-fills seedcompiled_codeunconditionally (manifest_nodes.rs:232-233) — nothing to compile, so the dashed dbt-parse treatment would be noise, not honesty. The CTE arm now gates to model-typed highlights (per-model CTE DAGs are model semantics; a source highlight keeps it locked instead of opening a misleading "no CTE structure" state).LINEAGE_COMPONENT_FIDELITY_JS(union-find over the payload graph vs the live Cytoscapecomponents()) runs against the committed golden dag.html in the zero-egress suite AND a synthetic all-types chain in a new interaction test: every node id in the manifest's connected component appears in ONE rendered component.fusion citations (clone @ 9977b6cbb1b761065536300037560d8e3c037011)
resource_typewire values: serde tag onDbtNode(model|test|snapshot|seed|operation|analysis|function),dbt-schemas/src/schemas/manifest/manifest.rs:52-64; snapshots/seeds live in thenodesmap, sources in the top-levelsourcesmap (ManifestSource), exposures inexposures.compiled_codenull-fill at parse:manifest_nodes.rs:616-617; compile-time backfill:dbt-tasks-sa/src/utils.rs:151-172.compiled_codenull-fill (unconditional — the seed loop never backfills):manifest_nodes.rs:232-233.What changed where
src/adapters/explore.rs) —LineageNodeTypewire vocabulary (exhaustivewire_keymatch = compile-time half of the node-vocab completeness guard, theedge_type_wire_keyprecedent; a template-grep test is the belt-and-braces half); typed union inbuild_lineage; type-aware badges (models keep the explicit 0/0 posture; snapshot/seed/source badge data-tests only when non-zero — unit tests cannot target them; exposures never badge);source_detail(column descriptions off the ingestedSourceNode); typed header counts. Zero changes tosrc/adapters/manifest.rs/src/domain/manifest.rs(coordinated with the in-flight ingestion wave; merged over adapters: ingest macro reference family — macros[].depends_on.macros #271).templates/explore-lineage.js) — per-type Cytoscape styles with redundant shape+color coding (model round-rectangle/blue, snapshot cut-rectangle/purple, seed barrel/teal, source ellipse/green, exposure round-tag/gray), per-type width padding, type-aware tooltip/detail-card facts (grain + files omitted for sources/exposures — not "unknown"), search across all nodes.templates/explore-cte.js) —modelTypedgate: the CTE arm unlocks only on model-typed highlights.templates/explore-dag.html) — canvas-paired legend chips per present type, typed header counts, detail-card type chip.examples/explore/dag.htmlregenerated (now 34 models + 1 snapshot + 16 sources + 1 exposure, 72 edges, all in connected lineage); the three report goldens and explore/tests.html are byte-identical (audited withgit diff --text -U0 -- examples/).Discovery answers
exposure, sink-only edges) — visible in the committed golden via the playground'sprovider_quality_dashboard.Gates
cargo fmt --check·cargo clippy --all-targets --locked -- -D warnings(by exit code) ·cargo nextest run(1417 passed) ·cargo test --test bdd(170 scenarios) · headless pair (headless_zero_egress11,headless_toggle76, both-- --ignoredwith real Chromium) ·RUSTDOCFLAGS="-D warnings" cargo doc --no-deps --locked(public +--document-private-items) ·cargo deny check— all green over the merged tree (origin/main @ edfdca4).🤖 Generated with Claude Code
Summary by CodeRabbit