Repository navigation
docs: Scheduled deletions section + DB-17 reference-resolution provenance - #507
Conversation
…ance
Two artifacts, landed together because they reference each other:
1. **Scheduled deletions** section in src/v3/ROADMAP.md (sibling to
"Active deferrals"). Discipline: every scaffold in live substrate
lives here with trigger, blocker, and enforcement path. When the
trigger fires, a PR deletes the scaffold AND removes the row.
Three enforcement-path kinds, three rules:
- Structural lens (preferred) — substrate already carries the
fact; lens walks user DAGs.
- Needs substrate amendment (DB-NN) — lens becomes writable
after the DB lands. Grep is NOT a substitute.
- Compiler-source ratchet (temporary) — scoped to
src/v3/compiler/; dissolves at self-hosting.
Initial table covers 9 scaffolds: ArrowBody::Pending/Unparsed,
ValueBody::Unparsed, TransformTarget::Operator (all lens-able now),
declaration_by_name + NameKeyedReference (need DB-17), Node.name
+ encoding_meet/join (compiler-source ratchet), keyword_to_name
(TBD recon).
Explicit §"Grep is not an enforcement path" explains WHY: grep is
heuristic over source; the compiler-as-dependency-analyzer
framing says every structural fact gets structural enforcement.
The banked-dissolutions ratchet (post-l15-phase-plan) is a
narrow exception because it operates on documentation text, not
on system behavior.
2. **DB-17 reference-resolution provenance** (docs/design-reference-
resolution-provenance.md). Splits AtomPayload::ResolvedIdentifier
into ResolvedByStructure and ResolvedByName. Lowering call sites
pick which variant based on resolution path (structural walk vs
declaration_by_name fallback). Consumers update to handle both
variants via pattern match.
This makes "reference was resolved by name fallback" a substrate-
visible fact that lens_structural_resolution.dag can query.
Without it, declaration_by_name's scheduled deletion has no
structural enforcement path.
Dissolution trigger: M2 module scoping lands; ResolvedByName
becomes unreachable; a final PR removes it from the substrate.
Size: M for substrate + lowering + consumer migration.
Both artifacts express the same underlying principle from tonight's
thesis-level reframe: the compiler IS the dependency analyzer, and
every named scaffold gets a structural enforcement path (or the
substrate amendment that unlocks one is filed). Grep over source is
the anti-pattern.
ChatGPT ReviewPrinciple audit. 1. Fail-closed. This is mostly in good shape. The new ROADMAP section explicitly rejects grep-as-enforcement and routes scaffolds toward lenses, DBs, or narrowly-bounded temporary ratchets, which is aligned with the thesis’s “lenses, not grep” direction. My one concern is that DB-17’s enforcement story is not fully closed yet: the doc says the future lens must distinguish bootstrap/std from user-range resolutions, but the substrate-visible boundary fact is still an open question in 2. Illegal states unrepresentable. Strong. The split from 3. Facts flow forward. Also strong. DB-17 is directly about carrying a fact that is currently dropped after lowering — resolution provenance — so a downstream lens can consume it structurally instead of reconstructing it from source heuristics ( 4. Coproduct dissolution. Mostly satisfied. The new substrate growth is treated honestly as scaffolded, not terminal: 5. Single-authority metadata. Good. The new ROADMAP section creates one canonical place for scaffold-deletion receipts, and DB-17 correctly rejects a provenance side table in favor of carrying the fact on the Atom itself ( 6. API-level enforcement. Mostly good because the variant split forces consumers to see the distinction. The only thing I would watch is the suggested Design question. Is DB-17 itself the full enforcement unlock, or is the real prerequisite “DB-17 plus a substrate-visible declaration-source/range fact”? Right now Verdict. APPROVE_WITH_COMMENTS. This is a useful docs round: the scheduled-deletions section makes scaffold debt auditable, and DB-17 is the right kind of upstream amendment rather than another grep rule. My comments are about tightening the receipts so the eventual implementation cannot quietly fall back to a Rust-side boundary heuristic, and about not leaving a LOOP HEALTH: converging — this round turns two recurring review themes (grep-as-enforcement and invisible name-fallback bridges) into explicit structural work items; |
Recon finding: bare `keyword_to_name` was renamed to `tok_keyword_to_name` during v2 Phase 0 parser restructure (see src/v2/parser-design.md:403-408). The new name still carries the scaffold (parser-side keyword-name logic duplicating SyntaxSpec), but lives in v2 code (src/v2/02_parse.dag, v2_compiler_parse.rs). Grep confirms zero equivalents in src/v3/. V2 is reference- implementation / test oracle per ROADMAP §"Sketch vs Oracle framing"; v2 scaffolds dissolve when v3 supersedes v2 entirely, not individually. Removing the TBD row from the v3 Scheduled Deletions table. Recording the recon outcome in §"Notes on specific rows" so the classification decision is visible to future readers.
ChatGPT ReviewPrinciple audit. 1. Fail-closed. Satisfied. This round keeps pushing enforcement away from grep/audit cadence and toward substrate-visible facts plus lenses. The new scheduled-deletions section explicitly constrains enforcement to structural lenses, DB-backed substrate amendments, or a temporary compiler-source ratchet, and DB-17 gives 2. Illegal states unrepresentable. Satisfied. DB-17’s split from 3. Facts flow forward. Satisfied. The whole point of DB-17 is to stop dropping resolution provenance after lowering and carry it into reflection/lens consumers as a first-class fact ( 4. Coproduct dissolution. Satisfied. 5. Single-authority metadata. One NON-BLOCKING concern. The new 6. API-level enforcement. Satisfied. The docs are moving the right way: DB-17 makes name fallback a substrate fact, the future lens consumes that fact, and the roadmap treats grep as a signal that a missing DB should be filed rather than as a gate ( Design question. What is the exact authority boundary of the Scheduled Deletions ledger: only v3 substrate scaffolds, all v3-scoped scaffolds including temporary Rust ratchets, or something wider? The added Verdict. APPROVE_WITH_COMMENTS. This is a better round than the previous one: DB-17 is a real structural unlock, and the LOOP HEALTH: converging — this round dissolves the earlier |
briansrls
left a comment
There was a problem hiding this comment.
codex · gpt-5.4 · e0d566ea
BLOCKING (1)
Root Cause
docs/design-reference-resolution-provenance.mdThe docs key the scheduled deletion to the helper name instead of the actual bridge classes; split the tracker into ResolvedByName reference fallbacks versus the remaining compiler-internal declaration_by_name lookups, each with its own dissolution path.
ROADMAP — Verified
- keyword_to_name v2 scope: The cited src/v2/parser-design.md, src/v2/02_parse.dag, and src/v2/stage0/src/v2_compiler_parse.rs do show keyword_to_name surviving only as tok_keyword_to_name in v2, and I found no v3 equivalent.
- DB-17 schema grounding: src/v3/compiler/src/dag.rs and src/v3/std/substrate.dag both currently expose AtomPayload::ResolvedIdentifier, so the proposed split is grounded in the live substrate rather than stale thesis text.
|
|
||
| ### Notes on specific rows | ||
|
|
||
| - **`declaration_by_name` and NameKeyedReference bridges** are the same underlying class: "reference resolved by name fallback rather than by structural edge." [DB-17 (reference-resolution provenance)](../../docs/design-reference-resolution-provenance.md) makes this a single structural fact in the substrate; landing DB-17 unlocks joint dissolution via one lens. |
There was a problem hiding this comment.
BLOCKING: This collapses two different debts into one: DB-17 would surface name-fallback AtomPayload edges, but it would not cover the other live declaration_by_name call sites (bootstrap caches and downstream name recovery), so "joint dissolution via one lens" misstates the single-authority/semantic-authority fix.
|
BLOCKING (1) Root Cause
ROADMAP — Verified
|
Meta-Review (Loop Health)According to a document from 2026-04-17, the loop is no longer earning another ordinary review round. It made one real move, then started polishing the accounting of that move instead of cashing it out structurally. Loop summary. Visible history shows 3 review rounds, 2 visible commit states, 1 codex review, and 2 browser reviews, over roughly 45 minutes from 15:31Z to 16:16Z on 2026-04-17. The second browser review explicitly references a follow-up commit ( Forward progress evidence. There was forward motion. The loop converted two previously hand-wavy review themes into explicit design artifacts: a scheduled-deletions ledger and DB-17’s proposed split of There is also one concrete response to review feedback: the earlier So this is not a dead loop. It did surface the class and start naming it correctly. Debt accumulation evidence. The problem is that the loop’s output is still mostly accounting, not authority. Principle 1 says consumers define correctness; without a consumer, 100% conformant prose is still substrate growth with no proof. This PR is docs-only: the visible progress is a proposed DB and a ledger, not a new test, lens, emitter path, interpreter path, or enforced invariant. The project’s own roadmap warns that it already has large areas of modeled structure with zero consumers, and that such modeling should be made load-bearing before adding more. The recurring finding also did not fully graduate. Across the three visible rounds, reviewers keep circling the same class: name-fallback bridges, grep-as-enforcement, and unclear scaffold accounting. First it was “DB-17 may still need a sibling boundary fact”; then “ledger scope is inconsistent”; then codex escalated that the tracker is still keyed to the helper name instead of the actual bridge classes. That is the same disease moving one level sideways each round, not a new class being eliminated. That matters because the project’s own rules are explicit: recurring adapter/bridge patterns are supposed to be stopped loudly, not normalized into tracked transitional prose. The “no short-term solutions” and “no bridges” invariants say a tracked bridge is still the wrong shape if it teaches downstream consumers the wrong authority boundary. The codex blocker is the clearest debt signal: the ledger is still tracking the wrong unit of debt. If the deletion row is keyed to Cheating signal. The implementer is not hiding compromises. That part is healthy. The scaffolds are named, the debt is being written down in a scheduled-deletions ledger, and the follow-up commit did respond directly to a review theme rather than pretending the first draft was already sufficient. But the fixes are still drifting toward “good enough with accounting” rather than structural completion. This is not hidden cheating; it is documented triage. The issue is that the loop is now spending its marginal effort on making the accounting cleaner instead of either landing the structural consumer or promoting the recurring class into a stronger invariant/test. The browser reviews call the remaining issues “scope clarity” and “boundary fact,” while codex says the row still overclaims what DB-17 dissolves. That is a smell that the loop has moved from solving the problem to refining the description of the problem. In this codebase’s own terms: the compromises are explicit, but they are still compromises. The loop is approaching the point where another pass buys cleaner prose, not a better substrate. Meta-verdict 🔁 PAUSE_AND_REGROUP The loop has made some forward progress, but it is now repeating at the wrong layer. Another ordinary review round will probably produce one more refinement of the ledger wording or the DB-17 scope note. That is not the highest-value next move. What should happen next is one of these, not another prose pass:
Why this is not Why this is not Why this is not So the blunt read is: the review loop is starting to shift debt unless it stops being a prose loop and becomes a structural decision. |
Hand-written shell scripts (.githooks/pre-push from PRs #503/#509, scripts/install-hooks.sh, scripts/check-stage0-freshness.sh, scripts/regenerate-stage0.sh) bypass the compiler's dependency- analysis machinery. Per the compiler-as-dependency-analyzer thesis (tonight's framing), they should be .dag programs composing existing service operations from extdeps/ and emitted as standalone shell scripts at build time. Recon done on existing substrate: - dsl/extdeps/git.dag: service git.Core with 7 operations + shell transport + mock_response pattern. 181 lines. - dsl/extdeps/cargo.dag: Build, Test, Clippy, Doc, Run. MISSING Fmt. - dsl/extdeps/shell.dag: POSIX Find, Env. Adequate. - dsl/extdeps/github/: auth, pulls, gists, actions (not on critical path for pre-push hook). Separable prerequisite deferrals named: - cargo.dag Fmt operations (S) — mechanical extension - Shell-emission target (M, needs design) — v3 emits Rust/Go/Python today; shell is used as transport but not as emission target. Needs a DB clarifying what "emit to shell" means structurally (likely E-9-pattern with output as standalone executable text). - "Hook-as-program" pattern (M, needs design) — how a .dag program declares its invocation contract (stdin format, env, exit codes). - Test coverage via DB-15 R2's MockBackedInvariant predicate. Yellow-flag threshold: triggers actively when a *second* hand- written workflow script needs the same modeling. Until then, the hand-written pre-push hook (PRs #503 + #509) is tolerated as the one instance. Added under Active Deferrals §"Cross-cutting — workflow scripts modeled in .dag". When PR #507 (Scheduled Deletions) merges, the hand-written .githooks/pre-push also gets a row there with trigger "emitted pre-push hook replaces it."
Codex review (BLOCKING) on PR #507 correctly caught that keying the scheduled-deletion row to the helper name 'declaration_by_name' conflated three distinct debt classes. 'declaration_by_name' has ~83 call sites across the compiler: 1. Bootstrap substrate_markers caches (dag.rs:1616+): looking up 'Int'/'Bool'/'String'/'Value'/'Transform'/... by name to populate marker fields. Compiler-internal cache; dissolves via substrate amendment (typed edges) OR self-hosting. 2. Pipeline authority wiring (bootstrap.rs, pipeline_authority.rs): walks Dag by name for 'PIPELINE_REALIZATION_META', 'parse', 'PipelineStageBinding', etc. Bootstrap-time machinery; dissolves at self-hosting. 3. Emitter algebra lookups (emit_go.rs, emit_python.rs): 'OrderedRing' etc. Emission-layer dispatch; dissolves via Lane 1e + self-hosting. 4. User-range fallback in lowering: structural walk can't resolve a user's identifier, falls back to name lookup. Produces an AtomPayload::ResolvedIdentifier that looks identical to a structurally-resolved one. DB-17's scope. Fix in ROADMAP scheduled-deletions table: - Old: one row conflating all four ('declaration_by_name' + 'NameKeyedReference bridges') - New: two rows — user-range 'ResolvedByName' AtomPayload (DB-17) vs compiler-internal call sites (compiler-source ratchet, separate dissolution paths). Fix in DB-17 doc: - Title/header narrowed: 'user-range AtomPayload scope' - New §'What DB-17 does NOT cover' enumerates the three compiler-internal classes with their separate dissolution paths. - Implementation scope clarifies: compiler-internal call sites of declaration_by_name do NOT produce AtomPayload, so they're out of scope for this change. - Dissolution section: notes that DB-17 dissolves only the user-range class; compiler-internal sites dissolve separately. Meta-review said 'loop has stopped earning rounds' — so this commit addresses the one remaining BLOCKING cleanly and is the final iteration before merge.
Addressed codex BLOCKING (commit fab812e) — split the conflated classCodex was right: keying the scheduled deletion to the helper name `declaration_by_name` conflated three distinct debt classes. Investigated: the Rust helper at `dag.rs:1459` has ~83 call sites that split cleanly into:
Fix in the scheduled-deletions table
Fix in the DB-17 doc
On the meta-reviewMeta-review signaled "this loop is no longer earning rounds." Taking that seriously — this commit addresses the one remaining structural issue and is the final iteration. The split is the fact the reviewer was pushing for; the rest of the scheduled-deletions discipline stays as-landed. Ready for merge. |
|
ChatGPT review in progress... (view conversation) Check back in ~30 minutes for the full review. |
… as first use case) (#510) * docs: ROADMAP — track the commit-pipeline-in-dag modeling work Hand-written shell scripts (.githooks/pre-push from PRs #503/#509, scripts/install-hooks.sh, scripts/check-stage0-freshness.sh, scripts/regenerate-stage0.sh) bypass the compiler's dependency- analysis machinery. Per the compiler-as-dependency-analyzer thesis (tonight's framing), they should be .dag programs composing existing service operations from extdeps/ and emitted as standalone shell scripts at build time. Recon done on existing substrate: - dsl/extdeps/git.dag: service git.Core with 7 operations + shell transport + mock_response pattern. 181 lines. - dsl/extdeps/cargo.dag: Build, Test, Clippy, Doc, Run. MISSING Fmt. - dsl/extdeps/shell.dag: POSIX Find, Env. Adequate. - dsl/extdeps/github/: auth, pulls, gists, actions (not on critical path for pre-push hook). Separable prerequisite deferrals named: - cargo.dag Fmt operations (S) — mechanical extension - Shell-emission target (M, needs design) — v3 emits Rust/Go/Python today; shell is used as transport but not as emission target. Needs a DB clarifying what "emit to shell" means structurally (likely E-9-pattern with output as standalone executable text). - "Hook-as-program" pattern (M, needs design) — how a .dag program declares its invocation contract (stdin format, env, exit codes). - Test coverage via DB-15 R2's MockBackedInvariant predicate. Yellow-flag threshold: triggers actively when a *second* hand- written workflow script needs the same modeling. Until then, the hand-written pre-push hook (PRs #503 + #509) is tolerated as the one instance. Added under Active Deferrals §"Cross-cutting — workflow scripts modeled in .dag". When PR #507 (Scheduled Deletions) merges, the hand-written .githooks/pre-push also gets a row there with trigger "emitted pre-push hook replaces it." * docs: commit-pipeline deferral — address codex BLOCKING + chatgpt non-blockings Three reviewer concerns addressed: 1. BLOCKING (codex): yellow-flag threshold understated. Four hand- written scripts already exist, so the deferral is active now — not 'tolerated until a second instance.' Reframed: marked ACTIVE; THESIS.md's meta-process claim is the justifying authority, not an arbitrary 'second script' trigger. 2. NON-BLOCKING (codex): 'First concrete use case' over-described the pre-push hook. At PR #509 HEAD the hook fmt-checks / fmt-fixes / commits; the stdin/delete/HEAD contract is implementation detail that belongs in the .dag design work, not the ROADMAP entry. Trimmed. 3. NON-BLOCKING (chatgpt facts-flow-forward): Track 15 (CLI tool modeling — bare command names are hidden PATH dependencies) wasn't threaded into the prerequisite list. Added as a prerequisite deferral referencing ROADMAP.md:810-826 directly. 4. SINGLE-AUTHORITY pressure (chatgpt): Shape A vs Shape B was an implicit lean toward Shape A ('shell-emission target'). Per Track 16 (ROADMAP.md:920-935), the thesis puts shell scripts as Shape B — .dag programs build them via concat/fold/match, parallel to Track 16's YAML emission. Updated entry to make Shape B explicit, remove the 'shell-emission target' framing, and align with tools/ratchet.dag's grep-command generation as precedent. Prerequisite list rewritten: - cargo.dag Fmt operations (S, mechanical) - Track 15 tool resolution (already-tracked prerequisite) - Shape B emission via existing Track 16 pattern (no new compiler concept) - Hook invocation contract as structural declaration (small type in a .dag program) - Test coverage via DB-15 R2 MockBackedInvariant Scope for the other three scripts (install-hooks.sh, check-stage0-freshness.sh, regenerate-stage0.sh) noted: same Shape B pattern, dissolves individually once pre-push proves it.
Summary
Two artifacts, landed together because they reference each other:
The framing these consume
Both artifacts express the thesis-level reframe from earlier tonight: the compiler IS the dependency analyzer. Every named scaffold gets a structural enforcement path, or the substrate amendment that unlocks one is filed as a DB. Grep over source is the anti-pattern — it's a heuristic that can't distinguish legitimate bootstrap from user-range drift.
Scheduled deletions table (initial population)
DB-17 shape
```rust
// Before
pub enum AtomPayload {
ResolvedIdentifier(DeclarationId),
// ...
}
// After
pub enum AtomPayload {
ResolvedByStructure(DeclarationId),
ResolvedByName(DeclarationId),
// ...
}
```
Lowering call sites pick the variant based on resolution path. Consumers update to handle both variants. Lens queries for `ResolvedByName` reachable from user-range roots.
Dissolution of `ResolvedByName` itself is scheduled for when M2 module scoping lands; at that point every reference is structural and the variant becomes unreachable.
Relationship to other PRs
Test plan
🤖 Generated with Claude Code