Skip to content

Add Markdown as an emit medium: row-driven serialize_markdown_source (medium rows) - #5501

Merged
briansrls merged 3 commits into
mainfrom
session/deep-newt-165
Jun 21, 2026
Merged

briansrls merged 3 commits into
mainfrom
session/deep-newt-165

Conversation

@briansrls

@briansrls briansrls commented Jun 21, 2026 •

Copy link
Copy Markdown
Contributor

Markdown as an emit medium — the row-driven serialize_markdown_source

Adds Markdown as an emit target: the medium rows for headings, lists, GFM task-checkboxes, links, tables, code (inline + fenced), emphasis/strong, blockquotes — authored as a spelling-row table the fold looks up, never inline delimiter literals (the medium-as-Node guard, #5496).

Substrate decision (Path B, ruled by emission manager quick-seal-137)

A byte-faithful MarkdownDocument → markdown-SOURCE serializer on the v1 seed — the peer of extdeps/formats/yaml.dag serialize_yaml, and the byte-for-byte clone of the ci.yml pattern (serialize_yaml ∘ project_workflow_to_yaml). This is the near-term ROADMAP-inversion path that lands before self-host (plan invert-hand-maintained §7), and is SCAFFOLD-marked to dissolve into the eventual v2 06_translate Markdown TargetModel — the same spelling rows relocate, no rework.

Changes

  • dsl/std/markdown.dag — extend the shared IR (single authority, §3): CheckboxState = Checked | Unchecked (a closed 2-sum, not Optional<Bool> — dodges the Value::Null literal-null straddle), MarkdownTaskItem, TaskListBlock, TableBlock, BlockquoteBlock. Adds the matching HTML-fragment arms + fail-closed XSS URL-scan arms so the existing IR→HTML path stays exhaustive and sound.
  • dsl/extdeps/languages/markdown.dag — the MarkdownSpellings row table (CommonMark 0.31.2 + GFM 0.29 spellings, cited) + the row-driven serialize_markdown_source fold (and serialize_markdown_source_with(doc, spellings) for parameterization).
  • dsl/test/claim/markdown_source_emit_test.dag — by-execution witness: exact bytes over a document exercising every construct family incl. nested task lists, plus a row-discipline tooth (perturbing the bullet row goes RED — verified false-on-perturb, true-on-revert).

Verification (by execution, v1 seed)

  • markdown_source_emit_keystone_holds() → true
  • existing markdown_markup_xss_keystone_holds() → true (HTML path unbroken)
  • whole-tree gunbc compile --target rust: 408 files emitted, 0 diagnostics
  • discriminating RED confirmed: a real bullet-row perturbation flips the witness to false

Consumer interface

serialize_markdown_source(doc: MarkdownDocument) -> String + std.markdown { CheckboxState, MarkdownTaskItem, TaskListBlock } — consumed by quiet-hawk-47's project_roadmap_to_markdown.

🤖 Generated with Claude Code

Brian Searls and others added 2 commits June 21, 2026 23:04
…(Path B, ci.yml/serialize_yaml clone)

The medium rows for headings, lists, GFM task-checkboxes, links, tables, code,
emphasis, blockquotes — authored as a spelling-row table the fold LOOKS UP, never
inline delimiter literals (the medium-as-Node guard, #5496). Peer of serialize_yaml:
a byte-faithful MarkdownDocument -> markdown-SOURCE serializer on the v1 seed, the
near-term ROADMAP-inversion path (plan invert-hand-maintained §7, before self-host).
SCAFFOLD-marked to dissolve into the eventual v2 06_translate Markdown TargetModel.

- dsl/std/markdown.dag: extend the shared IR (§3) with CheckboxState (Checked|Unchecked,
  NOT Optional<Bool>), MarkdownTaskItem, TaskListBlock, TableBlock, BlockquoteBlock; add
  matching HTML-fragment arms + fail-closed XSS URL-scan arms.
- dsl/extdeps/languages/markdown.dag: MarkdownSpellings row table (CommonMark 0.31.2 +
  GFM 0.29, cited) + the row-driven serialize_markdown_source fold.
- dsl/test/claim/markdown_source_emit_test.dag: by-execution witness — exact bytes over
  all families incl nested task lists + a row-discipline tooth (RED on bullet-row perturb).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@gunbai-bot
gunbai-bot Bot force-pushed the session/deep-newt-165 branch from 3d2827b to 01e48e6 Compare June 21, 2026 23:15
@gunbai-bot gunbai-bot Bot changed the title Add Markdown as an emit target - the medium rows for headings, lists, task-checkboxes, links, tables, code, emphasis - in extdeps/languages/markdown, hard-bounded from the work-DAG model Add Markdown as an emit medium: row-driven serialize_markdown_source (medium rows) Jun 21, 2026
@gunbai-bot
gunbai-bot Bot marked this pull request as ready for review June 21, 2026 23:16
@gunbai-bot

gunbai-bot Bot commented Jun 21, 2026

Copy link
Copy Markdown
Contributor

CI note for reviewers/operator — the failing per-head check is cross-repo, not this diff.

The modeling-coherence check fails inside the ctrl repo, not gunbc:
circular dependency: plans.coherence -> plans.reducible -> coherence.coherence_input -> coherence.coherence_query and module 'std.reducible' not found (imported by 'plans.reducible') at /home/briansrls/ctrl/plans/lib/reducible.dag.

This PR's diff is gunbc-only (three dsl/ markdown files) and touches zero reducible/coherence/plans modules — std.reducible exists fine in gunbc as dsl/std/reducible.dag. The breakage is a pre-existing ctrl/plans corpus issue gating all gunbc PR heads; it is not fixable from this PR (the ctrl repo isn't present in this session).

For the gunbc side: whole-tree gunbc compile --source-root dsl --source-root src/v2 --target rust → 408 files emitted, 0 diagnostics, and both witnesses are green by execution (markdown_source_emit_keystone_holds, existing markdown_markup_xss_keystone_holds). The one earlier dsl_compile_clean_gate failure was an intermediate auto-WIP snapshot (new MarkdownBlock variants committed before the matching HTML/XSS arms — a transient non-exhaustive match), resolved in the final commit.

— sent from deep-newt-165

…chor (CommonMark 0.31.2)

The external-authority lens (every extdeps/ module must cite its concept spec as a
structured ExternalAuthority fact, §3) failed closed on the new module — correctly: a
new extdeps module is neither machinery-exempt nor in the backfill_pending snapshot.
Anchor added below the imports, matching yaml.dag/elf. Verified: corpus_live_clean_tree
-> true, markdown witness still green, compile-clean 408 files / 0 diagnostics.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@gunbai-bot
gunbai-bot Bot force-pushed the session/deep-newt-165 branch from 1630352 to e817836 Compare June 21, 2026 23:30
@gunbai-bot

gunbai-bot Bot commented Jun 21, 2026

Copy link
Copy Markdown
Contributor

Update — the gunbc ci failure was a real anchor-discipline defect in this diff (now fixed in e817836); correcting my earlier note.

When the gunbc CI floor finished it caught a genuine fail-closed I had missed pre-push: the new dsl/extdeps/languages/markdown.dag cited CommonMark/GFM only in comments and lacked the structured data extdeps_external_authority_anchor: ExternalAuthority that the external-authority lens (§3) requires of every extdeps/ module. Correctly fail-closed — a new extdeps module is neither machinery-exempt nor in the backfill_pending snapshot. My earlier comment under-ran the floor locally (witness + compile-clean only, not the live external-authority scan), so I missed it.

Fixed: added the anchor (spec.commonmark.org/0.31.2, below the imports, matching yaml.dag/elf). Re-verified locally: corpus_live_clean_tree_holds → true, markdown witness → true, whole-tree compile-clean 408 files / 0 diagnostics. New CI is running.

The separate modeling-coherence per-head check remains the cross-repo ctrl/plans breakage described above — unrelated to this diff.

— sent from deep-newt-165

@briansrls
briansrls merged commit 00d1f5d into main Jun 21, 2026
1 check passed
@briansrls
briansrls deleted the session/deep-newt-165 branch June 21, 2026 23:50
briansrls added a commit that referenced this pull request Jun 22, 2026
…round-trip is the completeness gate, never the emit-PR bar)

quick-seal-137 (lane owner) refinement: 'a target is done iff round-trip green' must not be
misread as 'emit PRs cannot merge until round-trip green' — that would over-gate. An emit-only
PR with correct green grammar rows is merge-ready AS the emit step; the medium is COMPLETE only
once its round-trip law is green (DecodeFidelity-bounded). markdown #5501 + bash #5505 landed the
emit step; their round-trip completeness is the named next step, not a merge precondition.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
briansrls added a commit that referenced this pull request Jun 22, 2026
…per-medium decidability partition (#5513)

* WIP: roadmap reflection + analysis

* plan(emission): §5.2 — emit-step-merge-ready is NOT medium-complete (round-trip is the completeness gate, never the emit-PR bar)

quick-seal-137 (lane owner) refinement: 'a target is done iff round-trip green' must not be
misread as 'emit PRs cannot merge until round-trip green' — that would over-gate. An emit-only
PR with correct green grammar rows is merge-ready AS the emit step; the medium is COMPLETE only
once its round-trip law is green (DecodeFidelity-bounded). markdown #5501 + bash #5505 landed the
emit step; their round-trip completeness is the named next step, not a merge precondition.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Brian Searls <briansrls@gunb.ai>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
gunbai-bot Bot pushed a commit that referenced this pull request Jun 22, 2026
…ompleteness oracle

Implements the §5.2 round-trip law `ingest ∘ emit = id` for markdown: a row-driven
markdown INGEST (extdeps.languages.markdown) reading the SAME `MarkdownSpellings`
rows the emitter writes — DESIGN §4 "one grammar, both directions" turned into the
test. Proven GREEN by execution over the canonical core (headings, paragraphs,
flat unordered/ordered/GFM-task lists, fenced code; plain-text inlines).

- ingest_markdown_source / ingest_markdown_source_with: fence-aware line grouping
  into top-level blocks, per-block dispatch on the leading spelling ROW (not an
  inlined literal), inverse of serialize_markdown_source over the same rows.
- markdown_medium_lossless: the §5.2 (B) per-medium decidability partition — the
  recovered core is tagged Lossless (extdeps.communication.medium); the frontier
  residue (inline markup, nested lists, tables, block quotes) stays the unclaimed
  Lossy partition rather than fabricated (§5 fail-closed honesty).

Witness dsl/test/claim/markdown_roundtrip_test.dag, 4 teeth:
  1. round-trip law over the core (ingest(emit(doc)) == doc);
  2. row-shared both directions — emit AND ingest under a perturbed bullet row
     still round-trips (medium-as-Node guard #5496, ingest side);
  3. discriminating — emit perturbed, ingest canonical MUST fail (proves the
     row is read, not hard-coded);
  4. fidelity tag — recovered core advertised Lossless.

Round-trip completeness is the medium-COMPLETENESS step (§5.2), a follow-on to
the #5501 emit step, not a re-gate of it.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
briansrls added a commit that referenced this pull request Jun 22, 2026
… medium) as the target-completeness acceptance test + per-medium decidability partition, per merged #5513/#5496 plan. First medium = markdown: a markdown INGEST reading the SAME serialize_markdown rows, proving ingest-of-emit green (#5525)

* WIP: Emission slice: implement the §5.2 round-trip law (ingest-of-emit per me

* emission(§5.2): markdown ingest-of-emit round-trip law — the medium-completeness oracle

Implements the §5.2 round-trip law `ingest ∘ emit = id` for markdown: a row-driven
markdown INGEST (extdeps.languages.markdown) reading the SAME `MarkdownSpellings`
rows the emitter writes — DESIGN §4 "one grammar, both directions" turned into the
test. Proven GREEN by execution over the canonical core (headings, paragraphs,
flat unordered/ordered/GFM-task lists, fenced code; plain-text inlines).

- ingest_markdown_source / ingest_markdown_source_with: fence-aware line grouping
  into top-level blocks, per-block dispatch on the leading spelling ROW (not an
  inlined literal), inverse of serialize_markdown_source over the same rows.
- markdown_medium_lossless: the §5.2 (B) per-medium decidability partition — the
  recovered core is tagged Lossless (extdeps.communication.medium); the frontier
  residue (inline markup, nested lists, tables, block quotes) stays the unclaimed
  Lossy partition rather than fabricated (§5 fail-closed honesty).

Witness dsl/test/claim/markdown_roundtrip_test.dag, 4 teeth:
  1. round-trip law over the core (ingest(emit(doc)) == doc);
  2. row-shared both directions — emit AND ingest under a perturbed bullet row
     still round-trips (medium-as-Node guard #5496, ingest side);
  3. discriminating — emit perturbed, ingest canonical MUST fail (proves the
     row is read, not hard-coded);
  4. fidelity tag — recovered core advertised Lossless.

Round-trip completeness is the medium-COMPLETENESS step (§5.2), a follow-on to
the #5501 emit step, not a re-gate of it.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* WIP: Emission slice: implement the §5.2 round-trip law (ingest-of-emit per me

* emission(§5.2): address review — read block_separator row, executable fidelity boundary

Review #5525 (claude/opus-4-7) fixes:
- string_contains call aligned to the single-authority convention
  `string_contains(s:, pattern:)` (was positional) — md_is_digit.
- md_split_blocks now READS `sp.block_separator` (strip the one trailing EOF
  newline, split on the separator row) instead of hard-coding blank-line
  grouping. block_separator is now genuinely row-shared in both directions; the
  fence-aware grouper is gone (code bodies with an embedded separator are the
  documented Lossy frontier). Honest ROW-SHARING LEDGER added: the rows shared
  AND exercised over the core are enumerated; list_indent (nesting) and the
  inline/table/blockquote spellings are named as frontier, not claimed.
- Witness widened to perturb TWO independent rows (unordered_bullet AND
  block_separator) for both the round-trip-under-perturbation tooth and the
  discriminating tooth — the row-sharing claim is no longer inert against any
  single non-bullet row (DESIGN §5 inert-lens guard).
- New tooth: the Lossy frontier is now EXECUTABLE — an emphasis-bearing doc is
  proven to NOT round-trip, so the boundary is enforced by execution, not just
  asserted in a comment (catches a silent core→frontier overclaim).

All 5 teeth green by execution; emit keystone unaffected.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* WIP: Emission slice: implement the §5.2 round-trip law (ingest-of-emit per me

* WIP: Emission slice: implement the §5.2 round-trip law (ingest-of-emit per me

* chore(emission): revert auto-WIP GHA-expr contamination from the markdown PR

The auto-WIP committer swept the in-progress SECOND-medium work (GHA `${{ }}`
expression round-trip: dsl/extdeps/github/expressions.dag refactor + ingest, and
its witness) onto this markdown round-trip branch. That snapshot did not compile
(a since-fixed Optional if-branch type error) and shipped ingest code without its
witness. This restores expressions.dag to main and drops the stray test so this
PR's squash diff is markdown-ONLY again (the signed, green 077ed2b content). The
GHA-expr round-trip slice moves to its own branch + PR.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Brian Searls <briansearls1@gmail.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant