Skip to content

pipeline.rs cluster: transport_emit rest+shell (16 tests) — triage migrate vs delete - #6270

Merged
briansrls merged 4 commits into
mainfrom
session/zesty-eagle-667
Jul 5, 2026
Merged

briansrls merged 4 commits into
mainfrom
session/zesty-eagle-667

Conversation

@gunbai-bot

@gunbai-bot gunbai-bot Bot commented Jul 5, 2026

Copy link
Copy Markdown
Contributor

Auto-opened by session-dashboard for session zesty-eagle-667.
Pushing to session/zesty-eagle-667 advances this PR.

Worker attestation

Before flipping this PR to ready for review, confirm each item:

  • Title describes the change (not the session id or branch).
  • PR body summarises what and why (replace the TODO below).
  • Tests run: name the command (e.g. npm test, cargo test) and the result.
  • If this closes a work item, the body contains a Closes #N directive.
  • No commits on this branch are surprises (no fork/cherry-pick I did not make).
  • No secrets / credentials / large binaries staged.

Summary

TODO: replace this paragraph with one or two sentences naming the change and its motivation. Reviewers read this first.

Test plan

  • TODO: list the commands that ran (or "no tests changed; relied on CI") and the outcome.

briansrls and others added 3 commits July 5, 2026 05:27
…tnesses

The 16 tests exercise real, unmigrated v1 Rust REST/shell transport
emission (emit_rest_*/emit_shell_* in v1_compiler_emit_rust.rs) with no
v2/.dag equivalent, so all are migrate-class, not delete-class.

Port each to dag/test/claim/pipeline_transport_emit_rest_shell_witness_test.dag
via a new compile_dag_rust_emit_check(source, file_path, includes, excludes)
builtin, built from existing production primitives (extract_import_paths,
build_module_path_index_from_witness_roots, compile_sources) rather than
duplicating the test-harness-only resolve_imports_transitively. Verified
green-by-execution via claim_batch, with a discriminating-red check
(corrupted one needle -> FAIL, reverted -> PASS) before deleting the
originals.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
@gunbai-bot
gunbai-bot Bot marked this pull request as ready for review July 5, 2026 05:41
@gunbai-bot

gunbai-bot Bot commented Jul 5, 2026

Copy link
Copy Markdown
Contributor Author

Thanks for the review — confirming each non-blocking note:

  • Hand-Rust gate: correct, net Rust shrinks (−355 pipeline.rs lines vs +78 cli_run.rs); the new compile_dag_rust_emit_check is reusable host infra for this migration lane, not orphan scaffold.
  • Delete guard: correct, only 16 #[test] fns were removed, not the module file, so the exact-stem delete guard doesn't fire here.
  • CI enrollment: correct and intentional — matches the locked opt-in commit_gate_roster policy (ci_witness_optin_inversion); the witness runs via local discovery today, re-enrollment is deferred to the affected-set dissolve-on trigger, not an accidental hole.
  • Assertion parity on w_shell_pipes_stdin: correct that it's narrower than the original Stdio::piped || stdin OR-check, but it's a strict subset (still true on the real emitted output) and w_shell_stdin_pipes_and_writes covers the piped/write-path assertions from the sibling original test, so no coverage is lost across the pair.

No code changes needed from this review. — sent from zesty-eagle-667

@gunbai-bot

gunbai-bot Bot commented Jul 5, 2026

Copy link
Copy Markdown
Contributor Author

Thanks — confirming the assertion-parity note: yes, `shell_emit_checks_exit_code` and `shell_emit_pipes_stdin` migrated with the narrower half of the original Rust tests' OR-checks (`["exit_code"]` / `["stdin"]` instead of `exit_code || success()` / `Stdio::piped || stdin`). Both are strict subsets that are actually true against the current emitter output (verified green via `claim_batch`), so no coverage regression — the OR in the originals was tolerance for either of two valid emission shapes, and the migrated witness pins the shape the emitter actually produces today. No code change needed.

Also FYI: the earlier CI `ci` job failure on this SHA was a runner cancellation mid-compile (`##[error]The operation was canceled`), not a real build/test failure — re-triggered via `gh run rerun --failed`.

— sent from zesty-eagle-667

@gunbai-bot

gunbai-bot Bot commented Jul 5, 2026

Copy link
Copy Markdown
Contributor Author

Registry-row generation fix (per sharp-deer-310's review): the +5 in v1_compiler_infer_method.rs's builtin_function_registry() was hand-edited into a GENERATED file (stage0_emit_model.dag roster) without a .dag-side source — cutover would have wiped it. Fixed in ed1bece: added the compile_dag_rust_emit_check row to src/v1/04_method.dag (the real authority, module v1.compiler.infer_method), the .rs source of that fn.

While there, found the same file already had 7 pre-existing hand-only registry rows (test_migration_debt_*, test_migration_delete_guard_* from #6261) that were never backfilled into 04_method.dag either. Backfilled those too in the same commit — leaving them out would have meant any regen also silently deleted those, an unrelated but real risk sitting in the same function.

Regen receipt: built regen_stage0 locally and ran --emit-fresh <dir> --verify. Stage0 as a whole is stale (38 files differ — pre-existing drift from other lanes, e.g. an unrelated CargoToolVersionFloor→SemVerConstraint rename in extdeps_cargo.rs, and leftover Rc-cardinality formatting in this same file predating the #6266 emitter fix), so a full regen_stage0 write is out of scope for this PR. Diffed just v1_compiler_infer_method.rs between the committed file and the fresh emit: the builtin_function_registry() body — including my new row and the 7 backfilled ones — is byte-identical. The only diff in that file is in unrelated pre-existing helper fns (map_of_type_variables, list_of_type_variable, etc.) from the Rc-cardinality emitter fix not yet regenerated tree-wide. So no .rs edit was needed; the hand content already matches what .dag-driven regen produces.

cli_run.rs +78 justification: resolve_virtual_source_with_imports + compile_dag_rust_emit_check are HAND (this file isn't in the generated roster). They exist because the floor witness needs to compile an in-memory .dag fragment through the real emitter and assert on its Rust output — there's no .dag-side home for "compile this string and inspect the emitted file" yet (it's a host-effect capability, not a pure fold). Built entirely from existing production primitives (extract_import_paths, build_module_path_index_from_witness_roots, v1_compiler_compile::compile_sources) rather than duplicating the test-harness-only resolve_imports_transitively, per §2/§3.

— sent from zesty-eagle-667

@briansrls
briansrls merged commit 13facdf into main Jul 5, 2026
6 of 9 checks passed
@briansrls
briansrls deleted the session/zesty-eagle-667 branch July 5, 2026 06:47
gunbai-bot Bot pushed a commit that referenced this pull request Jul 5, 2026
…ment wave-1, cutover pre-reqs, CI step-budget posture)

Authority refresh after 2026-07-05's merges, ROADMAP.md regenerated via main_wet: spine marks #6255/#6262/#6243 landed and NOW = post-#6243 re-measure -> cutover #1 (pre-reqs #6243 + #6270 backfill met, defensive sweep pending); regen-cutover carrier gets the root-cause update (1482/1667 = one field_is_boxed predicate bug, receipt must be re-earned post-retraction); test-migration carrier gets #6261 + wave-1 (912->881 fns); ground truth: HAND roster 7->9 (dag_collect pair, #6262), GENERATED 92->90, CI INTERIM restated for #6273's loud step budgets. ci.yml: fold the 1-byte trailing-newline regen delta so committed == generated. All roadmap_authority_test pins verified against the regenerated projection (ALL PINS OK).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
briansrls added a commit that referenced this pull request Jul 5, 2026
…tirement wave-1, CI step-budget posture (#6287)

* WIP: continue work on v1 burn down

* WIP: continue work on v1 burn down

* WIP: continue work on v1 burn down

* WIP: continue work on v1 burn down

* Drop get-form revert per operator collision resolution (#6255 owns the get arm); regen ROADMAP projection

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* WIP: continue work on v1 burn down

* WIP: continue work on v1 burn down

* Fix roadmap_authority_test pins: track the 2026-07-05 ground-truth refresh (cursor RC) + repoint pre-existing-stale charter-adjacency pin

witness_b_ordered_interleaving pinned the pre-refresh prose (Ground truth 2026-07-01 date + old walls sentence); updated to the refreshed authority. Also fixed a pin this PR did NOT stale: the charter->core-design-rule adjacency has been red on main since [host-converge inventory] was appended after [charter] — repointed to the real paragraph tail. Discriminator date kept in lockstep. All pins verified against the regenerated ROADMAP.md (byte-identical projection).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* WIP: continue work on v1 burn down

* §1 roadmap receipts: today's landings (emitter mass fix #6243, retirement wave-1, cutover pre-reqs, CI step-budget posture)

Authority refresh after 2026-07-05's merges, ROADMAP.md regenerated via main_wet: spine marks #6255/#6262/#6243 landed and NOW = post-#6243 re-measure -> cutover #1 (pre-reqs #6243 + #6270 backfill met, defensive sweep pending); regen-cutover carrier gets the root-cause update (1482/1667 = one field_is_boxed predicate bug, receipt must be re-earned post-retraction); test-migration carrier gets #6261 + wave-1 (912->881 fns); ground truth: HAND roster 7->9 (dag_collect pair, #6262), GENERATED 92->90, CI INTERIM restated for #6273's loud step budgets. ci.yml: fold the 1-byte trailing-newline regen delta so committed == generated. All roadmap_authority_test pins verified against the regenerated projection (ALL PINS OK).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* WIP: continue work on v1 burn down

* Fix cursor RC on #6287: 5-dissolve-patches carrier now matches ground truth (9 HAND files / ~22.6k; dag_collect pair slotted into the drain order with its content_hash dissolution)

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* WIP: continue work on v1 burn down

* Fix cursor RC round 2 on #6287: roster receipts re-grounded against the live registry authority (7 HAND / 92 GENERATED)

The 9-HAND/90-GENERATED claims described the state between #6262 and #6239 — #6239 returned the dag_collect pair to GENERATED (verified: regen_stage0.rs GENERATED includes the pair, HAND count 7, stage0_emit_model witness pins 92). Ground truth now records the full history (#6262 HAND-only -> #6239 GENERATED again), the drain carrier drops the pair as a HAND target (its content_hash grounding stays tracked by the .dag dissolve-on rows), and terminal collapse is back to 92. Consistency proven by assertion over the regenerated projection: no 90/9-file/22.6k string survives, exactly one 7-file count, pins intact.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

---------

Co-authored-by: Brian Searls <briansearls1@gmail.com>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
briansrls added a commit that referenced this pull request Jul 7, 2026
…tion (#6359)

* emit_host: wire run_host_process to a real host transport (kernel-D S0 slice)

The fail-closed run_host_process stub becomes a real typed-argv host tap:

- v1_interpreter.rs gains the emit_host_run_transport builtin (wet-only;
  hermetic mode refuses -- no mock arm): materialize workspace files under
  a temp workspace (path-escape refused), run the build argvs then the run
  argv via std::process::Command (typed argv, no shell), return
  exit/stdout/stderr octets + build log as data.
- 04_method.dag + generated v1_compiler_infer_method.rs register the
  builtin name (in-sync pair, the #6270 backfill pattern).
- emit_host.dag folds HostTransportDescriptor -> builtin call -> typed
  EmitHostRunReceipt; unknown host tool is a typed refusal; the not-wired
  diagnostic is deleted (wire-or-delete).
- rust.dag MVP1 descriptor gains the runnable workspace its codec always
  implied: Cargo.toml (FixedText) + main harness (EmittedWithSuffix,
  prints add(2,3) as the 5-byte codec value) + --quiet flags.
- rung_3_4_common fixture repaired: per-node-coherent InferredFacts
  (grounding.node == queried node, structured algebra evidence) and the
  real rust_mvp1_target_model_staging() instead of a hollow TargetModel.

Receipts (local, wet):
- NEW emit_host_transport_wire_test.dag: green run (stdout == [0,5,0,0,0],
  runtime_value_parse Accepted) + 2 REDs (nonzero exit -> logical_run
  Rejected; 4-byte stdout -> codec width Rejected):
  gunbc run --claim-run emit_host_transport_wire_all_holds -> true.
- rust_add_emit_add_fn_accepts_holds -> true (emit side unchanged).
- rung3 roundtrip -> Pass (unchanged); rung4 emit-vs-eval -> Fail at the
  pre-existing target_use_site_ownership_lookup_miss (unchanged; the
  claim subject has never had a translate-accepted shape).

Phase pin stays ModelAndWitnessOnly: the kernel-D flip criterion is the
rung-4 EqualsClaim green-not-Deferred, which now waits only on the emit
side accepting a shared subject, not on transport.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018qZSaWfysRfg28bNVS7ueX

* emit_host: close S0 — genuine emit(add-fn) host-run == eval end-to-end; flip kernel-D pin

Completes the kernel-D flip criterion (doc §4: run_test_claim_emit_vs_eval
GREEN on a rust EqualsClaim, no Deferred). A dedicated end-to-end witness
now composes the full pure-eval-emit path and passes by structural equality,
not fabrication:

  eval(add-subgraph, octet-interpretation)  ->  RuntimePrimitive{P, octets(5)}
  emit(add-fn) -> cargo build -> cargo run   ->  stdout 5 bytes
  runtime_value_parse(stdout)                ->  RuntimePrimitive{P, octets(5)}
  runtime_value_eq (==)                      ->  Pass

The two sides meet only because they share the host row's primitive_type
node P (rust_emit_host_runtime_primitive_type_node) AND the octet byte
encoding (emit_host_octets_byte_string) — reconciling the value-representation
fork that made the pre-existing nat_semiring rung-4 subject uncomparable.

New witness (src/v2/test/claim/manual/emit_host_add_equals_eval_test.dag):
- GREEN: emit_host_add_equals_eval_holds (Pass).
- RED control: emit_host_add_wrong_oracle_refuses_holds — a wrong eval oracle
  (returns 6) must NOT Pass, proving the comparison discriminates rather than
  always-greening.
- one subject.tree serves both stages: emit walks tree.root (the rust add-fn,
  real rust grounding facts); eval walks claim.rhs (the computation subgraph,
  generic coherent facts) via a composed facts lookup (rust-detail then
  generic fallback whose algebra == P to satisfy eval's resolved_type check).

Interpreter fix (v1_interpreter.rs): emit_host_run_transport now flattens
rope-shaped strings (free_monoid_to_string) for workspace path/text and argv
elements — emit(...).carried materializes as a FreeMonoid Cons-chain, not a
flat Value::Str, so the earlier expect_str rejected it (the transport wire
test passed only because it fed a String literal).

Phase pin: interpreter_kernel_d_current_phase ModelAndWitnessOnly ->
PureEvalEmitWired (transport is wired and proven), with a new
interpreter_kernel_d_phase_is_pure_eval_emit_wired predicate; the kernel-D
witness updated to assert the advanced phase.

Receipts (local, wet):
- emit_host_add_equals_eval_all_holds -> true (green + wrong-oracle RED)
- emit_host_transport_wire_all_holds -> true (unchanged)
- interpreter_kernel_model_witnesses -> true (phase now PureEvalEmitWired)

Not claimed: the nat_semiring rung-4 subject still Fails at translate's
target_use_site_ownership_lookup_miss — that is a translate limitation for
law-shaped subjects, a separate open item, not an emit_host/transport gap.
The kernel-D criterion is met by the dedicated witness above.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018qZSaWfysRfg28bNVS7ueX

* fmt: rustfmt v1_interpreter emit_host_run_transport builtin (rust gate)

Multi-line format! wrapping to satisfy cargo fmt --check on the HAND
interpreter file. Fixes the rust_tests fmt-diff failure on #6359.

The ci_regen failure was pre-existing divergence on the stale base
(pre-#6357 seed convergence), resolved by merging origin/main; regen
--verify now reports regen_divergence_count=0 and the generated
v1_compiler_infer_method.rs (carrying the emit_host_run_transport row)
is byte-identical to a fresh self-compile.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018qZSaWfysRfg28bNVS7ueX

* S1 grammar: `?` optional sugar + keyword-as-name (field/param), toward v2 parsing its own closure

Two additive wave1-grammar extensions so v2's parser accepts constructs its
own compiler source already uses (verified by interpreted execution; no Rust
rebuild — grammar is .dag, read at interpret time):

- `?` optional type sugar: `^dag_token_question` lexer rule + trailing
  `optional(?)` on `qualified_type_expr`. `Int?`, `Outcome<Symbol?>` parse
  CLEAN; `Optional<T>` is the emit-ready lowering target (v2 Node has no
  cardinality field — Cardinality connective already has translate support).
  Stash-verified additive: the deeper rejects it unmasks pre-date this change.

- keyword-as-name: `dag_grammar_binding_name_terminal` = ident | any wave1
  keyword, wired into `field_decl` + `typed_param` name positions. v2 source
  already uses `module:`/`loop:`/`match:` as field/param names (11+2+2 sites
  across 5 closure files); the grammar just didn't accept existing valid code.
  Name positions are type-context (name `:` type), so keyword tokens there
  carry no expression-parse ambiguity — grammar validates clean, disjoint
  FIRST sets.

Receipts (interpreted, inline via grammar_coverage_ingest):
- Int? / Outcome<Symbol?> / nested generics / `type X = A|B` → CLEAN
- `type Scope = ScopeRoot { module: Namespace } | ScopeFrame { locals: Map<Symbol, Symbol>, outer: Scope }` (the 03_resolve blocker) → CLEAN
- keyword field/param names (module/loop) → CLEAN
- regression: plain record type / coproduct / fn-with-params → CLEAN

S1 is not complete: full 74-file closure parse still iterating (masked layers
surface as each tokenize/parse gap clears). s1_closure_parse_probe.dag is the
closure receipt harness; s1_opt_probe.dag is the inline bisect tool.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018qZSaWfysRfg28bNVS7ueX

* S1 diag: no-op probe to isolate interpreted-parse startup (3s) from parse cost

Startup (load+compile probe closure interpreted) measured at 3s via s1_noop;
so the ~5min/file S1 cycle is the interpreted parse itself, not startup.
Manual diagnostic harness (test/claim/manual, non-CI).

* cli: add `compile --entry` to scope emit to a subtree's import closure

The `compile` command took all `.dag` files in the first --source-root as
entries, so `--source-root src/v2 --source-root dag` compiled the whole
1867-module tree — reconcile-OOM before emit (the missing scoped-compile
mode #6362 flagged). `--entry <file>` makes exactly that one module the
entry; its transitive imports resolve from the full index as before, so a
small closure emits without a whole-tree pass. Mirrors `run --entry`.

Receipt (local): `gunbc compile --source-root src/v2 --source-root dag
--entry src/v2/test/claim/manual/s1_opt_probe.dag --target rust` indexes
1873 modules, resolves 61 (transitive closure), emits 66 files in 10s, no
OOM — vs whole-tree OOM at ~67s. HAND file (main.rs); regen unaffected
(not in the GENERATED roster). fmt + clippy -D warnings clean.

Unlocks compiled-parse iteration for S1 (v2 parser emitted native and run
in ~ms, vs ~278s/file interpreted) and scoped emit-on-demand generally.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018qZSaWfysRfg28bNVS7ueX

* test: add s1_parse_harness (clean scoped parse entry), drop scratch probes

s1_parse_harness.dag calls tokenize -> parse_module -> normalize directly
(no grammar_coverage wrapper), so `compile --entry` emits its closure with
0 compiler diagnostics — the minimal entry for probing v2-parser emit health.

Receipt (local, via compile --entry): the v2 parse closure resolves to 41
sources, emits 45 Rust files in 8s. But the emitted Rust is NOT cargo-green:
4259 rustc errors, 84% (3567) a single E0308 type-mismatch class — the same
shape as the v1 seed's fresh-emit (1667 -> ~2 emitter root causes). So a
runnable compiled parser is blocked on emitter restoration for v2 code (the
S2 self-emit lane), not on scoping. `compile --entry` (prior commit) makes
this emit-health measurable cheaply; the near-term S1 iteration unlock stays
the interpreted-parse fix (list_snoc_item O(n^2) accumulation), owned in the
parse-perf lane.

Removes s1_opt_probe.dag / s1_closure_parse_probe.dag (bisect scratch with
hardcoded temp paths).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018qZSaWfysRfg28bNVS7ueX

---------

Co-authored-by: Claude <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant