Skip to content

CollectionOps consume Rust HO MethodTemplateContract refs - #1665

Merged
briansrls merged 108 commits into
mainfrom
r3-string-family-diagnostic-ordering-carrier
May 4, 2026
Merged

briansrls merged 108 commits into
mainfrom
r3-string-family-diagnostic-ordering-carrier

Conversation

@briansrls

@briansrls briansrls commented May 4, 2026 •

Copy link
Copy Markdown
Contributor

Summary

This PR completes the CollectionOps higher-order MethodTemplateContract consumption slice for Rust filter / flat_map / any / all. The existing MethodEmitTemplate coproduct remains the authority: Rust CollectionOps now reads contract references carried by CollectionOps.{filter,flat_map,any,all}_contract, validates the referenced method identity, requires HigherOrderTemplates, and selects the inline template for current render paths. It fails closed on SingleTemplate, wrong method refs, malformed payloads, or missing fields.

The HO Rust contract rows are now named declarations and are reused both by rust_method_template_contracts and by rust_collection_ops, so Grounding can migrate the Rust HO quartet without copying literal templates into a third authority. Go/Python CollectionOps also receive matching contract refs because CollectionOps now structurally requires these fields; their legacy literal fields remain for current unmigrated render paths.

This also addresses the blocking review on first-match list scanning: the Rust CollectionOps parser no longer scans rust_method_template_contracts by list name. It consumes the explicit contract ref from the CollectionOps value instead.

Validation

  • cargo run -p v3-compiler --features bootstrap-regen-fresh --bin regen_bootstrap
  • cargo fmt --all --check
  • cargo test -p v3-compiler rust_collection_ops_selects_higher_order_inline_templates --lib
  • cargo test -p v3-compiler rust_collection_ops_rejects_single_template_for_higher_order_selection --lib

Attempted but blocked by existing integration-test target setup before the selected test could run:

  • cargo test -p v3-compiler --test integration rustc_roundtrip_list_filter_then_fold_prints_seven failed compiling src/v3/compiler/tests/integration/p0_std_render_repeat_string_test.rs with can't find crate for v2_compiler_tests.

regen_bootstrap -- --verify was also attempted after the successful regen, but Cargo repeatedly stuck on a stale artifact-directory lock with no compiler process; I did not treat that optional pass as blocking after the successful regen plus focused tests.

briansrls added 30 commits May 2, 2026 12:14

@briansrls briansrls left a comment

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review metadata

  • Provider / model: codex / unknown
  • Commit: 46c84cab · Trigger: schedule
  • Thinking: 285s wall

Non-blocking — Strengths

  • src/v3/std/emit_model.dag The legacy CollectionOps literal-field scaffold now names the dissolution trigger required by INVARIANTS.md P5.

✅ No blocking concerns in the current diff.

@briansrls

Copy link
Copy Markdown
Contributor Author

Verified against current #1665 head a18c16bbd: this finding is already addressed.

The reference-row branch now resolves the referenced declaration, fetches the canonical MethodTemplateContract declaration id, requires the referenced declaration connective to be TypeConnective::Instantiation, and rejects unless template == method_template_contract before reading the structural body. The fail-closed error is MethodTemplateProjectionError::RowReferenceNotMethodTemplateContract; the regression test reference_row_requires_method_template_contract_instance covers the structural-lookalike case.

Relevant current code is src/v3/compiler/src/pb_method_template_projection.rs:529-553. The review appears to be against an older commit before that guard landed.

— sent from silent-hawk-421

@briansrls

Copy link
Copy Markdown
Contributor Author

Review metadata

  • Provider / model: cursor / composer-2
  • Commit: a18c16bb · Trigger: schedule
  • Comparison: origin/main @ bfb39ac0 ... review/pr-1665-a18c16bb @ a18c16bb
  • Thinking: 95s wall

Findings (if any)

  • NON-BLOCKING — TESTING.md (clear failure diagnostics / one claim per failure path): In method_ref_decl_from_row, method_template_contract_row_fields is called with a hard-coded row index 0 and row_context (e.g. "rust row 5") passed where list_name is intended, so panics from method_template_contract_row_fields can report the wrong row index and an odd pseudo–list name instead of rust_method_template_contracts. See 257:257:src/v3/compiler/tests/integration/method_template_contract_test.rs let fields = method_template_contract_row_fields(dag, row, row_context, 0); Fix would be to pass the real row index and list name (extend method_ref_decl_from_row’s parameters) so messages stay truthful.

No INVARIANTS / modeling-discipline / CODING.md issues stood out in the substantive diff: Rust CollectionOps reads MethodTemplateContract (including HigherOrderTemplates) from the DAG instead of duplicating template text; fail-closed Result / EmitError paths for shape mismatches; emit_model.dag documents the legacy filter string with a named dissolution trigger (tracked bridge, not orphan scaffold); PB projection and integration tests follow the new FieldValue::Reference row shape with typed errors in production code.

Verdict

APPROVE_WITH_COMMENTS — The main change is coherent and aligned with single-authority contracts and fail-closed parsing; only the test-helper messaging above is worth a quick follow-up (optional in this PR).

Exploratory observations (optional)

  • flat_map / any / all contracts are validated in parse_collection_ops but their inline templates are not yet stored on CollectionOpsBinding (only filter is wired into emission). That is consistent with “validate substrate early” and matches current Rust callable coverage (filter-only in the edited emission path); not raised as a violation because nothing in the diff claims those emit paths are migrated yet.

@briansrls

Copy link
Copy Markdown
Contributor Author

Review metadata

  • Provider / model: cursor / composer-2
  • Commit: 022f63a3 · Trigger: schedule
  • Comparison: origin/main @ 671b3602 ... review/pr-1665-022f63a3 @ 022f63a3
  • Thinking: 49s wall

Findings

None. The substantive diff moves Rust filter (and validates flat_map / any / all) through MethodTemplateContract + HigherOrderTemplates, maps &'static str contract errors into EmitError::MalformedClosed-style MalformedTargetSyntax, documents the remaining literal CollectionOps fields with a clear dissolution trigger in emit_model.dag, and extends PB projection + tests for FieldValue::Reference rows without introducing a second live authority for the Rust HO inline path you actually emit.

Verdict

APPROVE — Scoped implementation-layer change: single-source contracts for Rust collection HO templates, explicit fail-closed checks, reference rows handled in projection, and a focused rustc roundtrip regression for non-copy filter. Nothing in the reviewed diff clearly violates INVARIANTS, docs/modeling-discipline.md, CODING.md, or TESTING.md.

@briansrls

Copy link
Copy Markdown
Contributor Author

Review metadata

  • Provider / model: cursor / composer-2
  • Commit: 3f78994b · Trigger: schedule
  • Comparison: origin/main @ 522fa3d3 ... review/pr-1665-3f78994b @ 3f78994b
  • Thinking: 62s wall

Findings

None. The diff moves Rust filter (and validates flat_map / any / all) through MethodTemplateContract with fail-closed shape checks (require_single_template / require_higher_order_inline_template in rust_target.rs), adds a 🟢 terminal classification on MethodTemplateContractEmitTemplate in collection_ops_method_contract.rs, documents a named dissolution for the legacy filter string on CollectionOps in emit_model.dag, and extends projection/tests for declaration-reference rows without introducing a second template authority for the wired Rust path.

Verdict

APPROVE — Scoped, aligns with single-authority template data in .dag, fail-closed errors and typed projection errors are in good shape, and the tracked bridge for legacy literals is explicitly bounded and triggered.

Exploratory observations (optional)

  • rust_target.rs unit tests pin full inline template strings (e.g. around the new rust_collection_ops_selects_higher_order_inline_templates test), so they will churn on any .dag template edit; that is a tradeoff for regression signal, not a rubric violation.
  • flat_map / any / all contracts are validated at parse time but only filter is carried on CollectionOpsBinding today; that matches a partial consumer migration and is consistent with the PR title’s emphasis on the path that actually emits.

…nostic-ordering-carrier

# Conflicts:
#	src/v3/compiler/src/bootstrap_generated.rs
#	src/v3/compiler/src/bootstrap_generated_without_parse_surface.rs

@briansrls briansrls left a comment

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review metadata

  • Provider / model: codex / unknown
  • Commit: 022f63a3 · Trigger: schedule
  • Thinking: 317s wall

BLOCKING (1)

Root Cause

  • src/v3/std/rust_method_template_contracts.dag the HO filter template changed the receiver from method-call auto-borrow to a value let-binding → render the loop over ({iter}).iter() or add a typed borrowed-receiver placeholder before using MethodTemplateContract here.

⚠️ One Rust filter ownership regression remains.

dag_method: { decl: filter_method },
runtime_template: "{ let __filter_source = {iter}; let mut __result = Vec::new(); for {param} in __filter_source.iter() { if {body} { __result.push((*{param}).clone()); } } __result }",
emit_template: HigherOrderTemplates {
inline_template: "{ let __filter_source = {iter}; let mut __result = Vec::new(); for {param} in __filter_source.iter() { if {body} { __result.push((*{param}).clone()); } } __result }",

This comment was marked as resolved.

@briansrls

Copy link
Copy Markdown
Contributor Author

Verified against current #1665 head 43917487b: this specific receiver-move issue is addressed by the latest commit.

The template still binds {iter} into __filter_source, but the Rust emitter now supplies {iter} as a borrowed receiver: let iter = format!("&({list})") in src/v3/compiler/src/emit/rust_target.rs. So the generated block binds a reference by value, not the named Vec, and then iterates __filter_source.iter(). The non-copy regression test rustc_roundtrip_list_filter_non_copy_record_predicate_prints_two and the existing rustc_roundtrip_list_filter_then_fold_prints_seven both passed after this change.

— sent from silent-hawk-421

@briansrls

Copy link
Copy Markdown
Contributor Author

Review metadata

  • Provider / model: cursor / composer-2
  • Commit: 43917487 · Trigger: schedule
  • Comparison: origin/main @ b4203b5a ... review/pr-1665-43917487 @ 43917487
  • Thinking: 61s wall

Findings

None. The diff tightens single-authority for Rust filter emission by reading MethodTemplateContract.emit_template through filter_contract, adds fail-closed validation (require_higher_order_inline_template, structured EmitError, projection errors for bad FieldValue::Reference rows), and documents the remaining literal CollectionOps.filter field as tracked work with a named dissolution trigger in emit_model.dag. The new MethodTemplateContractEmitTemplate enum carries an explicit terminal classification at the consumer boundary (per docs/modeling-discipline.md practice 4). Tests match TESTING.md: a focused rustc roundtrip for the regression and unit-level checks on template selection / projection.

Verdict

APPROVE — Substrate and Rust wiring look consistent with boundary discipline and fail-closed emission; no invariant or style violation surfaced in the reviewed diff.

@briansrls

Copy link
Copy Markdown
Contributor Author

Review metadata

  • Provider / model: cursor / composer-2
  • Commit: 49c61def · Trigger: schedule
  • Comparison: origin/main @ b4203b5a ... review/pr-1665-49c61def @ 49c61def
  • Thinking: 63s wall

Findings: None. The diff tightens single authority for Rust filter by reading MethodTemplateContract.emit_template (including HigherOrderTemplates.inline_template) instead of the CollectionOps.filter literal and render_list_item_construct_expr, wires CollectionOps’s new contract refs in emit_model.dag / target specs, resolves list rows by FieldValue::Reference in pb_method_template_projection.rs with typed errors, and documents the remaining legacy literal fields with a named dissolution trigger in emit_model.dag (aligned with P5 scaffold discipline). Fail-closed paths map parse failures to EmitError::MalformedTargetSyntax / MethodTemplateProjectionError rather than fabricating templates (P3). The new MethodTemplateContractEmitTemplate enum carries an explicit terminal classification comment (modeling-discipline practice 4). flat_map / any / all contracts are validated at parse_collection_ops time even though only ListFilter is wired through those templates in rust_target.rs today; that matches the current callable surface and does not introduce a second emission authority for those ops.

Verdict: APPROVE — Scoped, consistent with INVARIANTS / modeling-discipline / CODING / TESTING for what this diff changes; no concrete rubric violations observed.

@briansrls

Copy link
Copy Markdown
Contributor Author

Review metadata

  • Provider / model: cursor / composer-2
  • Commit: ba3106fa · Trigger: schedule
  • Comparison: origin/main @ b8c9821a ... review/pr-1665-ba3106fa @ ba3106fa
  • Thinking: 49s wall

Findings

None. The diff tightens single authority for Rust higher-order collection templates by routing filter (and validating flat_map / any / all) through MethodTemplateContract refs, with fail-closed EmitError / projection errors and explicit MethodEmitTemplate shape checks. emit_model.dag documents the remaining legacy CollectionOps string fields and names a dissolution trigger, which satisfies the tracked-scaffold bar in your instructions.

Verdict

APPROVE — Implementation-layer refactor aligned with P2 (one contract authority, consumers read through declared refs) and P3 (malformed contracts surface as typed errors, not silent defaults). No concrete rubric violations tied to specific diff lines.

Exploratory observations (optional)

  • .github/workflows/ci.yml bumps self_host_ratchet from 20 to 30 minutes; if that is only to absorb this PR’s cost, a one-line rationale in the PR description helps reviewers.
  • parse_collection_ops validates flat_map / any / all higher-order templates but only persists filter on CollectionOpsBinding (the others are checked then dropped). That is coherent if emission for those list operations is still elsewhere or not wired; worth confirming intent so it does not read like dead validation.

@briansrls briansrls left a comment

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review metadata

  • Provider / model: codex / unknown
  • Commit: 49c61def · Trigger: schedule
  • Thinking: 284s wall

Non-blocking — Strengths

  • src/v3/std/emit_model.dag The legacy CollectionOps literal scaffold is documented, bounded to unmigrated render paths, and has a named deletion trigger, so it is tracked rather than accumulating debt.

✅ No blocking concerns; the PR resolves the prior fail-closed and reference-row issues while keeping the remaining literal bridge tracked.

@briansrls

Copy link
Copy Markdown
Contributor Author

Review metadata

  • Provider / model: openai-pro / gpt-5-5-pro
  • Commit: ba3106fa · Trigger: manual
  • Comparison: main @ b8c9821a ... r3-string-family-diagnostic-ordering-carrier @ ba3106fa
  • Conversation: View conversation

1. Story of the diff

This PR moves Rust CollectionOps filter rendering off the legacy literal filter: String field and onto the same MethodTemplateContract authority already used by the per-target method-template rows. The substrate shape grows four contract-reference fields on CollectionOps for filter / flat_map / any / all (src/v3/std/emit_model.dag:372-375), each target spec wires those fields to named contract declarations, and the Rust spec points rust_collection_ops at the Rust higher-order contracts (src/v3/spec/rust.dag:1408-1411). On the Rust side, parse_collection_ops now resolves those declaration refs, validates that each contract targets the expected std method, selects HigherOrderTemplates.inline_template for the higher-order path, and uses that selected template for list-filter emission (src/v3/compiler/src/emit/rust_target.rs:1570-1581, src/v3/compiler/src/emit/rust_target.rs:4507-4510).

The supporting change is that method-template contract lists can now contain declaration references to named MethodTemplateContract data rows instead of duplicating record literals in-place. project_row follows a FieldValue::Reference only if the referenced declaration exists, instantiates MethodTemplateContract, and carries a structural body (src/v3/compiler/src/pb_method_template_projection.rs:521-555), while the target contract lists now read shared named rows such as rust_filter_method_template_contract (src/v3/std/rust_method_template_contracts.dag:82-90, src/v3/std/rust_method_template_contracts.dag:177-180). The generated bootstrap and parse manifest updates are the expected downstream refresh from those new declarations and ID shifts.

2. Invariant categories

  1. LAYER MODEL — Compliant. This touches substrate-facing CollectionOps, but the new fields are DeclarationRef identity edges, not duplicate render strings; emit_model.dag explicitly says template strings live only on the contracts (src/v3/std/emit_model.dag:362-366), and the Rust spec reads those contract declarations by ref (src/v3/spec/rust.dag:1408-1411).
  2. INVARIANTS.md + modeling-discipline.md — Compliant. Single-authority / facts-flow-forward is handled by replacing inline list rows with references to named contract authorities (src/v3/std/rust_method_template_contracts.dag:177-180) and by fail-closing reference projection unless the target is a structural MethodTemplateContract instance (src/v3/compiler/src/pb_method_template_projection.rs:521-555). The new Rust enum mirror is explicitly classified terminal at the projection boundary and says it mirrors the substrate MethodEmitTemplate coproduct rather than inventing a second taxonomy (src/v3/compiler/src/emit/collection_ops_method_contract.rs:9-20).
  3. CODING.md — Compliant. The new logic is mostly data + free-function shape: method_template_contract_decl_emit_template exposes a clear Dag + DeclarationId + expected method -> Result<MethodTemplateContractEmitTemplate, _> contract (src/v3/compiler/src/emit/collection_ops_method_contract.rs:38-50), and Rust-specific selection is isolated in small require_single_template / require_higher_order_inline_template helpers returning typed EmitError rather than silent defaults (src/v3/compiler/src/emit/rust_target.rs:1636-1664). The added Dag methods are narrow typed accessors over cached anchors (src/v3/compiler/src/dag.rs:3260-3277), consistent with the existing accessor pattern.
  4. TESTING.md — Compliant. The diff adds focused unit coverage for selecting higher-order inline templates and rejecting SingleTemplate for Rust higher-order CollectionOps fields (src/v3/compiler/src/emit/rust_target.rs:5881-5965), projection coverage for rejecting a reference row that is not a MethodTemplateContract (src/v3/compiler/src/pb_method_template_projection.rs:1050-1079), and a behavior-level Rust boundary regression proving non-Copy record filter keeps work through borrowed predicate + cloned output (src/v3/compiler/tests/boundary/m1_3_emit_rust_test.rs:1028-1042). The integration helper was also updated so existing uniqueness checks exercise both inline records and declaration refs (src/v3/compiler/tests/integration/method_template_contract_test.rs:206-249).
  5. LOCKED DESIGN DECISIONS — N/A. The diff does not alter the locked Arrow.body external-realization model, reflection completeness contract, host-process boundary, or other locked design surfaces; it only threads existing MethodTemplateContract declarations through CollectionOps.
  6. TRACKED vs UNTRACKED DEBT — Compliant. The only new temporary shape is the retained legacy collection render literals; the comment documents what remains, bounds it to legacy CollectionOps render literals, and names the dissolution trigger: delete them once all CollectionOps render paths select templates through contract refs (src/v3/std/emit_model.dag:377-381). No new TODOs or unbounded scaffolds show up in the diff.

3. Verdict

APPROVE

The PR moves Rust filter emission toward the declared contract authority without introducing a parallel template source, and the temporary leftover literal fields are explicitly bounded with a dissolution trigger. The fail-closed projection and Rust selection tests cover the risky seams I would expect for this change.

@briansrls

Copy link
Copy Markdown
Contributor Author

Review metadata

  • Provider / model: cursor / composer-2
  • Commit: c22075ac · Trigger: schedule
  • Comparison: origin/main @ 1e03ace8 ... review/pr-1665-c22075ac @ c22075ac
  • Thinking: 89s wall

Findings: None. Nothing in the diff clearly breaks INVARIANTS.md, docs/modeling-discipline.md, CODING.md, or TESTING.md in a way that needs a blocking or must-fix callout. Rust filter emission now follows MethodTemplateContract / HigherOrderTemplates from substrate; emit_model.dag documents the remaining legacy filter / map strings with a named dissolution trigger (P5 scaffold exception). pb_method_template_projection.rs fail-closedly resolves FieldValue::Reference rows against method_template_contract_decl(), which reinforces single authority rather than splitting it.

Verdict: APPROVE — Scope matches the title: CollectionOps and Rust emission consume HO contract refs; Go/Python get parallel CollectionOps fields and named contracts without claiming Rust emission changes there. The large bootstrap_generated*.rs churn is consistent with substrate/spec edits and is mechanical from the authored side.

Exploratory observations (optional): src/v3/compiler/src/emit/rust_target.rs (around 5957–5963) matches EmitError::MalformedTargetSyntax including the full detail string; that is slightly brittle if copy ever changes (TESTING.md discourages pinning prose), though the variant + declaration already carry the behavioral claim. src/v3/spec/rust.dag still carries a legacy filter: template with different placeholders than the contract; harmless for Rust if only the contract path is used, but easy to misread until the documented cleanup lands.

…nostic-ordering-carrier

# Conflicts:
#	src/v3/compiler/src/bootstrap_generated.rs
#	src/v3/compiler/tests/integration/parse_corpus_manifest.txt
@briansrls
briansrls merged commit 5a06f98 into main May 4, 2026
3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant