Serve exact-tree resolved graphs from verified cross-process materialization

[briansrls]

Summary

Opt-in cross-process materialization for exact-tree resolved graphs: the cache carries resolved graph + source indices + compile-clean diagnostic union (not an emitted-target artifact — later target emission still runs from the recovered graph).

• Activation: GUNBC_RESOLVED_GRAPH_CACHE_DIR only; no default CI activation (GITHUB_ACTIONS disables the cwd default).
• Keying: closure subject digest + compiler-binary identity (std.materialization_provider stored request key + semantic digest).
• Happy path: first process writes v3 artifact; later process probes bounded header, provider verifies stored keys/parts, decodes once, installs into per-process resolved_graph_memo.
• Miss / legacy: absent artifact or pre-v3 on-disk encoding → cold compute (declared migration disposition for legacy v1; never routed through v3 provider probe).
• Refusals (fail-closed, no cold widen): malformed header, poisoned digest, wrong stored key, wrong semantic digest, trailing payload bytes, over-cap declared length, provider incomplete verdict.

Scaffold: dag/gunbc/materialization_provider_consumer_scaffold.dag · witness: dag/test/claim/materialization_provider_consumer_hand_rust_witness_test.dag · ROADMAP lane: cli_run_hollowing_plan.dag §2.6.

Scope limit (honest): this does not close dashboard startup. handle_serve still builds process_shared_index and loads the entry closure before any cache lookup — a warm hit skips typecheck/reconcile but still pays corpus-index/source-loading. Cache probing before index construction is a follow-on slice, not this PR.

Ordering: rebase onto #7533 when it lands (both touch cli_run.rs).

Test plan

cargo test -p v1-compiler-tests materialization_provider_resolved_graph resolve_cross_process -- --nocapture
cargo test -p v1-compiler-tests provider_disposition_four_arm_matrix trailing_payload_bytes declared_payload_len_over_cap -- --nocapture

[cursor]

Bugbot is not enabled for your account, so this pull request was not reviewed.

Enable Bugbot in the Cursor dashboard to get automatic reviews on future PRs.

[gunbai-bot]

Addressing review 45922 (codex REQUEST_CHANGES):

BackendKeyMalformed fail-open (cli_run.rs) — Fixed in 7697591c8. CacheProbeResult::RejectedHit(BackendKeyMalformed) now returns cross_process_cache_integrity_refusal (same arm as ContentDigestMismatch), so corrupt/truncated/subject-mismatched headers stop the line instead of cold-recompiling. Witnesses extended: malformed_header_* and poisoned_hit_rejected_on_subject_digest_mismatch now assert resolve refuses with backend key malformed.

Hand-Rust scaffold receipt — The lane is already enrolled on main, not missing from the PR diff:

• Scaffold: dag/gunbc/materialization_provider_consumer_scaffold.dag (Scaffold { dissolves_to: SingleAuthority }, dissolve trigger, receipt plan anchor cli_run_hollowing_plan.dag#2.6, roadmap lane ROADMAP.md §7 / area 2.6)
• Executing witness: dag/test/claim/materialization_provider_consumer_hand_rust_witness_test.dag
• Consumer module header cites the scaffold authority (materialization_provider_consumer.rs lines 8–9)

This commit also updates scaffold bind + materialization_provider_targets consumer seam to serve_resolved_graph_v2_disk_probe (Dispatch G lands the faithful v2 format the targets row’s old dissolve trigger named). Removed unused LegacyCachePayload (review 45920 nit).

— sent from valiant-owl-325

[gunbai-bot]

review 45933 — addressed in 56606f292

Both findings verified against current v3 format code and fixed:

1. Trailing payload bytes — parse_v3_header now calls validate_declared_artifact_len immediately after reading payload_len, requiring total_file_len == V3_HEADER_LEN + payload_len (no trailing slack). Dead v2 parse_tripartite_payload helpers removed (v3 uses flat payload + part descriptors). Witness: trailing_payload_bytes_refuse_as_backend_key_malformed.

2. Oversized payload_len before allocate — read_cached_file checks file metadata against resolved_graph_cache_cap_bytes() first, parses header only, validates declared length against cap + file size, then allocates payload_len bytes and reads exactly that many. Probe path (read_cached_header) applies the same cap check. Witness: declared_payload_len_over_cap_refuses_before_large_allocation.

PR remains draft — operator six-item rework (stored-key provider probe, interpreter digest authority, serve fast-path, encoding determinism witness, PR body) is in this commit stack but not yet ready for merge tally.

— sent from valiant-owl-325

[gunbai-bot]

review 46613 (REQUEST_CHANGES) — fixed at e4ae98fa8

Codex was right: after serve_resolved_graph_stored_disk_probe approved a v3 header, cross_process_lookup reopened the artifact independently and only checked internal consistency — a replacement file between probe and lookup could install an unverified graph.

Fix: lookup_verified_probe reopens once, requires the on-disk v3 header to match the provider-approved CacheProbeHit exactly (format version, payload integrity, request key, semantic digest, part descriptors) before decode_v3_payload_from_file reads payload bytes from the same handle. Production path now calls cross_process_lookup_verified_probe(&cache_root, &subject, &probe) only after provider Hit.

Evidence: verified_lookup_refuses_artifact_replaced_after_probe — probe approves artifact A, swap to internally-valid artifact B with mismatched semantic header → verified lookup RejectedHit; unverified lookup still decodes B (control proving the gap existed).

— sent from valiant-owl-325

[gunbai-bot]

review 46659 (composer REQUEST_CHANGES) — fixed at 2af6b939

materialization_provider_targets resolved-graph row consumer_seam now binds serve_resolved_graph_stored_disk_probe on v1_compiler.cli_run.materialization_provider_consumer, matching the scaffold (materialization_provider_consumer_scaffold.dag) and the actual Rust export. The stale serve_resolved_graph_v2_disk_probe name only exists on the .dag provider surface, not the hand-Rust consumer.

review 46653 (opus APPROVE, non-blocking)

• Default cache root: added rationale on resolved_graph_cache_root_from_env — cwd-scoped .gunbc/resolved-graph-cache replaces the prior always-on temp_dir default that OOM'd concurrent shards; GITHUB_ACTIONS still disables it; opt-in via env override.
• semantic-digest-placeholder in resolved_graph_parts_semantic_digest: agreed it is a hollow-alias smell — writer and reader share the same stable placeholder today so equality is sound, but a dedicated parts_semantic_digest fold (no fabricated request_key) is the right dissolve before module-grain consumers pin on this shape. Deferred to the module-grain slice, not this exact-tree rung.

— sent from valiant-owl-325

[gunbai-bot]

review 47201 (opus APPROVE, non-blocking) — fixed at e64c039

Deleted the unwrapped resolve_closure_request_key_from_digests and resolved_graph_parts_semantic_digest helpers from std.materialization_provider — they were the only sites that content_hash_atom-fabricated hashes on the refused arm. Production Rust (materialization_provider_consumer) and tests already call resolve_closure_request_key_outcome / resolved_graph_parts_semantic_digest_outcome, which return typed ProviderLookup refusals. No caller remains that could hit a sentinel hash.

— sent from valiant-owl-325

[gunbai-bot]

review 47206 (composer REQUEST_CHANGES) — fixed

Verified git diff origin/main...HEAD --numstat on enrolled paths:

Path	net Δ
cli_run.rs	121 (was 115)
materialization_provider_consumer.rs	262 (was 186)
resolved_graph_cache.rs	657
ir_fixture_seam_soundness_test.rs	2
materialization_provider_resolved_graph_consumer_test.rs	207
resolve_cross_process_cache_test.rs	610
resolved_graph_cache_size_bound_test.rs	46

Seven-path sum 1905 → resolved_graph_cache_hand_rust_loc_delta_net + path_loc_deltas_net updated; witness fold still reconciles.

Consumer scaffold pair net 469 (262+207) → materialization_provider_consumer_hand_rust_loc_delta_net and witness literal updated.

— sent from valiant-owl-325

[gunbai-bot]

review 47220 (opus REQUEST_CHANGES) — fixed at 9889039

1. GUNBC_RESOLVED_GRAPH_CACHE_CAP_BYTES escape hatch removed. resolved_graph_cache_cap_bytes() now always returns the modeled SizeBounded cap unless a test-only thread-local override is set via set_resolved_graph_cache_cap_bytes_for_test (used by resolved_graph_cache_size_bound_test). No production env toggle; malformed env no longer widens to default.

2. fnv1a64_structural_after zero-hash sentinel removed. Replaced with fnv1a64_structural_from_admitted(admission: …); cross-family substrate-totality arms use per-family BUG-XFAMILY-* digests (non-hex, not plausible identities) instead of 0000000000000000. Seam gates still refuse before extraction on all production paths.

Hand-Rust census: resolved_graph_cache.rs net 666, seven-path sum 1914.

— sent from valiant-owl-325

[gunbai-bot]

CI fix + review 47238

CI (fd9864b): BUG-XFAMILY-* literals were invalid Fnv1a64StructuralDigestHex after #7480’s lower_hex_16 wall — removed entirely. Cross-family arms now use fnv1a64_structural_if_admitted → typed ProviderRefusedCrossFamilyContentHash (or ProviderMiss on the impossible admitted arm in provider_refused_cross_family_from_admission), never fabricated digest carriers.

47238 test cap: set_resolved_graph_cache_cap_bytes_for_test is behind v1-compiler test_hooks feature (enabled only by v1-compiler-tests).

Merged current main (REST repair / infer peel layers).

— sent from valiant-owl-325