Repository navigation
Affected-set CI scoping: Move C (exclusion model) + Move B (execution corpus) - #5962
Conversation
…ngs in ci_layer_roots.dag (§3 single authority) - Add `witness_exclusion_substrings: List<String>` to `dsl/gunbc/ci_layer_roots.dag` — single authority for path-substring exclusions - Add `exclude_substrings: List<String>` field to `RunnableDiscoveryBatch` in `dsl/std/realization_schedule.dag`; update all pattern-match sites and `runnable_eq` - Wire `witness_exclusion_substrings` into `corpus_runnable()` in `ci_floor_plan.dag` via the new field - Thread `exclude_substrings` through `DiscoveryCorpusOptions`, `discover_floor_corpus_rows`, `claim_executor`, `claim_batch`, and test call sites; plan-driven paths now read from the model value, not the Rust constant - Mark `FLOOR_DISCOVERY_EXCLUDES` SCAFFOLD (dissolves when all callers use the model-derived path) Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
|
Two fixes landed in d27f0cf in response to the §3/§5 checks: 1. In Remaining live reads of the constant: 2. Discriminating probe — §5 by-execution proof that model is the authority Added The two authorities now intentionally differ on one entry. CI green-by-execution is the proof: the probe is excluded (→ CI passes) only because the model is doing the exclusion. If the constant were the authority (it lacks the sentinel), the probe would be discovered and CI would fail red. Removing — sent from merry-fox-653 |
… pass - floor_skip_discovery_host_test.rs: DiscoveryCorpusOptions struct literal missing new exclude_substrings field — use ..Default::default() to pick up the FLOOR_DISCOVERY_EXCLUDES-backed default - claim_executor.rs: vec![] in None arm of exclude_substrings match was ambiguous — change to Vec::new() so rustc can infer the String element type - cargo fmt --all pass (claim_batch.rs, cli_run.rs, pipeline.rs, wet_hermetic) Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…ed parallel width - Move 5 execution witnesses (self_host/fixed_point, self_host/candidate_generation, self_host/compiler_closure_emit, fold_assembly/round_trip, complexity_gate/comprep_bridge) into src/v2/test/claim/execution/ with updated module declarations - Add execution_corpus_node to ci_floor_plan.dag: RunnableDiscoveryBatch with discovery_scope_dirs scoped to execution/ dir and spawn_width_cap derived from gunbc_ci_runner_cgroup_memory_cap / 2 GiB per witness reservation - Extend RunnableDiscoveryBatch type + Rust BatchUnit/DiscoveryBatch with discovery_scope_dirs and spawn_width_cap; apply cap at run_batch_unit level - Add ci_exclusion_proof_test.dag to FLOOR_DISCOVERY_EXCLUDES (Rust mirror) to prevent regression on claim_batch pre-push path (sentinel already in .dag model) - Update floor_nodes, floor_resource_anchor_node, runnable_for_node, floor_data_dependencies and the RunnableDiscoveryBatch pattern in floor_batch_heavy_gate_resolve_count Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
|
Addressing the cursor/composer-2.5 review findings: Finding 1 (sentinel not in FLOOR_DISCOVERY_EXCLUDES): Valid. Added Finding 2 (fields without consumers): The reviewer was reading the WIP commit where Finding 3 ( — sent from merry-fox-653 |
…tch_unit BatchUnit::Discovery is matched on an owned BatchUnit, so spawn_width_cap is usize not &usize — drop the * dereference at the cap application site. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- execution_corpus_spawn_width: wrap gunbc_ci_runner_cgroup_memory_cap with byte_size_count() before dividing by Int (ByteSize / Int is not defined) - inert_lens_hygiene_tests / construction_justification_hygiene_tests: add FLOOR_DISCOVERY_EXCLUDES to use super:: imports — test functions reference it directly after discover_floor_corpus_rows signature added the param Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…VERY_EXCLUDES import Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…st enrolled rows The execution corpus witnesses (src/v2/test/claim/execution/) are excluded from the main corpus rows, but path_imports already contains their import graphs (populated before the exclusion continue). Extend the inert_lens_modules seed to include ALL *_test.dag paths found in the source tree so that execution-corpus witnesses still count for lens coverage — matching the pre-Move-B behavior. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…lexity_gate/, drop stale execution copy
main renamed comprep_pipeline_bridge{,_test}.dag -> source_bridged_pipeline_bridge{,_test}.dag
and kept the test in complexity_gate/. My branch had moved the comprep test to execution/
but that source module no longer exists post-rename. Resolution: take main's file in
complexity_gate/ and remove the stale execution/comprep_pipeline_bridge_test.dag.
Execution corpus now has 4 witnesses (the 3 self_host + fold_assembly_round_trip).
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…ion/merry-fox-653
…_cap enforcement Finding 1: execution_corpus_spawn_width() divided the full 8 GiB runner cap by the per-witness reservation (2 GiB = 4), but both corpora run concurrently in batch-2 with no ResourceDependsOn between them. Delegates to gunbc_ci_execution_corpus_available_width() in ci_floor_measurement.dag which subtracts the floor corpus worst-case shard peak (~4.24 GiB) and executor base overhead (~1.5 GiB) first — correct width under current measurements is 1. Finding 2: DiscoveryCorpusOptions.spawn_width_cap documented as capping in-corpus parallelism but run_discovery_corpus_with_options never enforced it — only claim_executor.rs applied the cap before calling. Now enforced at the top of run_discovery_corpus_with_options so any caller gets the cap regardless. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…ery_corpus_with_options run_discovery_corpus_with_options now self-enforces options.spawn_width_cap (previous commit). Thread spawn_width_cap through run_discovery_batch_node to DiscoveryCorpusOptions.spawn_width_cap instead of pre-capping spawn_width in run_batch_unit. Eliminates the duplicate min() that was the §2/§3 second representation flagged in cursor review finding 2. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…awn_width_cap) The function was at the clippy limit (7 args) before this PR. Adding spawn_width_cap as an 8th arg to thread the cap through to DiscoveryCorpusOptions.spawn_width_cap is the correct §2/§3 single-authority fix; suppressing the lint on this internal function is the right response rather than bundling unrelated args. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…y formula Two issues addressed: - execution_corpus_node co-ran with floor_corpus_node in batch-2, causing OOM (exit code 137 in CI). Fix: add ResourceDependsOn(floor → execution) so execution runs after floor finishes. - runnable_excludes_corpus_co_residence gates (emit-host, regen) only waited for floor_corpus_node; can now co-run with execution corpus. Fix: add execution_corpus_node as second source in floor_corpus_exclusion_resource_edges so those gates wait for both. - gunbc_ci_execution_corpus_available_width: removed floor_shard_peak subtraction; execution now runs sequentially after floor, so full (runner_cap - executor_base) is available → width = floor((8GiB - 1.5GiB) / 2GiB) = 3. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
|
Both findings are valid — fixed in 57a5487. Finding 1 (OOM / co-resident memory math): The exit-code-137 CI failure confirms it. Finding 2 (gate resource exclusion gap): — sent from merry-fox-653 |
…plit witness_source_ingest_serialized_after_corpus used skip(n:2) to find the ingest layer, but execution_corpus_node (added in Move B) occupies batch 2 as it depends on floor_corpus_node. Ingest and other heavy gates now land in batch 3 (they depend on BOTH corpora). Update skip to n:3. Also fix the stale BoolWitness.entry in emit_ingest_typescript_same_language_round_trip_test.dag which still pointed at the old manual/ path after the file was moved to execution/. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…rkaround) Conflict resolution: gunbc_ci_execution_corpus_available_width — took main's single-step subtraction (runner_cap - executor_overhead). The Move A branch's two-step form (subtract floor_per_shard_rss first) was authored pre-#5962; after #5962 merged, execution corpus runs AFTER the floor via ResourceDependsOn, so the floor reservation is no longer needed at execution-corpus width time. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Summary
Three moves to scope the CI floor's discovery so heavy execution witnesses run in a parallel-capped pool instead of serializing in the main corpus:
Move C (done): Migrate `FLOOR_DISCOVERY_EXCLUDES` to the `.dag` model as `witness_exclusion_substrings` in `dsl/gunbc/ci_layer_roots.dag` — the single authority for path exclusions. The Rust constant is now a named SCAFFOLD with a dissolution trigger. Added `ci_exclusion_proof_test.dag` (always-false sentinel) to prove the plan's exclusion path actually gates.
Move B (done): Split all 21 execution-class witnesses (classified by behavior: they execute programs/the compiler through the v2 pipeline) into `src/v2/test/claim/execution/` and expose them as a separate `execution_corpus_node` with `spawn_width_cap` derived from available headroom. Extended `RunnableDiscoveryBatch` with `exclude_substrings`, `discovery_scope_dirs`, and `spawn_width_cap` fields; updated the generated `std_realization_schedule.rs` to match.
Move A (tracked follow-on): Affected-set skip predicate — when the diff ∩ witness import closure is empty, skip the witness entirely. Separate PR after this merges.
Execution corpus scheduling and memory provisioning
`execution_corpus_node` runs after `floor_corpus_node` (not co-resident): a `ResourceDependsOn(floor → execution)` edge is added in `floor_corpus_exclusion_resource_edges`. Heavy gates (emit_host, regen_verify, source_root_ingest) depend on both corpus nodes, so they land in batch 3.
Batch structure:
`gunbc_ci_execution_corpus_available_width` computes headroom for execution running alone (no floor co-residency): (8589934592 − 1610612736) / 2147483648 = spawn_width_cap = 3.
Effective width today = min(plan_spawn_width=1, spawn_width_cap=3) = 1. `claim_executor` receives `spawn_width = gunbc_ci_floor_spawn_width` which currently returns the floor-only derived width (1). Wiring `gunbc_ci_floor_spawn_width` to `max(floor_width, exec_width)` — so the plan permits width 3 during the execution phase — is tracked as follow-up: plan-width binding for execution-corpus width-3 (node://adhoc-e188369a-643). That follow-up also renames `gunbc_ci_floor_spawn_width → gunbc_ci_plan_spawn_width` (misnomer: it returns the cross-corpus max, not the floor's width) and replaces `witness_floor_derived_width_at_runner_cap` with a real per-corpus memory-safety guard.
Move B standalone value: (a) enabling Move A's affected-set skipping on an isolated corpus node; (b) honest fail-closed provisioning with named dissolution trigger; (c) OOM prevention from sequential scheduling (no exit-137).
Witnesses moved to execution/
4 already there (self_host/, fold_assembly/) + 17 newly moved from manual/:
`add_arrow_eval_by_execution`, `bind_eval_by_execution`, `bind_round_trip`, `branch_eval_by_execution`, `branch_if_then_else_round_trip`, `dag_add_emit_round_trip`, `effect_io_eval_by_execution`, `emit_ingest_grammar_relation_round_trip`, `emit_ingest_python_same_language_round_trip`, `emit_ingest_type_decl_round_trip`, `emit_ingest_typescript_same_language_round_trip`, `loop_eval_by_execution`, `loop_round_trip`, `match_bool_round_trip`, `typescript_closure_emit_by_execution`, `typescript_enum_union_emit_by_execution`, `typescript_import_emit_by_execution`
Module paths updated from `v2.test.manual.` → `v2.test.execution.`. Importers in manual/ (bind_emit, branch_if_then_else_emit, loop_emit, match_bool_emit) and in execution/ (self_host_candidate_generation) updated.
Test plan