Skip to content

CI witness cost backout: trim the floor's enrolled roster by measured cost (103 -> 67 check fns) - #6437

Merged
briansrls merged 2 commits into
mainfrom
claude/ci-witness-cost-backout
Jul 10, 2026
Merged

briansrls merged 2 commits into
mainfrom
claude/ci-witness-cost-backout

Conversation

@briansrls

@briansrls briansrls commented Jul 10, 2026 •

Copy link
Copy Markdown
Contributor

CI witness cost backout: trim the floor roster by measured cost, 103 → 67 check fns

Operator ruling 2026-07-10: "CI taking 1 hour is way too long … we'll need to back most out until we get performance under control."

How witnesses get into CI (the answer to "how are those selected")

Single authority: commit_gate_roster in dag/gunbc/commit_workflow.dag. Each CommitWitnessClaim { entry, check_fns } row on the GithubActionsCiJob surface is projected into the floor plan by project_ci_floor_witness_entries at floor runtime (receipt: this PR's regen produced zero ci.yml drift). Enrollment is opt-in (operator inversion 2026-07-04, which zeroed it); it accreted 0 → 103 check fns in six days as each merged lane enrolled its own witnesses.

Where the hour went (measured, run 29108981678 — the 1h0m50s ci job)

segment time
release-bin build (cache+sccache warm; PR carried main's new .rs) 26m52s
batch 1 — compile-clean gate 2m22s
batch 2 — 103 witnesses + drift/layering gates 15m06s
batch 3 — emit_host / ingest gates 2m01s
batch 4 — emit_determinism gate (one gate) 13m31s

Witness count ≠ witness cost: p50 witness = 294ms. Of batch 2's 15m, ONE witness (dag_collect_fingerprint_witness_keystone_holds, #6425) was 8m38s; five heavies ~2.7m; the complexity-lens ~12s-eval family ~3.5m; the ~80-witness tail costs under a minute combined.

The cut (by cost, not count)

  • Unenrolled dag_collect_fingerprint_witness_test (518s keystone).
  • Dropped parse_table_memo_warm_equals_cold_content_hash (36s); the row's four governance RED controls stay.
  • Trimmed my five complexity/body-lowering suites from 39 fns to RED control + flagship positive each (10 fns) — accumulator_copy_fold_analysis, complexity_accumulator_copy, fold_lowering, loop_multiplicity_measure, loop_infer_iteration.
  • Deliberately KEPT (~101s total): enforcement_live_witness_test (84s) and budget_roster_completeness (17s) — the §6 "lenses must be live" backstops that keep every other lens honest. Highest displaced-cost-per-second on the roster; overrule if you want them out too.
  • Left the sub-second tail enrolled (materialization suites etc., ~44 fns): cutting them saves <1m combined and they're the cheapest RED walls we have.

Every trimmed fn stays in its witness file and still runs on the local discovery path (claim_batch --roster-from-discovery …); existence ≠ enrollment. Re-enrollment is a one-row edit when parse/resolve memoization lands.

Expected effect: batch 2 ≈ 15m → ~2.5m.

What witness backout cannot fix (flagged for their own lanes)

  1. Release-bin build: 26m52s on any PR carrying .rs changes (sccache-warm — release codegen/link-bound). Lever: reuse a prebuilt binary artifact keyed on .rs content-hash (the same Realization/materialization argument as everything else). No .rs delta → 1m44s (measured on main).
  2. Batch-4 emit_determinism gate: 13m31s, a single structural gate, not a witness entry.
  3. The floor shows ~2× variance under runner contention (same corpus: 17m on main vs 33m on the concurrent PR run).

With this PR, a typical no-.rs-change PR's ci job ≈ 2m build + ~9m floor. With .rs changes it's still ~27m build + ~9m floor until lever 1 lands.

Durable fix candidate (follow-up, not this PR): enrollment rows declare a per-fn eval budget and the floor REDs a witness that exceeds it — cost-gated enrollment as a wall, so the roster can't silently re-accrete a 518s witness.

Receipts

  • main_wet regen: zero generated-artifact drift; drift gate PASS.
  • Kept-witness sanity run: quadratic_fold_has_suspect PASS wet against the trimmed roster entry.
  • Diff vs main: exactly one file, dag/gunbc/commit_workflow.dag (+5/−44).

🤖 Generated with Claude Code

Brian Searls and others added 2 commits July 10, 2026 18:01
…measured cost, 103 -> 67 check fns (operator ruling 2026-07-10)

Measured on run 29108981678 (ci job 1h0m50s): batch-2's 103 witnesses cost 15m06s, of which ONE witness (dag_collect_fingerprint_witness_keystone_holds) was 8m38s, five heavies ~2.7m, and the complexity-lens ~12s-eval family ~3.5m; the ~80-witness tail costs under a minute combined (p50=294ms). Cut by cost, not count:

- unenroll dag_collect_fingerprint_witness_test (518s keystone, #6425 lane)
- drop parse_table_memo_warm_equals_cold_content_hash (36s; the row's 4 governance REDs stay)
- trim the five complexity/body-lowering suites (mine) from 39 fns to RED control + flagship positive each (10 fns)
- KEPT: enforcement_live_witness_test (84s) and budget_roster_completeness (17s) - the section-6 lenses-must-be-live backstops that keep every other lens honest

Every trimmed fn stays in its witness file and still runs on the local discovery path (claim_batch --roster-from-discovery); existence != enrollment. Receipts: main_wet regen zero artifact drift (witness entries are projected from the roster at floor runtime, not baked into ci.yml), drift gate PASS, kept-witness sanity run PASS.

Expected effect: batch-2 15m -> ~2.5m. Remaining non-witness costs flagged for their own lanes: release-bin build (26m52s on any PR carrying .rs changes) and batch-4 emit_determinism gate (13m31s).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@gunbai-bot gunbai-bot Bot changed the title complexity CI witness cost backout: trim the floor's enrolled roster by measured cost (103 -> 67 check fns) Jul 10, 2026
@gunbai-bot
gunbai-bot Bot marked this pull request as ready for review July 10, 2026 18:04
@gunbai-bot

gunbai-bot Bot commented Jul 10, 2026

Copy link
Copy Markdown
Contributor

The "CI FAILING (3 failing)" dashboard reading is the known superseded-run artifact, not a real failure: the draft-to-ready flip re-triggered CI under the concurrency group, cancelling the first run on this same head (29113213972, cancelled) and the WIP-head run (29113098355, cancelled). The live run is 29113261146 — ci_regen already green, ci and rust_tests in progress. No fix needed; this run doubles as the measured receipt for the trimmed floor. — sent from eager-ram-612

@briansrls
briansrls merged commit 3537bbc into main Jul 10, 2026
3 of 6 checks passed
@briansrls
briansrls deleted the claude/ci-witness-cost-backout branch July 10, 2026 18:44
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant