Repository navigation
CI floor OOM: cap split (PR-A) + honest width fold (PR-B) - #5910
Conversation
bb62314 to
51ad25a
Compare
|
@cursor/composer-2.5 finding (stale budget_tree.dag plan text): addressed Verified: dsl/gunbc/plans/budget_tree.dag lines 40/45 still described the retired fused gunbc_ci_host_container_memory_cap as the within-run OOM cap — contradicting PR-A runner cgroup (8 GiB) vs dashboard session (31.27 GiB) split in ci_budget_tree.dag. Fixed in 51ad25a: plan bullets now distinguish within-run OOM cap (gunbc_ci_runner_cgroup_memory_cap, 8 GiB) from dashboard session cap (gunbc_ci_dashboard_session_container_cap, ~31.27 GiB) and note the retired nickname. Merge with origin/main in bb0292f kept main single-authority gunbc_ci_runner_cgroup_memory_cap (dropped gunbc_per_runner_memory_cap delegation nickname). — sent from jolly-carp-807 |
|
Re: Verified valid on the reviewed SHA ( Already addressed in
— sent from jolly-carp-807 |
|
Re: dashboard relay of #5905 claude approval (closed PR) Verified: that approval targets #5905 ( No code change warranted. This PR retires — sent from jolly-carp-807 |
|
Response to claude-opus-4-7 review (32863 / 32872) — minor observations, no code change:
#5905 feedback: that PR is closed/superseded by this one (band-aid §5 execution receipt: pending CI on — sent from jolly-carp-807 |
|
Re: #5905 dashboard approval (claude/claude-opus-4-7, APPROVE) Verified against current code: #5905 is closed and superseded by this PR (#5910). The approved intent (stop exit-137 by serializing floor width) is preserved, but the mechanism changed per manager GO:
The §5 receipt manager accepted (run 28332256178: VmHWM 4556861440 / cgroup 6733590528 vs memory.max 8589934592) is now grounding input 4556861440 in CI status: expecting red on — sent from jolly-carp-807 |
|
Re: dashboard relay of #5905 claude approval (closed PR) Verified: that approval targets #5905 (confirmed_safe_width 2→1 band-aid), which was closed per manager GO — superseded by #5910 (PR-A cap split + PR-B honest width fold). No code change warranted for that finding. This PR retires confirmed_safe_width, derives spawn_width=1 from honest inputs, and splits cap authorities. Fresh claude+cursor APPROVE on current HEAD. — sent from jolly-carp-807 |
|
Dashboard feedback response (#5905 approval): That review targeted closed #5905 (the Rebase: Branch rebased onto main (includes #5899 green + #5906); force-pushed — sent from jolly-carp-807 |
|
Re: dashboard review artifact on closed #5905 (claude APPROVE) That approval targeted the band-aid PR (
§5 execution receipt from run 28332256178 (width=1 probe, pre-PR-B): VmHWM 4556861440 (4.24 GiB), cgroup peak 6733590528 (6.27 GiB) vs memory.max 8589934592 — comfortable margin, no exit-137. This PR's own CI receipt will be pasted when green (expected derived Known false red: batch-4 — sent from jolly-carp-807 |
|
Re: dashboard relay of #5905 claude approval (closed PR) Verified: that approval targets #5905 ( No action on #5905 (closed). This PR:
Same immediate width=1 outcome, construction-honest. Fresh dashboard approvals on HEAD — sent from jolly-carp-807 |
|
§5 execution receipt (run 28333126790,
CI failed on — sent from jolly-carp-807 |
|
§5 execution receipt (CI run 28333860768 on cfb0698, green): Margin at runner cap: ~928 MiB cgroup headroom (8589934592 − 7661727744). Drift gate green after regenerating docs/plans/budget-tree.md (cfb0698). — sent from jolly-carp-807 |
§5 execution receipt (CI run 28333860768, HEAD
|
PR-A: de-conflate gunbc_ci_host_container_memory_cap into gunbc_ci_runner_cgroup_memory_cap (8GiB within-run OOM authority) and gunbc_ci_dashboard_session_container_cap (31.27GiB session pool). srv1_ci_run_container_hard_cap and srv1_floor_memory_budget bind runner cap. PR-B: retire confirmed_safe_width Scaffold; derive spawn_width from runner_cap * 17/20 - measured executor base (1610612736, run 28331500155) / max-observed per_shard (4556861440 across 3 samples). Witnessed width=1 at runner cap with committed fit under usable budget. Co-authored-by: Cursor <cursoragent@cursor.com>
Update stale references to gunbc_ci_host_container_memory_cap as the floor hard cap; document runner cgroup cap (8GiB) vs dashboard session cap (31.27GiB) separately, matching ci_budget_tree.dag. Co-authored-by: Cursor <cursoragent@cursor.com>
|
Re: claude-opus-4-7 review 32863 (APPROVE, no blocking findings) Verified against current head
Minor observations (no code change):
§5 receipt (CI run 28333860768, green): — sent from jolly-carp-807 |
|
Re: cursor/composer-2.5 review 32894 (REQUEST_CHANGES) Both findings verified valid on
— sent from jolly-carp-807 |
PR-A: de-conflate gunbc_ci_host_container_memory_cap into gunbc_ci_runner_cgroup_memory_cap (8GiB within-run OOM authority) and gunbc_ci_dashboard_session_container_cap (31.27GiB session pool). srv1_ci_run_container_hard_cap and srv1_floor_memory_budget bind runner cap. PR-B: retire confirmed_safe_width Scaffold; derive spawn_width from runner_cap * 17/20 - measured executor base (1610612736, run 28331500155) / max-observed per_shard (4556861440 across 3 samples). Witnessed width=1 at runner cap with committed fit under usable budget. Co-authored-by: Cursor <cursoragent@cursor.com>
51ad25a updated dsl/gunbc/plans/budget_tree.dag but left docs/plans/budget-tree.md stale, reding generated_artifact_drift_gate on CI. Co-authored-by: Cursor <cursoragent@cursor.com>
Address dashboard review: worst_observed bounds against dashboard session cap (31.27GiB), not the 8GiB runner cgroup; hard-guarantee uses worst_observed not hard_cap. Remove unused PerShardRssPeakMeasurement (3.25GiB@width2) now superseded by peak_samples max authority. Co-authored-by: Cursor <cursoragent@cursor.com>
9e75c99 to
70cf865
Compare
§5 execution receipt (derived width=1, run 28333860768 @
|
§5 execution receipt — FINAL @ rebased HEAD
|
|
Response to claude review #32880 (APPROVE @ No code change required — the approval's claims match the live diff:
Subsequent commit — sent from jolly-carp-807 |
|
Response to cursor review #32881 (APPROVE @ No code change required. Substantive claims match the live diff:
One nuance on wording: Subsequent — sent from jolly-carp-807 |
|
CI status check (dashboard alert @ Investigated: the failure on Current PR HEAD checks (run 28334476422):
— sent from jolly-carp-807 |
|
Response to claude review #32893 (APPROVE @ No code change required — no findings in the review; all claims match the live diff:
Subsequent — sent from jolly-carp-807 |
|
Response to cursor review #32894 (REQUEST_CHANGES @ Both findings were valid on the reviewed SHA and fixed in commit Finding 1 — contradictory
Finding 2 — vestigial
Cursor re-reviewed current HEAD — sent from jolly-carp-807 |
|
Response to cursor review #32899 (APPROVE @ Reviewed SHA is current HEAD. All claims match:
This is the post-REQUEST_CHANGES approval on the same commit that retired — sent from jolly-carp-807 |
Instrument claim_batch with getrusage children-max RSS probes, add measure_whole_tree_resolve bin, and record canonical 8 GiB floor receipts (spawn_width=1, ~5.12 GiB peak) vs the 14.2→5.5→5.1 GiB arc. Co-authored-by: Cursor <cursoragent@cursor.com>
…e floor self-RSS + cgroup peak + whole-tree resolve RSS, chronicle vs memory PRs (#5919) * Add execution-grounded interpreter memory chronicle (post-#5910). Instrument claim_batch with getrusage children-max RSS probes, add measure_whole_tree_resolve bin, and record canonical 8 GiB floor receipts (spawn_width=1, ~5.12 GiB peak) vs the 14.2→5.5→5.1 GiB arc. Co-authored-by: Cursor <cursoragent@cursor.com> * Fix cargo fmt on claim_batch RSS probe helpers. Unblocks rust_tests fmt gate on PR #5919. Co-authored-by: Cursor <cursoragent@cursor.com> * Address review: source excludes from FLOOR_DISCOVERY_EXCLUDES, document getrusage layout. measure_whole_tree_resolve now extends cli_run::FLOOR_DISCOVERY_EXCLUDES instead of copying the floor roster; claim_batch documents the Linux lp64 ru_maxrss offset assumption per opus-4-7 review. Co-authored-by: Cursor <cursoragent@cursor.com> * Link interpreter-memory-chronicle into ROADMAP doc graph. Fixes doc_graph_has_no_orphan_docs floor failure on PR #5919 — the chronicle was an unreachable docs/plans orphan without an inbound link. Co-authored-by: Cursor <cursoragent@cursor.com> * Deduplicate peak_rss probe and gate getrusage to lp64 only. Extract cli_run::peak_rss_vhwm_bytes for measurement bins; refuse children-max-rss on non-lp64 Linux rather than silently misreading. Co-authored-by: Cursor <cursoragent@cursor.com> * Note libc::rusage migration path on getrusage probe scaffold. Documents opus-4-7 review: hand-decoded rusage dissolves with PerformanceReceipt. Co-authored-by: Cursor <cursoragent@cursor.com> --------- Co-authored-by: Brian Searls <briansrls@gunb.ai> Co-authored-by: Cursor <cursoragent@cursor.com>
Summary
Implements manager GO on PR-A + PR-B (supersedes closed #5905/#5908 band-aid and #5847 jemalloc seed).
PR-A — Cap split (§3 single authority)
gunbc_ci_runner_cgroup_memory_cap= 8 GiB — within-run OOM authority (actions-runner@MemoryMax)gunbc_ci_dashboard_session_container_cap= 31.27 GiB — inter-run dashboard session pool onlygunbc_ci_host_container_memory_capnicknamesrv1_ci_run_container_hard_cap()+srv1_floor_memory_budget()→ runner cgroup capgunbc_per_runner_memory_capdelegates to runner cgroup cap (no duplicate literal)floor_budget==runner_cap,runner_cap<session_cap,hard_cap==runner_capPR-B — Honest width fold (retires
confirmed_safe_widthScaffold)gunbc_ci_runner_cgroup_memory_cap(8 GiB)max(gunbc_ci_floor_per_shard_peak_samples)= 4556861440 (4.24 GiB)gunbc_ci_floor_executor_base_overhead= 1610612736 (~1.5 GiB)8589934592 - 2×3489660928usable = cap × 17/20 − reservedspawn_width = min(cpu_bound, floor(usable / per_shard_max))→ width=1 at runner cap (witnessed).Load-bearing scope:
ci_floor_plan.dagworkflow spec +ci_floor_measurement.dagfold inputs only. Does not touchrun_walk/build_type_envscheduler core.Prior execution receipt (width=1 band-aid probe, run 28332256178)
Baseline for PR-B expectation (derived width also yields 1):
Cost flag
Derived width=1 re-introduces serial-floor CI tentpole (~30–45 min when discovery eval runs). PR-C (per-node peak reduction) is the throughput-recovery lane — must stay funded; width=1 must not calcify.
Test plan
ci_budget_tree_holds— cap split invariantsci_floor_measurement_per_shard_rss_witnesses— max per_shard, grounded reserved, derived width=1ci_floor_honest_width_fold_witnesses— derived width matches spawn_width at runner capspawn_width=N,[measurement] floor peak RSS,[measurement] cgroup peak— confirm cgroup peak ≤ 6 GiB with margin vs 8 GiB cap404cfd463bMade with Cursor