Skip to content

Tune falsifier discovery-batch memory envelope to prevent OOM at width 4 - #6396

Merged
briansrls merged 1 commit into
mainfrom
session/nimble-newt-334
Jul 9, 2026
Merged

briansrls merged 1 commit into
mainfrom
session/nimble-newt-334

Conversation

@gunbai-bot

@gunbai-bot gunbai-bot Bot commented Jul 9, 2026 •

Copy link
Copy Markdown
Contributor

Summary

The affected-set falsifier's first post-#6393 run completed plan setup but OOM-killed (exit 137) ~16min into the cold full-corpus discovery batch on srv1 (spawn_width=4, live runner cap ~25.7GiB, run 28993330753). Floor gate per-shard samples (~4.25GiB) underestimated cold scan-dir discovery load.

This adds a falsifier-specific per-shard authority (gunbc_falsifier_per_shard_peak_rss_bytes, max of local width-5 calibration and CI cap÷width floor from the OOM run) and routes gunbc_falsifier_batches + gunbc_falsifier_plan_spawn_width through gunbc_falsifier_memory_derived_width. At the live 24GiB cap that backs spawn width off from 4 → 3 while keeping committed demand inside the 17/20 usable margin (~19.5GiB committed vs ~20.4GiB margin).

Test plan

  • claim_batch --entry dag/test/claim/ci_floor_measurement_per_shard_test.dag --function ci_floor_measurement_per_shard_rss_witnesses — PASS
  • claim_batch --entry src/v2/test/claim/ci_floor_plan_witness_test.dag --function falsifier_plan_spawn_width_witnesses — PASS

@gunbai-bot gunbai-bot Bot changed the title affected-set-falsifier OOM: first-ever complete plan setup (post-#6393) but cold full-corpus run killed exit 137 (~16min in, run 28993330753, spawn_width=4, runner budget ~25.7GB; local cold run peaked 18.2GB@width5 under 33.5GB). Tune falsifier discovery-batch resource envelope (per-shard predicted Tune falsifier discovery-batch memory envelope to prevent OOM at width 4 Jul 9, 2026
@gunbai-bot
gunbai-bot Bot marked this pull request as ready for review July 9, 2026 05:12
@gunbai-bot

gunbai-bot Bot commented Jul 9, 2026

Copy link
Copy Markdown
Contributor Author

Verified both dashboard review findings against 3e7dd37 — no code changes required.

claude (36426) / cursor (36431): Confirmed in tree:

  • gunbc_falsifier_per_shard_peak_samples → _max → _rss_bytes() with grounded Terminal receipt (local 3.9GiB/shard @ width 5; CI OOM floor 6GiB/shard from run 28993330753).
  • Single width authority: gunbc_falsifier_memory_derived_width feeds both gunbc_falsifier_spawn_width_from_live_cap() (plan-time spawn_width_cap) and gunbc_falsifier_plan_spawn_width() (runtime eval) → width 3 at live 24GiB cap (CI floor width 4).
  • Discriminating witnesses green locally: falsifier_plan_spawn_width_witnesses, ci_floor_measurement_per_shard_rss_witnesses.

Merge readiness @ 3e7dd37: 2/2 dashboard approvals (claude, cursor), 0 REQUEST_CHANGES, mergeable=CLEAN, CI green (ci / ci_regen / rust_tests all pass on run 28995715415). Awaiting operator merge per standing policy.

— sent from nimble-newt-334

@briansrls
briansrls merged commit a892619 into main Jul 9, 2026
4 of 6 checks passed
@briansrls
briansrls deleted the session/nimble-newt-334 branch July 9, 2026 06:05
briansrls added a commit that referenced this pull request Jul 9, 2026
…tants

#6396 landed falsifier-specific per-shard samples (max 6.44GB) and its own
width authority; this branch re-measured executor base overhead to 8.91GB.
Merged semantics: the memory formula at the live 24GiB cap now derives 2
(not #6396's 3, computed against the old 1.5GiB base) - renamed/updated
witness_falsifier_derived_width_at_live_cap_is_three -> _is_two; the shipped
falsifier width stays 1 (cpu_bound pins shard_count: 1), so the runtime
witnesses hold unchanged. Note text updated to the post-merge numbers.
Verified by execution: measurement, plan, budget-tree, and ci_spec witness
files all PASS on the merged tree.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant