fix(ci): slice-4 ejections — kernel mock-proc memory leak + O(n^2) launchctl pattern (t_b303235e) - #1183
Conversation
…unchctl pattern (t_b303235e) Two independent causes of 'Run tests slice 4/8' failing in MQ groups: 1. r-1060-139 / r-1078-139 (runner "shutdown signal", exit 143): the only unfinished file was tests/tools/test_local_env_blocklist.py. Its execute_code composition test returned a MagicMock proc into the always-on session kernel; _stderr_reader's read1() never returns b"", so the daemon thread spins forever recording mock calls. Reproduced in a linux py3.11 container with the stderr reader delayed 2s: 45 GB RSS after 9 min, main thread parked until the 300s cell deadline -> OOM on hosted runners. Fix: abort the fake spawn after env capture (same pattern as test_code_execution_modes.TestPythonPathComposition). 2. r-1078-19e (per-file ceiling): test_benign_segment_scaling_benchmark took 170s on an idle container. cProfile: 98% in one DANGEROUS_PATTERNS entry, the unanchored double-lookahead launchctl pattern, re-scanned at every offset (O(n^2)). Anchor with \A — a match at any offset implies one at offset 0, so detection is unchanged. Verified (container, py3.11, uv sync --locked): both files 20/20 green in isolation (10 with forced reader race); benchmark 170s -> 1.36s; test_approval + test_execute_code_surfaces_blocks + flag detection 197 passed; launchctl positive/negative cases unchanged.
|
🤖 merged-by: daedalus · lane: unspecified · gate: BYPASS: FleetReview paused (FR-paused bypass per card t_b303235e) · why: CI green on 504aeaa (all 16 slices + All required checks). Test-only change plus \A anchor on one regex. Fixes the slice-4 MQ ejections. |
|
Apollo 19:0x PT (Ace 19:02 'do all of it'): jumped to the front of the merge queue. #1183 fixes the slice-4/8 ejector class (kernel MagicMock leak OOM + O(n^2) launchctl regex) that ejected 3 groups in 5h; #1060/#918 are the last PRs of the 09-25 boil-ocean sweep. Batched in one jump so in-flight groups restart once. |
|
🤖 merged-by: apollo · lane: boil-ocean · gate: BYPASS: FR PAUSED by Ace ruling 2026-09-22 (state/fleetreview-pause-20260922.md); Apollo-reviewed lands via bypass · why: re-enqueue #3 after update-branch (head a8c3c3d): twice removed from the queue with no group CI run and no red; slice-4/8 ejector fix (t_b303235e) |
|
Found 1 test failure on Blacksmith runners: Failure
|
|
🤖 merged-by: apollo · lane: unspecified · gate: BYPASS: FleetReview daily budget exhausted (spent=600 paged=true), no terminal record possible today; Ace directed admin-merge · why: ROOT-CAUSE fix for merge-queue ejections: anchors O(n^2) launchctl regex in tools/approval.py; test_execution_flag_detection.py 104s->23s locally (main vs branch), 6/6 sampled MQ ejections in 12h were this file timing out. Ace: admin-merge the fixes (21:06 PT) · red-ci allowed [All required checks pass,Python tests / Tests complete,Python tests / Run tests slice 13/16]: slice 13/16 red = tests/honcho_plugin/test_pin_peer_name.py, not in diff (2 files: tools/approval.py, tests/tools/test_local_env_blocklist.py); 25/25 green 3x on branch + on clean fork/main hermetic HOME; slice-order pollution |
|
🤖 merged-by: apollo · lane: unspecified · gate: BYPASS: FleetReview daily budget exhausted (spent=600 paged=true), no terminal record possible today; Ace directed admin-merge · why: ROOT-CAUSE fix for merge-queue ejections: anchors O(n^2) launchctl regex in tools/approval.py; test_execution_flag_detection.py 104s->23s locally (main vs branch), 6/6 sampled MQ ejections in 12h were this file timing out. Ace: admin-merge the fixes (21:06 PT) · red-ci allowed [All required checks pass,Python tests / Tests complete,Python tests / Run tests slice 13/16]: slice 13/16 red = tests/honcho_plugin/test_pin_peer_name.py, not in diff; 25/25 green 3x on branch + on clean fork/main hermetic HOME; slice-order pollution |
|
🤖 merged-by: apollo · lane: unspecified · gate: BYPASS: FleetReview daily budget exhausted (spent=600 paged=true), no terminal record possible today; Ace directed admin-merge · why: ROOT-CAUSE fix for merge-queue ejections: anchors O(n^2) launchctl regex in tools/approval.py; test_execution_flag_detection.py 104s->23s locally (main vs branch), 6/6 sampled MQ ejections in 12h were this file timing out. Ace: admin-merge the fixes (21:06 PT) · red-ci allowed [All required checks pass,Python tests / Tests complete,Python tests / Run tests slice 13/16]: slice 13/16 red = tests/honcho_plugin/test_pin_peer_name.py, not in diff; 25/25 green 3x on branch + on clean fork/main hermetic HOME; slice-order pollution |
FleetReviewBelow the merit threshold, review skipped (28 changed lines < 150 and 2 changed files < 6, no hot path touched; skipped by policy). FleetReview · reviewKind: skipped-by-policy |
Card t_b303235e. Both causes of the 3 MQ ejections on 'Run tests slice 4/8' (36179091722, 36180608091, 36204054240).
Verified in the container (uv sync --locked, py3.11): both files ran 20/20 green in isolation, 10 of those with the reader race forced. test_approval + test_execute_code_surfaces_blocks + test_execution_flag_detection: 197 passed. launchctl positive/negative probes behave the same as before.
Not a regression from the 18:11-18:35Z landings. Both defects predate them (#643, 2026-08-31, and 3c8d3ec, 2026-06-26). The 3700d36 LPT change only moved which files ran together.
Need help on this PR? Tag
@codesmith-botwith what you need. Autofix is disabled.