test(fd-drain): fix the FLAKY burn_fds fixture that made the high-fd guard red

[Kyzcreig]

Card: t_15ec143f (ban-forensics). Found by Apollo while E2E-verifying the FD_SETSIZE fix (#553 / t_fd198374).

The card's premise was close, but not quite right

The card read the failure as "the fixture is too weak to reach the cliff". Measured here: the fixture is not weak — it is racy. On this Mac it reaches fd 1087 comfortably and passes 59 of 60 runs. The reported assert 523 >= 1024 was the ~1-in-60 case, not the normal case.

That distinction matters, because the card's suggested fix (burn harder, to fd 1100) would not have fixed it — the fixture already gets past 1100.

Root cause (measured, not inferred)

POSIX hands a new fd the lowest free number. burn_fds allocated pipes until its own last fd cleared FD_SETSIZE, then checked only held[-1]. It never verified fds below 1024 were exhausted.

Any single descriptor freed while the fixture runs — a GC'd file object, a rotating log handler, pytest's own capture machinery — leaves a hole under the cliff, and the next subprocess.Popen pipe is handed that hole.

Instrumented run caught it exactly:

free<1024 AFTER burn : [911]
free<1024 AFTER popen: []
[FAIL] popen_fd=911

One fd (911) freed mid-burn; the pipe took it.

Fix

• Burn past the ceiling, then plug every remaining hole below FD_SETSIZE and assert none is left before yielding.
• _spawn_high_fd_echo retries a bounded number of times, parking (not closing) any pipe that lands in a hole opened between burn and spawn — each retry permanently fills one more hole, so it converges.
• Skips name the measured ceiling, so a genuinely constrained platform is visible in CI rather than red.

The anti-vacuous assertion is kept (card DO #2) and still fires on the real measured fd if every attempt lands low. Threshold unchanged at >= 1024. Only the race is removed.

Verification

A/B, 60 runs each arm:

arm	result
old fixture (origin/main)	59/60 — failed assert 931 >= 1024
new fixture	60/60

Deterministic hole-injection (1-in-60 is weak evidence, so I forced it — close one fd below the cliff after the burn):

OLD fixture + injected hole -> Popen stdout fd = 403   <- guard fires
NEW fixture + injected hole -> Popen stdout fd = 1086  <- survives

Mutation proof (card DO #4) — revert poll() -> select() with the historical silent swallow:

PASS  control: high-fd test PASSES with the poll() fix
PASS  mutant differs from original (mutation is real)
PASS  MUTANT KILLED: reverting poll()->select() turns the test RED
E     AssertionError: high-fd stdout was silently dropped - this is the blackout bug
E     assert 'HIGH_FD_MARKER_OK' in ''
PASS  mutant failed for the RIGHT reason
PASS  tree restored byte-clean after mutation
TOTAL: 5 passed, 0 failed

That assert 'HIGH_FD_MARKER_OK' in '' is the literal blackout signature — the test genuinely gates the bug.

Constrained-platform arm (hard rlimit 300, card DO #3):

SKIPPED: RLIMIT_NOFILE soft ceiling is 300, at or below the 1084 fds
         needed to push a pipe past FD_SETSIZE (1024)

Canonical runner: scripts/run_tests.sh tests/tools/test_base_environment_high_fd_drain.py -> 6 tests passed, 0 failed, exit 0.

Scope

tools/environments/base.py is untouched (card constraint) — the poll() fix is proven correct and live. One file changed, +133/-24.

Unrelated pre-existing failure

tests/tools/test_modal_snapshot_isolation.py fails 2 tests. Confirmed inherited: it fails identically on pristine origin/main with this change stashed. Not caused by this PR, not fixed here.

[Kyzcreig]

Wrong base repo — opened against upstream NousResearch instead of the ANG-Ventures fork, so it swept in the entire fork divergence (5,701 files). Reopening correctly against ANG-Ventures/hermes-agent:main.