perf: accelerate local validation with bounded concurrency

[kunchenguid]

Intent

Cut Firstmate LOCAL validation wall-clock by >=50% via CONCURRENCY plus POLL-SLEEP REDUCTION. Captain-authorized 2026-09-03 after the timeout scout. SCOPED to these levers only - do NOT broaden.

What Changed

• Route the no-mistakes changed-file test baseline through fm-test-run.sh, excluding live Herdr tests, and enable bounded automatic concurrency for explicit script lists.
• Admit the proven pr-forge family at up to four workers while separating family-scoped concurrency into isolated phases and keeping unproven tests serial.
• Add scheduler regression coverage and document concurrency evidence, refused family proofs, and why coarser polling intervals were not adopted.

Risk Assessment

✅ Low: The concurrency changes are bounded by recorded isolation families, cross-family work is separated into phases, and the timeout fix restores unbounded plain script-list behavior as requested.

Testing

Inspected the base-to-target change, ran the focused runner suite, and exercised the end-user named-script CLI flow; two one-second scripts began concurrently, completed in about 1.3 seconds with four resolved workers, and required no timeout helper, while focused coverage also confirmed serial override, mixed serial tails, explicit timeout behavior, and isolation-family phase separation. The worktree was clean after removing the temporary fixture.

Evidence: Named-script automatic concurrency CLI transcript

Source: Named-script automatic concurrency CLI transcript

END-USER COMMAND: bin/fm-test-run.sh tests/fm-cd-pretool-check.test.sh tests/fm-pr-merge.test.sh
FIXTURE CONTRACT: no bin/fm-timeout-lib.sh is present; both named scripts sleep for one second.
FM_TEST_BEGIN 2026-09-03T16:35:57Z tests/fm-cd-pretool-check.test.sh family=pure-contract-unit expected_gate_skip=none
FM_TEST_BEGIN 2026-09-03T16:35:57Z tests/fm-pr-merge.test.sh family=pr-forge expected_gate_skip=none
ok - named-script fixture completed
FM_TEST_END 2026-09-03T16:35:58Z tests/fm-cd-pretool-check.test.sh exit=0 duration_ms=1170 gate_skip=false
ok - named-script fixture completed
FM_TEST_END 2026-09-03T16:35:58Z tests/fm-pr-merge.test.sh exit=0 duration_ms=1173 gate_skip=false
FM_TEST_SUMMARY total=2 failed=0 skipped_gate=0 duration_ms=1306
FM_TEST_SUMMARY_FAMILY family=pr-forge count=1 duration_ms=1173 failed=0
FM_TEST_SUMMARY_FAMILY family=pure-contract-unit count=1 duration_ms=1170 failed=0
FM_TEST_SLOWEST rank=1 script=tests/fm-pr-merge.test.sh duration_ms=1173
FM_TEST_SLOWEST rank=2 script=tests/fm-cd-pretool-check.test.sh duration_ms=1170

TIMING ARTIFACT:
{
    "families": [
        {
            "count": 1,
            "duration_ms": 1173,
            "failed": 0,
            "name": "pr-forge"
        },
        {
            "count": 1,
            "duration_ms": 1170,
            "failed": 0,
            "name": "pure-contract-unit"
        }
    ],
    "finished_at": "2026-09-03T16:35:58Z",
    "run_id": "fm-test-run-1788453357237-58892",
    "scripts": [
        {
            "duration_ms": 1170,
            "exit": 0,
            "expected_gate_skip": "none",
            "family": "pure-contract-unit",
            "gate_skip": false,
            "path": "tests/fm-cd-pretool-check.test.sh"
        },
        {
            "duration_ms": 1173,
            "exit": 0,
            "expected_gate_skip": "none",
            "family": "pr-forge",
            "gate_skip": false,
            "path": "tests/fm-pr-merge.test.sh"
        }
    ],
    "selection": "scripts;jobs=4",
    "started_at": "2026-09-03T16:35:57Z",
    "summary": {
        "duration_ms": 1306,
        "failed": 0,
        "skipped_gate": 0,
        "total": 2
    }
}

Evidence: Named-script timing artifact

Source: Named-script timing artifact

{
  "families": [
    {
      "count": 1,
      "duration_ms": 1173,
      "failed": 0,
      "name": "pr-forge"
    },
    {
      "count": 1,
      "duration_ms": 1170,
      "failed": 0,
      "name": "pure-contract-unit"
    }
  ],
  "finished_at": "2026-09-03T16:35:58Z",
  "run_id": "fm-test-run-1788453357237-58892",
  "scripts": [
    {
      "duration_ms": 1170,
      "exit": 0,
      "expected_gate_skip": "none",
      "family": "pure-contract-unit",
      "gate_skip": false,
      "path": "tests/fm-cd-pretool-check.test.sh"
    },
    {
      "duration_ms": 1173,
      "exit": 0,
      "expected_gate_skip": "none",
      "family": "pr-forge",
      "gate_skip": false,
      "path": "tests/fm-pr-merge.test.sh"
    }
  ],
  "selection": "scripts;jobs=4",
  "started_at": "2026-09-03T16:35:57Z",
  "summary": {
    "duration_ms": 1306,
    "failed": 0,
    "skipped_gate": 0,
    "total": 2
  }
}

Pipeline

Updates from git push no-mistakes

✅ **intent** - passed

✅ No issues found.

✅ **Rebase** - passed

✅ No issues found.

🔧 **Review** - 2 issues found → auto-fixed (2) ✅

• 🚨 CONTRIBUTING.md:112 - The authoritative intent requires the >=50% reduction via “CONCURRENCY plus POLL-SLEEP REDUCTION,” but this change explicitly directs maintainers to leave the 0.1s poll sleeps unchanged. The authorized poll-sleep lever is therefore absent and needs user approval before replacing it with concurrency alone.
• 🚨 bin/fm-test-run.sh:446 - Adding pr-forge to the family registry makes it globally compatible with every other admitted family, although its proof only covers pr-forge members together. For example, a plain list containing fm-pr-check-security.test.sh and fm-calm-pi-extension.test.sh now runs concurrently despite belonging to separately proven families; the isolation record explicitly excludes the former from the mixed pool. Enforce compatibility at the batch boundary by running each family-proof group in a separate concurrent phase, while only globally proven scripts may mix across families.

🔧 Fix: Separate concurrent runs by isolation proof family
2 errors still open:

• 🚨 CONTRIBUTING.md:112 - Intent requires “>=50% via CONCURRENCY plus POLL-SLEEP REDUCTION,” but the change explicitly preserves every sleep 0.1; its directly reported verification benchmark is only a 48% reduction. Replacing the required poll-sleep lever with concurrency alone needs user approval.
• 🚨 bin/fm-test-run.sh:1861 - The intent says “SCOPED to these levers only - do NOT broaden,” but plain script-path runs now also acquire a new automatic 900-second timeout. For example, fm-test-run.sh tests/slow.test.sh changes from unbounded execution to termination with exit 124 despite no timeout being requested. This non-concurrency behavior expansion requires user approval.

🔧 Fix: Limit automatic timeouts to changed-file validation
✅ Re-checked - no issues remain.

✅ **Test** - passed

✅ No issues found.

• git diff 3d2a08b2097dd24f9ce03fdafe6501e39b79dbd0..58a5db6d9b71a3116dab61537be5dd89bb450880
• bash tests/fm-test-run.test.sh
• bin/fm-test-isolation-proof.sh --list
• bin/fm-test-run.sh --help
• Ran a fixture through bin/fm-test-run.sh tests/fm-cd-pretool-check.test.sh tests/fm-pr-merge.test.sh --json <evidence-path> with no timeout helper installed
• Parsed the generated timing JSON and verified selection=scripts;jobs=4, successful exits, and concurrent wall-clock behavior
• git status --short and transient-fixture cleanup check

✅ **Document** - passed

✅ No issues found.

✅ **Lint** - passed

✅ No issues found.

✅ **Push** - passed

✅ No issues found.

---

Measurement evidence

All figures measured on the captain's Mac, 2026-09-03, 0 failures on both sides of every comparison unless noted. The machine carried unrelated fleet load throughout, so paired comparisons were run back to back in the same window.

The number that changes a verification round

A round of four scripts, run the way an agent runs one today versus named to the runner:

	wall	failures
chained bash tests/X.test.sh, one at a time	448s	0
the same four named to bin/fm-test-run.sh	231s	0
cut	48.4%

Scripts: fm-captain-hold-lifecycle, fm-pr-check-security, fm-teardown, fm-x-mode.

Per-family, one worker versus four

family	j1	j4	speedup	cut
watcher-wake-lock (already admitted)	1311.1s	539.3s	2.43x	58.9%
pr-forge (admitted here)	409.2s	237.9s	1.72x	41.9%
pure-contract-unit (already admitted, prior measurement)	510.7s	252.4s	2.02x	50.6%
secondmate (refused)	1432.1s	~478s at proof	~3.0x	not admitted

pr-forge cannot beat 2.1x: its longest script alone runs 198.5s of the 409.2s.

The gate's configured baseline, run exactly as configured: bin/fm-test-run.sh --changed --exclude-family real-herdr-gated selected 33 scripts and finished in 199.8s with 0 failures.

Families admitted and refused

pr-forge is admitted on two consecutive clean proofs at four workers: total=6 failed=0 duration_ms=198594 and total=6 failed=0 duration_ms=186796.

Two families were proven and refused, and docs/fm-test-isolation-proof.md records the exact script and reason for each so the refusals are actionable:

• secondmate — 14.3 minutes of the suite and it scales about 3x, so it is the largest remaining prize. One script blocks it: tests/fm-backlog-handoff.test.sh failed in two runs of three, both times on its crash-recovery case. That case SIGKILLs a real handoff mid-operation; under four workers the kill lands after the move instead of before it, so recovery reports Task "pre-move-crash" not found in this backlog. The other 19 scripts passed in every run, so the blocker is one crash-injection race, not shared secondmate state.
• session-bootstrap — one failure, tests/fm-session-start.test.sh reporting the digest waited 9s for inactive reconciliation's 8s state read. That assertion measures elapsed time rather than shared state, and the host's five-minute load average was above 12 while the proof ran. Recorded as a refusal because admission requires a passing proof and the harness never retries a failure into green; re-run it on an idle host before deciding.

unclassified (12.0 minutes) was targeted and not attempted: it currently holds tests/fm-backend-herdr-focus-flash-e2e.test.sh, a real-Herdr lab regression whose family should be real-herdr-gated, and an opt-in live-harness script that gate-skips on its first line. Worth noting separately: because that Herdr regression sits in the portable serial lane today, Linux CI gate-skips it and it never actually runs.

Whole-suite result, and why it is short of 50%

Against the scout's 77.4-minute serial baseline:

family	serial	after	basis
watcher-wake-lock	15.1	6.2	measured 2.43x
pure-contract-unit	6.2	3.1	measured 2.02x
pr-forge	5.8	3.4	measured 1.72x
everything else	50.3	50.3	unproven, stays serial
total	77.4	63.0	18.6% cut

With the three refused families admitted the same arithmetic gives about 41 minutes, a 47% cut — which is exactly what the scout's revised section 11 predicted for concurrency alone at the four-worker ceiling. Concurrency was never going to reach 50% on the whole suite by itself.

The second lever was meant to close that gap, and the measurement refused it. Raising the bounded-wait sample interval from 0.1s to 0.5s, charging each sample proportionally so no budget shrank:

fm-watch-triage.test.sh	secs
unchanged	390, 393
coarser sample interval	440, 435

Back to back, same window: 11.8% slower. The scout named this exact falsifier — if 394s does not fall to roughly 300s, the per-sleep penalty is not recoverable overhead. Those sleeps are not overhead added to the clock; they are how a test waits for a subject that only moves on fm-watch.sh's own one-second FM_POLL cadence, so sampling less often only delays detection. The same change also broke tests/fm-watcher-lock.test.sh, which catches a transient (a watcher alive and holding the lock it just reclaimed) rather than waiting for a settled condition — it failed in 16s where the unchanged script passed in 116s.

The lever is therefore not in this change, and CONTRIBUTING.md records the result so the experiment is not repeated.

[greptile-apps]

Confidence Score: 5/5

The PR appears safe to merge, with no concrete changed-code defect remaining after review.

The scheduler preserves family proof boundaries through separate phases, leaves unproven scripts serial, and retains dedicated CI coverage for the locally excluded Herdr family.

_{Reviews (1): Last reviewed commit: "no-mistakes(document): Clarify validatio..." | Re-trigger Greptile}