Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
19 commits
Select commit Hold shift + click to select a range
5f2da9d
fix(bin): bound each lint root in its own ShellCheck process
kunchenguid Sep 26, 2026
c2835d3
fix(bin): fail closed on unenforceable lint bounds and size the cap
kunchenguid Sep 26, 2026
245fad1
no-mistakes(review): Prove memory cap binds, pass watchdog owner, dro…
kunchenguid Sep 26, 2026
e21391a
no-mistakes(review): Capture watchdog owner before startup for every …
kunchenguid Sep 26, 2026
9ed5c17
no-mistakes(document): Clarify bounded lint documentation and telemetry
kunchenguid Sep 26, 2026
2b8e535
no-mistakes(document): Correct bounded lint documentation and sidecar…
kunchenguid Sep 26, 2026
e562b3b
docs(bin): restore the per-root memory cap sizing rationale
kunchenguid Sep 26, 2026
27ec114
no-mistakes(review): Document memory cap RSS reduction threshold and …
kunchenguid Sep 26, 2026
0082f9d
no-mistakes(review): Scope owner-death escalation docs to the perl wa…
kunchenguid Sep 26, 2026
e0f139f
no-mistakes(document): Clarify bounded lint and timeout documentation
kunchenguid Sep 26, 2026
7f70e72
no-mistakes(review): Install perl watchdog signal handlers before for…
kunchenguid Sep 26, 2026
8f31324
no-mistakes(document): Correct bounded lint documentation and stale w…
kunchenguid Sep 26, 2026
20107c9
no-mistakes(ci): Fixed the supervision-host test’s obsolete expectati…
kunchenguid Sep 26, 2026
42699d1
no-mistakes(ci): The two lint checks failed when eight canonical root…
kunchenguid Sep 26, 2026
0d323ea
no-mistakes(review): Restore source directives, raise cap to 8 GiB, c…
kunchenguid Sep 26, 2026
3451024
no-mistakes(review): Classify memory deaths from root stderr, not sou…
kunchenguid Sep 26, 2026
a3c54b5
no-mistakes(review): Match only whole runtime memory-error lines for …
kunchenguid Sep 26, 2026
86139b1
no-mistakes(document): Clarify lint memory classification in script d…
kunchenguid Sep 26, 2026
c94a5fb
no-mistakes(ci): Fixed both lint checks’ memory-limit failure: each C…
kunchenguid Sep 26, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 8 additions & 1 deletion .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -53,6 +53,11 @@ jobs:
# and the pre-push gate on this script so a self-broken ci.yml still
# fails locally before merge.
- name: Lint canonical partition
env:
# Fail closed rather than lint uncapped when a configured per-root
# bound (wall deadline or memory rlimit) cannot be enforced here.
FM_LINT_REQUIRE_BOUNDS: '1'
FM_LINT_JOBS: '1'
run: |
set -eu
mkdir -p "$RUNNER_TEMP/fm-lint"
Expand All @@ -63,7 +68,9 @@ jobs:
uses: actions/upload-artifact@v4
with:
name: fm-lint-telemetry-${{ matrix.partition }}
path: ${{ runner.temp }}/fm-lint/partition-${{ matrix.partition }}.tsv
path: |
${{ runner.temp }}/fm-lint/partition-${{ matrix.partition }}.tsv
${{ runner.temp }}/fm-lint/partition-${{ matrix.partition }}.roots.tsv
if-no-files-found: warn

# Deterministic proof that portable parallel shards + portable serial + Herdr
Expand Down
498 changes: 440 additions & 58 deletions bin/fm-lint.sh

Large diffs are not rendered by default.

56 changes: 42 additions & 14 deletions bin/fm-timeout-lib.sh
Original file line number Diff line number Diff line change
Expand Up @@ -22,10 +22,21 @@
# group at the bound, and KILL once <grace-seconds> more have passed,
# for a command that ignores TERM or is mid-way through work it will not
# abandon. A TERM, INT, or HUP delivered to the bounding process is
# forwarded to the group and starts the same grace. Exit status is the
# command's own, except 124 (the bound was hit) or 137 (GNU timeout's
# status when its KILL had to fire); fm_timed_out accepts both. Both
# values must be positive integers (125 otherwise). The perl watchdog is
# forwarded to the group and starts the same grace. The perl watchdog
# also starts that escalation when its own parent dies before it could
# be signalled (an owner torn down by an outer group-kill cannot leave
# the bounded subtree orphaned behind it). The owner is captured before
# the watchdog starts: FM_EXEC_TIMED_OWNER_PID when the caller names it,
# else the calling script ($$) when fm_exec_timed runs in a subshell,
# else the shell's parent. The escalation starts once that owner is gone
# or the watchdog's parent changes, so an owner that dies while the
# watchdog is still starting is detected too. The timeout/gtimeout
# fallback does not track the owner: it bounds the command only by its
# deadline and grace, so owner death alone does not stop the command.
# Exit status is the command's own, except 124 (the bound was hit) or
# 137 (GNU timeout's status when its KILL had to fire); fm_timed_out
# accepts both. The seconds and grace values must be positive integers
# (125 otherwise). The perl watchdog is
# preferred: once termination has begun it also KILLs whatever the group
# left behind, so a descendant that outlives the command and holds its
# output cannot keep a capturing caller waiting, and GNU timeout, the
Expand Down Expand Up @@ -180,7 +191,7 @@ fm_timed_out() { # <status>
# which keeps the bound off perl's platform-dependent syscall-restart signal
# semantics and off the drift of counting sleep intervals.
fm_exec_timed() { # <seconds> <grace-seconds> <command...>
local seconds=${1:-} grace=${2:-} value
local seconds=${1:-} grace=${2:-} value owner
for value in "$seconds" "$grace"; do
case "$value" in
'' | 0* | *[!0-9]*)
Expand All @@ -194,18 +205,32 @@ fm_exec_timed() { # <seconds> <grace-seconds> <command...>
echo "fm_exec_timed: usage: fm_exec_timed <positive-seconds> <positive-grace-seconds> <command> [args...]" >&2
exit 125
fi
owner=${FM_EXEC_TIMED_OWNER_PID:-$$}
[ "$owner" != "$BASHPID" ] || owner=$PPID
unset FM_EXEC_TIMED_OWNER_PID
if command -v perl >/dev/null 2>&1; then
exec perl -MPOSIX=WNOHANG,setpgid -MTime::HiRes=time -e '
my ($bound, $grace) = (shift, shift);
my $pid = fork;
exit 127 unless defined $pid;
if ($pid == 0) { setpgid(0, 0); exec @ARGV; exit 127 }
setpgid($pid, $pid);
my $deadline = time + $bound;
my ($kill_at, $timed_out) = (0, 0);
my ($bound, $grace, $owner) = (shift, shift, shift);
my $parent = getppid();
my ($pid, $pending, $kill_at, $timed_out) = (0, "", 0, 0);
for my $sig (qw(TERM INT HUP)) {
$SIG{$sig} = sub { kill $sig, -$pid; $kill_at ||= time + $grace };
$SIG{$sig} = sub {
if ($pid) { kill $sig, -$pid } else { $pending = $sig }
$kill_at ||= time + $grace;
};
}
my $child = fork;
exit 127 unless defined $child;
if ($child == 0) {
$SIG{$_} = "DEFAULT" for qw(TERM INT HUP);
setpgid(0, 0);
exec @ARGV;
exit 127;
}
setpgid($child, $child);
$pid = $child;
kill $pending, -$pid if $pending;
my $deadline = time + $bound;
sub finish {
my $status = shift;
kill "KILL", -$pid if $kill_at;
Expand All @@ -226,10 +251,13 @@ fm_exec_timed() { # <seconds> <grace-seconds> <command...>
$timed_out = 1;
$kill_at = time + $grace;
kill "TERM", -$pid;
} elsif (getppid() != $parent || !kill(0, $owner)) {
$kill_at = time + $grace;
kill "TERM", -$pid;
}
select undef, undef, undef, 0.05;
}
' -- "$seconds" "$grace" "$@"
' -- "$seconds" "$grace" "$owner" "$@"
elif command -v timeout >/dev/null 2>&1; then
exec timeout -k "$grace" "$seconds" "$@"
elif command -v gtimeout >/dev/null 2>&1; then
Expand Down
6 changes: 3 additions & 3 deletions bin/fm-watch.sh
Original file line number Diff line number Diff line change
Expand Up @@ -182,7 +182,7 @@ WATCH_HOME_EXISTED=0
# without sourcing the entire watcher graph.
# The shared transition owner is a canonical lint root itself. Stop duplicate
# source-graph expansion here: following its backend graph from this large
# runtime can exceed the bounded CI lint worker while adding no uncovered file.
# runtime needlessly spends per-root CI lint memory while adding no uncovered file.
# shellcheck source=/dev/null
. "$SCRIPT_DIR/fm-push-transition-lib.sh"
# shellcheck source=bin/fm-pr-lib.sh
Expand All @@ -198,8 +198,8 @@ WATCH_HOME_EXISTED=0
# This library is a canonical lint root in its own right, and it reaches the
# wake queue, PR identity, and secondmate parent libraries. Keep it an analysis
# boundary here for the same reason as the transition and inbox owners above and
# below: following its graph from this large runtime exceeds the bounded CI lint
# worker while adding no uncovered file.
# below: following its graph from this large runtime needlessly spends per-root
# CI lint memory while adding no uncovered file.
# shellcheck source=/dev/null
. "$SCRIPT_DIR/fm-merge-outcome-lib.sh"
# The durable merge-authority owner is shared with bin/fm-pr-merge.sh. The
Expand Down
5 changes: 3 additions & 2 deletions docs/fm-test-portable-shards.md
Original file line number Diff line number Diff line change
Expand Up @@ -106,9 +106,10 @@ Portable shards, each portable serial shard, and the Herdr lane upload runner-ge

## Lint partitions and end-to-end latency

`bin/fm-lint.sh` owns two canonical CI partitions, each running the same full source-aware ShellCheck analysis with two bounded workers, pinned versions, workflow validation, and backend-purity checks.
`bin/fm-lint.sh` owns two canonical CI partitions, each running full source-aware ShellCheck analysis, workflow validation, and backend-purity checks.
CI requires its per-root bounds, so an unenforceable deadline or address-space limit refuses lint rather than running uncapped; the script header owns the envelope and per-root execution contract.
Its `--list-files` interface exposes partition membership; `tests/fm-lint.test.sh` verifies complete/disjoint executed roots and unchanged analysis flags.
The workflow uploads each partition's quiet telemetry to distinguish analysis cost, memory use, and host contention.
The workflow uploads each partition's quiet telemetry plus its per-root lifecycle sidecar to distinguish analysis cost, memory use, and host contention.
No fast mode, path skips, reduced checks, or paid runner provisioning is part of this layout.

The performance objective is a complete green run under fifteen minutes including start delay: roughly twelve minutes of longest-path execution, at most two minutes of runner delay, and less than one minute of other overhead.
Expand Down
Loading
Loading