Skip to content

fix: make TERM→KILL escalation poll for group death and document the zombie-probe invariant - #3887

Merged
max-sixty merged 5 commits into
mainfrom
escalation-probe-design
Aug 23, 2026
Merged

max-sixty merged 5 commits into
mainfrom
escalation-probe-design

Conversation

@max-sixty

@max-sixty max-sixty commented Aug 23, 2026 •

Copy link
Copy Markdown
Owner

The TERM→KILL escalation probed group liveness with one killpg(pgid, 0) after a fixed 200 ms sleep, and that probe counts an exited-but-unreaped member as alive (Linux answers Ok, macOS EPERM). On the timed-out-command path the caller deliberately holds the group leader unreaped, so the group read alive for the whole grace and the final group SIGKILL fired every time.

That SIGKILL is inert — a signal to a fully-exited group is discarded and cannot change its recorded exit status (verified empirically, and pinned by a new test) — so this was never a correctness bug: whenever the probe over-reports, the KILL hits nothing; whenever the KILL matters, the group genuinely is alive. The costs were latency and legibility: every escalation step slept its full 200 ms even when the tree was already dead (every Ctrl-C of an isolated streamed child, every wt step tether teardown), and the probe read as a broken correctness gate.

The change keeps the semantics and reframes the roles, matching coreutils timeout -k (which KILLs unconditionally after its grace, with no probe at all):

  • The post-grace SIGKILL is the documented contract: whatever remains after the grace is swept. It is also the only signal that removes a SIGSTOP'd member, which runs no TERM handler.
  • The probe becomes an early-exit optimization: a 20 ms poll with an immediate first probe replaces the fixed sleep. Callers whose children are reaped concurrently (the signal forwarder, tether's supervisor) return within one poll interval; the timed-out path still runs its full grace and sends the inert sweep, which its docstring now states.
  • Signals sent to live groups are identical to before at all three call sites; only signals with provably no live target are skipped, and returns are never later than before.

The correctness argument lives in the docstrings on process_group_alive, forward_signal_with_escalation, and kill_timed_out_tree. Three new unit tests pin the invariants: immediate return for an empty group, full grace on a live group, and — replaying the timed-out sequence with the leader held unreaped — a preserved exit code through both the TERM and the post-grace sweep (barriered on the child's stdout EOF, so no scheduling assumptions).

Testing: the unit tests above plus the existing signal-forwarding and tether integration tests; full pre-merge gate green locally. The full-grace test pins the zombie-probe reading on both Linux and macOS CI (the test (linux)/test (macos) jobs run the lib unit tests).

This was written by Claude Code on behalf of max-sixty

max-sixty and others added 4 commits August 23, 2026 11:44
…zombie-probe invariant

process_group_alive counts an exited-but-unreaped member as alive (Linux: Ok,
macOS: EPERM), so on the timed-out-command path — whose caller holds the group
leader unreaped — the group read alive for the whole grace and the final
SIGKILL always fired. That sweep is provably inert against a dead group
(signals to fully-exited processes are discarded), so the fix reframes rather
than re-plumbs: the probe is now an early-exit optimization (20ms poll,
immediate first probe) and the post-grace SIGKILL is the documented straggler
sweep. Callers whose children are reaped concurrently (Cmd::stream's signal
forwarder, tether) now return within one poll interval instead of always
sleeping the full 200ms per escalation step.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Barrier on the child's stdout EOF (the pipe closes at process exit) instead of
racing TERM delivery against the grace: the child exits on its own before
escalation starts, so no CI-scheduling assumption remains, and the preserved
exit code pins that both the TERM and the post-grace sweep are inert against
an exited-but-unreaped group.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

@worktrunk-bot worktrunk-bot left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The equivalence argument holds: the next signal in the chain still only fires after the full grace, so signals to live groups land at exactly the same times as before, and the early return is reachable only via ESRCH — which requires every member reaped, and reaping is monotonic. The one place the two differ is pid recycling, where polling is strictly safer than the single end-of-grace probe: a group reaped at 100 ms whose pgid was recycled by 200 ms used to read alive and draw a killpg at a stranger.

One thing worth folding back into the PR body: test (macos) is green on this head, and wt hook pre-merge --yes runs the lib unit tests, so test_escalation_full_grace_and_inert_sweep_when_leader_unreaped executed there. That makes the macOS half of the process_group_alive docstring's zombie claim empirically pinned by CI, not just Linux — the description currently scopes the pinning to Linux.

One non-blocking test suggestion inline.

Comment thread src/shell_exec.rs
A 30s grace makes returning early structurally distinguishable from sleeping
it, so the wall-clock bound no longer races CI scheduling.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@max-sixty
max-sixty merged commit 6aad9e1 into main Aug 23, 2026
38 checks passed
@max-sixty
max-sixty deleted the escalation-probe-design branch August 23, 2026 21:06
social4hyq pushed a commit to social4hyq/homebrew-core that referenced this pull request Sep 20, 2026
worktrunk 0.75.0

Created-by: HarmonybrewBot
Commit-by: HarmonybrewBot
Merged-by: HarmonybrewBot
Description: Created by `brew bump`

---

Created with `brew bump-formula-pr`.<details>
  <summary>release notes</summary>
  <pre>## Release Notes

### Improved

- **The `wt switch` picker opens on one unified diff**: local rows combine committed, staged, unstaged, and untracked changes. Tab skips empty subsidiary views, while `Alt-1` through `Alt-8` retain direct access. [Docs](https://worktrunk.dev/switch/#interactive-picker) ([#3865](max-sixty/worktrunk#3865))

- **Git 2.43 is now the minimum supported version**: older Git exits with an upgrade message before Git-dependent commands run; `wt config shell` remains available so shell startup continues. (Breaking.) ([#3895](max-sixty/worktrunk#3895))

### Fixed

- **`wt list` no longer grows `.git/objects` during advisory conflict checks**: untracked content stays visible but is excluded from synthetic trees, and all probe-only objects use temporary storage. ([#3906](max-sixty/worktrunk#3906), fixes [#3883](max-sixty/worktrunk#3883), thanks @srobroek for reporting and the original fix)

- **`HEAD±` counts untracked files without inflating moves**: `wt list`, the picker, and statusline include untracked lines. Tracked deletions paired with untracked destinations count as renames, so pure moves are line-neutral and edited moves show only their edits. `HEAD±` now always detects renames, regardless of `diff.renames`. ([#3925](max-sixty/worktrunk#3925))

- **Timed child processes no longer abort `wt` in restricted sandboxes**: `wt switch --create`, picker pagers, and other bounded commands survive denied signal-handler wakes. TERM→KILL cleanup also returns promptly once the process group is gone. ([#3857](max-sixty/worktrunk#3857), [#3887](max-sixty/worktrunk#3887), fixes [#3856](max-sixty/worktrunk#3856), thanks @tomascamargo for reporting)

- **JSON list output ignores display-column gates**: `[list] columns` no longer makes `wt list --format json` contact a forge or generate summaries. CI requires `--full`; summaries also require `[list] summary = true` and a configured generator. (Breaking: schema 1 loses config-driven `ci` and `summary` fields.) ([#3812](max-sixty/worktrunk#3812), thanks @emeren for the request)

- **Long Windows paths compare consistently**: paths beyond 260 characters could retain a `\\?\` prefix and appear to be on another drive. `copy-ignored` then refused them, while switch, remove, merge, and relocate landed at the worktree root instead of the original subdirectory. ([#3899](max-sixty/worktrunk#3899), fixes [#3898](max-sixty/worktrunk#3898), thanks @Persedes for reporting and verifying the fix)

- **Shell configuration rechecks before it writes**: overlapping installs lock and reread rc files; Fish completion installs preserve files created after preview; uninstall applies only previewed rc removals and rejects changed Worktrunk-owned files. ([#3853](max-sixty/worktrunk#3853), [#3924](max-sixty/worktrunk#3924))

- **`-vv` profiles exclude their own collector commands**: command counts and cache summaries no longer include duplicate-looking work performed only to assemble the diagnostic report; raw traces still retain it. ([#3900](max-sixty/worktrunk#3900))

- **Fenced HTML comments survive picker Markdown rendering**: PR descriptions and comments now preserve fenced `<!-- … -->` lines; fenced `<!-- wt list … -->` markers also no longer affect the following block. ([#3908](max-sixty/worktrunk#3908))

### Documentation

- **Agent CLIs without a plugin can publish activity markers**: the integration guide now specifies the session-start, turn-end, and session-end calls, working-directory requirement, error guard, and cleanup contract. [Docs](https://worktrunk.dev/claude-code/#agent-clis-without-a-plugin) ([#3848](max-sixty/worktrunk#3848), thanks @AsafMah for requesting generic-agent guidance and @ortonomy for the related Pi use case)

- **Agent guidance explains worktree selection**: commands that name a branch already select its worktree; `-C` changes repository context and is needed only for commands without a worktree selector or callers outside the repository. ([#3890](max-sixty/worktrunk#3890))

- **The `wt up` recipe safely updates dirty worktrees**: it fetches all remotes, fast-forwards dirty branches without autostash, rebases clean branches, and continues past an ordinary refusal or a failed remote. [Docs](https://worktrunk.dev/extending/#recipe-rebase-every-worktree-onto-its-upstream) ([#3882](max-sixty/worktrunk#3882))

- **The docs site has a new responsive design**: rebuilt on Astro and Starlight while preserving public routes, anchors, and crawler URLs; generated reference pages remain synchronized. ([#3866](max-sixty/worktrunk#3866))

### Internal

- **Library API rework** (Breaking library API): `BranchDiffSpec` gained `working_base`, while remote-URL, shell-path, branch-push, approval, temporary-index, and repository helpers were removed. ([#3833](max-sixty/worktrunk#3833), [#3853](max-sixty/worktrunk#3853), [#3865](max-sixty/worktrunk#3865), [#3866](max-sixty/worktrunk#3866), [#3875](max-sixty/worktrunk#3875))

- **Codex loads repository maintainer skills**: `.agents/skills` now exposes the canonical `.claude/skills` tree; checkouts without symlink support keep the existing limitation. ([#3903](max-sixty/worktrunk#3903))

## Install worktrunk 0.75.0

### Install prebuilt binaries via shell script

```sh
curl --proto '=https' --tlsv1.2 -LsSf https://github.com/max-sixty/worktrunk/releases/download/v0.75.0/worktrunk-installer.sh | sh && wt config shell install
```

### Install prebuilt binaries via powershell script

```sh
powershell -ExecutionPolicy Bypass -c "irm https://github.com/max-sixty/worktrunk/releases/download/v0.75.0/worktrunk-installer.ps1 | iex"; git-wt config shell install
```

### Install prebuilt binaries via Homebrew

```sh
brew install worktrunk && wt config shell install
```

## Download worktrunk 0.75.0

|  File  | Platform | Checksum |
|--------|----------|----------|
| [worktrunk-aarch64-apple-darwin.tar.xz](https://github.com/max-sixty/worktrunk/releases/download/v0.75.0/worktrunk-aarch64-apple-darwin.tar.xz) | Apple Silicon macOS | [checksum](https://github.com/max-sixty/worktrunk/releases/download/v0.75.0/worktrunk-aarch64-apple-darwin.tar.xz.sha256) |
| [worktrunk-x86_64-apple-darwin.tar.xz](https://github.com/max-sixty/worktrunk/releases/download/v0.75.0/worktrunk-x86_64-apple-darwin.tar.xz) | Intel macOS | [checksum](https://github.com/max-sixty/worktrunk/releases/download/v0.75.0/worktrunk-x86_64-apple-darwin.tar.xz.sha256) |
| [worktrunk-x86_64-pc-windows-msvc.zip](https://github.com/max-sixty/worktrunk/releases/download/v0.75.0/worktrunk-x86_64-pc-windows-msvc.zip) | x64 Windows | [checksum](https://github.com/max-sixty/worktrunk/releases/download/v0.75.0/worktrunk-x86_64-pc-windows-msvc.zip.sha256) |
| [worktrunk-aarch64-unknown-linux-musl.tar.xz](https://github.com/max-sixty/worktrunk/releases/download/v0.75.0/worktrunk-aarch64-unknown-linux-musl.tar.xz) | ARM64 MUSL Linux | [checksum](https://github.com/max-sixty/worktrunk/releases/download/v0.75.0/worktrunk-aarch64-unknown-linux-musl.tar.xz.sha256) |
| [worktrunk-x86_64-unknown-linux-musl.tar.xz](https://github.com/max-sixty/worktrunk/releases/download/v0.75.0/worktrunk-x86_64-unknown-linux-musl.tar.xz) | x64 MUSL Linux | [checksum](https://github.com/max-sixty/worktrunk/releases/download/v0.75.0/worktrunk-x86_64-unknown-linux-musl.tar.xz.sha256) |



### Install via Cargo

```sh
cargo install worktrunk && wt config shell install
```

### Install via Winget (Windows)

```sh
winget install max-sixty.worktrunk && git-wt config shell install
```

### Install via AUR (Arch Linux)

```sh
paru worktrunk-bin && wt config shell install
```
</pre>
  <p>View the full release notes at <a href="https://github.com/max-sixty/worktrunk/releases/tag/v0.75.0">https://github.com/max-sixty/worktrunk/releases/tag/v0.75.0</a>.</p>
</details>
<hr>

See merge request: Harmonybrew/homebrew-core!17938
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants