Skip to content
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion docs/architecture/git-isolation.md
Original file line number Diff line number Diff line change
Expand Up @@ -370,7 +370,7 @@ If an agent container crashes without cleanup:
3. **Fail-safe**: If the orchestrator is unreachable at startup, the sweep is **skipped entirely** rather than run blind. A redeploy with no answer from the orchestrator is safer than silently deleting a parked pipeline's work.
4. **Cleanup**: Remove working directories for confirmed orphans; branches are **never** deleted by the sweep — the sweep cannot know whether branches were pushed, and deleting them turns a recoverable mistake into data loss.

**Restart recovery:** When an agent is restarted (via `restart_agent` or `restart_phase`), the gateway's idempotent `create_worktrees` API detects the existing worktree keyed by `{pipeline_id}-{role}` and returns its host paths. The respawned container mounts the same worktree with all committed work intact. Uncommitted changes from the previous container are lost — agents should commit work incrementally to maximize recovery. See [Agent Recovery Reference](../reference/agent-recovery.md#worktree-preservation) for details.
**Restart recovery:** When an agent is restarted (via `restart_agent` or `restart_phase`), the gateway's idempotent `create_worktrees` API detects the existing worktree keyed by `{pipeline_id}-{role}` and returns its host paths. The respawned container mounts the same worktree with all committed work intact. Uncommitted changes from the previous container are not carried into the respawned worktree — since [#3639](https://github.com/jwbron/egg/issues/3639) they are first snapshotted into a commit and salvaged to a recovery ref rather than silently discarded, but agents should still commit work incrementally so it lands on the branch directly instead of depending on manual recovery. See [Agent Recovery Reference](../reference/agent-recovery.md#worktree-preservation) for details.

```python
def cleanup_orphaned_worktrees():
Expand Down
8 changes: 4 additions & 4 deletions docs/reference/post-agent-commit.md
Original file line number Diff line number Diff line change
Expand Up @@ -28,7 +28,7 @@ When the orchestrator detects uncommitted work in an agent's worktree after exit
|--------|-------------|
| **Recover** | A recovery agent or human manually reviews and commits the work |
| **Discard** | The worktree is cleaned up with uncommitted changes discarded |
| **Retry agent** | The agent is respawned and the gateway rediscovers the existing worktree (including all committed work on the branch); uncommitted changes from the old container are not carried over |
| **Retry agent** | The agent is respawned and the gateway rediscovers the existing worktree (including all committed work on the branch); uncommitted changes from the old container are not carried over into the reused worktree, but since [#3639](https://github.com/jwbron/egg/issues/3639) are first snapshotted into a recovery-ref commit rather than silently discarded — see [Committed but Unpushed: Auto-Salvage](#committed-but-unpushed-auto-salvage) |

## Worktree Lifecycle

Expand All @@ -38,7 +38,7 @@ Per-agent worktrees persist after container exit (until pipeline cleanup). Uncom

A different failure class exists when an agent **commits** work locally but its **pushes** to the remote are wedged — gateway branch-allowlist rejection from a wrong-branch spawn-time env var, transient infra failure, or restart-reconciliation marking a still-running pipeline `failed`. In these cases the commits sit on the local `egg/{worktree_id}/work` branch and are lost when `cleanup_pipeline` deletes the worktree.

Since [#2438](https://github.com/jwbron/egg/pull/2438), `cleanup_pipeline` automatically calls `auto_salvage_pipeline` before deleting any worktree. Since [#2526](https://github.com/jwbron/egg/pull/2526), `restart_phase` does the same — making phase restart (the scenario most likely to produce wedged pushes) a safe worktree-deletion path. Since [#2855](https://github.com/jwbron/egg/pull/2855), **agent restart** (`restart_agent_job`) also calls `auto_salvage_pipeline`, with `salvage_uncommitted=True` — so uncommitted edits present at restart time are committed to a synthetic `[salvage] pre-crash working-tree state` snapshot before the recovery push, extending salvage coverage to the mid-`Edit` crash window where work exists only in the dirty working tree. Since [#3509](https://github.com/jwbron/egg/issues/3509), **worktree re-attach**'s dirty-discard reset (the R6 `_clean_reused_worktree` path) also auto-salvages: any commits its `git reset --hard` is about to discard are pushed to a recovery ref first via `salvage_discarded_tip`, and the discarded tip + recovery ref are durably recorded as a message-bus system message so a resuming agent can find and resume its prior work instead of re-deriving it. Every per-agent worktree with local commits not reachable from `origin/<assigned_branch>` is pushed to a recovery ref:
Since [#2438](https://github.com/jwbron/egg/pull/2438), `cleanup_pipeline` automatically calls `auto_salvage_pipeline` before deleting any worktree. Since [#2526](https://github.com/jwbron/egg/pull/2526), `restart_phase` does the same — making phase restart (the scenario most likely to produce wedged pushes) a safe worktree-deletion path. Since [#2855](https://github.com/jwbron/egg/pull/2855), **agent restart** (`restart_agent_job`) also calls `auto_salvage_pipeline`, with `salvage_uncommitted=True` — so uncommitted edits present at restart time are committed to a synthetic `[salvage] pre-crash working-tree state` snapshot before the recovery push, extending salvage coverage to the mid-`Edit` crash window where work exists only in the dirty working tree. Since [#3509](https://github.com/jwbron/egg/issues/3509), **worktree re-attach**'s dirty-discard reset (the R6 `_clean_reused_worktree` path) also auto-salvages: any commits its `git reset --hard` is about to discard are pushed to a recovery ref first via `salvage_discarded_tip`, and the discarded tip + recovery ref are durably recorded as a message-bus system message so a resuming agent can find and resume its prior work instead of re-deriving it. Since [#3639](https://github.com/jwbron/egg/issues/3639), that same re-attach path first snapshots any dirty (uncommitted) state into a `[salvage] pre-reset working-tree state` commit before the reset — so uncommitted work, not just prior commits, becomes an ordinary orphan that the same salvage-and-record path recovers. Every per-agent worktree with local commits not reachable from `origin/<assigned_branch>` is pushed to a recovery ref:

```
egg/recovered/<pipeline_id>/<scope>/<short_sha>
Expand Down Expand Up @@ -98,11 +98,11 @@ await mcp.call_tool("list_agent_local_commits", {

| Before (auto-commit) | After (HITL recovery) |
|----------------------|----------------------|
| Uncommitted changes auto-committed on exit | Uncommitted changes remain in worktree (except on agent restart, where they're salvaged — see above) |
| Uncommitted changes auto-committed on exit | Uncommitted changes remain in worktree (except on agent restart or worktree re-attach, where they're salvaged — see above) |
| WIP commits pushed to branch automatically | No automatic push; HITL decides |
| Phase-restricted files filtered before commit | No filtering needed — nothing is auto-committed |
| Symlink filtering applied | No filtering needed |
| `egg/salvage-<id>` branches created for main | `egg/recovered/<pipeline>/...` refs auto-created by `cleanup_pipeline`, `restart_phase`, `restart_agent_job`, or worktree re-attach's dirty-discard reset for committed-but-unpushed work |
| `egg/salvage-<id>` branches created for main | `egg/recovered/<pipeline>/...` refs auto-created by `cleanup_pipeline`, `restart_phase`, `restart_agent_job`, or worktree re-attach's dirty-discard reset for committed-but-unpushed work (and, since [#3639](https://github.com/jwbron/egg/issues/3639), worktree re-attach's uncommitted dirty state too) |

### What operators should know

Expand Down
Loading