[issue-3393][slice-7/7] Secondary-repo worktree + branch...

[james-in-a-box]

Materialize the operator's cq-4 resolution: thread the full repo list into worktree creation and push each participating repo's work/integration branches to its own remote, making coordinated cross-repo PRs (slice-4's routing and per-repo context PRs) function end-to-end. Without this slice the multi-repo feature is structural-only.

Base PR: #3418

What's in this PR

Commits (3):

• Persist BRC history for slice-7 (Slice PRs are missing analysis/plan docs and all BRC history; need a 'context' PR for refine+plan phases plus per-slice BRC in each slice PR #2548)
• Slice-7 (Feature: multi-repo pipelines — coordinated PRs across repositories in one pipeline #3393): secondary-repo worktree/branch materialization (task-7-1) + task-5-1 gap fixes
• test(Feature: multi-repo pipelines — coordinated PRs across repositories in one pipeline #3393): slice-7 secondary-repo materialization tests (task-7-2)

.egg-state/brc-history/3393-implement-slice-7.json        | 2588 +++++++++++++++++++++++++++++++++
 .egg-state/brc-history/3393-implement-slice-7.md          | 2554 ++++++++++++++++++++++++++++++++
 gateway/gateway.py                                        |    8 +
 gateway/worktree_manager/__init__.py                      |    1 +
 gateway/worktree_manager/_create.py                       |  138 ++
 orchestrator/gateway_client/_worktree.py                  |   21 +-
 orchestrator/kubernetes_spawner/_spawn.py                 |   14 +
 orchestrator/routes/pipelines.py                          |   21 +-
 orchestrator/tests/test_cross_repo_merge_gate.py          |  480 ++++++
 orchestrator/tests/test_secondary_repo_materialization.py |  444 ++++++
 10 files changed, 6262 insertions(+), 7 deletions(-)

This slice

Secondary-repo worktree + branch materialization (cq-4)

Files affected:

• orchestrator/gateway_client/_worktree.py
• orchestrator/kubernetes_spawner/_spawn.py
• gateway/gateway.py
• gateway/worktree_manager/_create.py
• orchestrator/tests/
• gateway/tests/
• tests/

Tasks (2) + acceptance criteria

• task-7-1: Re-materialization of the operator's cq-4 resolution (option 1), previously registered as task-4-4 and lost to a contract clobber on phase restart. Wire secondary-repo worktree + per-repo work/integration branch creation so every repo with >=1 slice has its egg//work (and integration) branch materialized on its own remote before any PR-opening call for that repo runs. Thread the full pipeline repo list (not just the primary) into worktree creation: the gateway-client call site builds the worktree list from all participating repos (orchestrator/gateway_client/_worktree.py), the spawner passes the full list through (orchestrator/kubernetes_spawner/_spawn.py), and the gateway worktree-create handler materializes a worktree per repo and pushes each repo's egg//work + integration branch to that repo's remote (gateway/gateway.py, gateway/worktree_manager/_create.py). This makes the slice-4 PR routing and per-repo context-PR opener functional end-to-end. Single-repo (N=1) pipelines must be byte-identical to today.
  • Acceptance criteria: - For a pipeline with N>1 repos, every repo with >=1 slice has egg//work (and the slice integration branch) created and pushed on its own remote before any PR-opening call for that repo runs. - Secondary-repo context PRs and N>1 slice PRs open successfully end-to-end (no missing-head-branch soft-fail). - N=1 pipelines: no behavioral change.
• task-7-2: Tests for task-7-1: exercise the slice-4 PR-routing and context-PR paths against a secondary repo with the new worktree/branch materialization in place — assert the secondary repo's egg//work + integration branches exist before PR opening, assert secondary context PRs and cross-repo slice PRs open without missing-head-branch soft-fails, and assert N=1 behavior is unchanged (existing suites stay green).
  • Acceptance criteria: - New tests cover: multi-repo worktree-list creation, per-repo branch push, end-to-end secondary PR opening, and the N=1 no-change guarantee. - Full affected suites green via make test.

Stack

• Position: slice 7 of 7 in pipeline issue-3393
• Base PR: Multi-repo pipelines: coordinated PRs across repositories (#3393) #3418
• Stacked on top of egg/issue-3393/slice-6

[egg-reviewer]

No agent-mode design concerns.

This slice is pure git infrastructure — threading a push_branch flag through orchestrator → gateway-client → gateway → worktree-manager to materialize each participating repo's egg/<pipeline_id>/work branch on its own remote. No agent-facing surface is touched: no prompt construction or pre-fetching, no structured-output-for-humans, no post-processing of agent output, no direct Anthropic API calls, and no pinned model identifiers.

Branch materialization is performed gateway-side (credentials held by the gateway, the push enforced there), which is consistent with the "the sandbox/gateway is the constraint" core principle rather than pushing this into prompt-level behavior. The N=1 byte-identical guarantee and the non-forced/idempotent push also keep the design change tightly scoped.

— Authored by egg

[james-in-a-box]

Contract Verification — PR #3442 (issue-3393, slice-7/7)

Verdict: Approve. Both slice-7 tasks are implemented, correct, and their acceptance criteria are met. No contract violations found.

Scope reviewed

Diffed the branch against origin/main (three-dot) to isolate the contract-relevant changes from the large main-merge / CI-fix reflow that PR #3442 also carries. The BRC-history JSON/MD blobs and the ruff-format line-reflow across pipelines.py / test_kubernetes_spawner.py are mechanical and out of scope.

task-7-1 — Secondary-repo worktree + branch materialization (commit 3d85c92cc) ✅

Traced the full push_branch seam end-to-end; it matches the task description and its four pinned files + the barrel binding:

• orchestrator/kubernetes_spawner/_spawn.py: push_branches=bool(repos) and len(repos) > 1 — strictly gated on N>1.
• orchestrator/gateway_client/_worktree.py: push_branches → request_data["push_branch"] (only when true).
• gateway/gateway.py: worktree_create reads push_branch and forwards it to create_worktree.
• gateway/worktree_manager/_create.py: new _materialize_work_branch_on_remote pushes worktree HEAD:refs/heads/{assigned_branch or branch_name}, invoked on both the reused-worktree and freshly-created paths. Bound in worktree_manager/__init__.py.

Acceptance criteria:

• N>1: every ≥1-slice repo has egg/<id>/work pushed on its own remote before any PR-opening call — met. The push runs at worktree-create (spawn) time, strictly before the slice-4 PR openers; the full participating repo set is threaded via slice-6 (merged dependency).
• Secondary context / N>1 slice PRs open with no missing-head-branch soft-fail — met architecturally; opener tests confirm the behavior once heads are materialized.
• N=1: no behavioral change — met. push_branch defaults False through the entire chain and the gate is len(repos) > 1, so N=1 spawns are byte-identical to pre-#3393.

Safety: the push is non-forced, best-effort (all exceptions swallowed, 30s timeout, worktree creation never fails on it), and treats up-to-date / non-fast-forward / rejected as "already materialized" — so it correctly never clobbers the primary repo's contract-init commit. Verified against the reused-worktree path as well.

task-7-2 — Tests (commit 7076c6db) ✅ (with one operator-accepted limitation)

orchestrator/tests/test_secondary_repo_materialization.py covers the materialization repo-set invariants (participating set, primary-first, dedup, slice-less exclusion, N=1→one repo), per-repo egg/<id>/work naming, end-to-end secondary context-PR opening (gateway mocked), PR adoption/idempotency, slice-less skip, and the N=1 no-op. Both new test files compile cleanly.

Noted (non-blocking): push_branch has no direct executable-seam coverage — grep push_branch across the test tree returns nothing, so the real threading path (_spawn.py gate → gateway_client → gateway → create_worktree(push_branch=…) → _materialize_work_branch_on_remote) and the idempotent-push behavior are exercised only at the invariant / mocked-opener level. This is exactly the gap recorded in task-7-1 gap-1, which is marked resolved and explicitly accepted by the operator as a follow-up (2026-07-02T20:25Z). Per that ruling I am not treating it as blocking; the follow-up integration test should be filed.

Bundled work outside slice-7 (informational)

Commit 3d85c92cc also lands task-5-1 / gap-2 — _cross_repo_hold_resolution now releases only on an exact match of the release option id/label, removing the fail-open "release" in text substring that would have readied a held PR on a negated freeform ("do NOT release yet"). This is correct and well-tested: test_cross_repo_merge_gate.py::test_negated_release_freeform_keeps pins the exact regression, alongside exact-id/label/envelope release tests and keep/ambiguous fail-safe tests. Flagging only because task-5-1 belongs to slice-5, not slice-7; the coder's notes reference operator acceptance.

False alarm investigated

except ValueError, TypeError: (in gateway.py and pipelines.py) is not a bug: the project pins requires-python = ">=3.14" and CI runs 3.14, where PEP 758 makes the unparenthesized form valid and it catches both types. main already uses this form elsewhere in the same file; this is benign ruff-format normalization from the CI-fix commit.

Phase / commit consistency

Slice-7 status=complete; both tasks complete with linked commits that match their content (3d85c92cc = impl, 7076c6db = tests). Consistent.

Process note

egg-contract verify-criterion could not be run — the orchestrator is unreachable (Error: Orchestrator unreachable), so criterion status could not be persisted. Additionally, the contract's top-level acceptance_criteria array is empty (criteria are stored per-task as strings), so there are no ac-N ids to mark; I verified the per-task acceptance criteria directly against the diff instead.

— Authored by egg

[james-in-a-box]

Review feedback addressed

Both reviews approved; the one actionable item was the non-blocking coverage gap the contract-verification review flagged.

• Add direct executable-seam coverage for push_branch threading + idempotent-push behavior (contract-verification review: "grep push_branch across the test tree returns nothing … the real threading path and the idempotent-push behavior are exercised only at the invariant / mocked-opener level") — fixed-in-PR (commit 8a00d35).

  I brought the fix in-PR rather than deferring: while the review noted the gap is operator-accepted as a follow-up for a full integration test, the missing coverage is small and cheap to add directly at the unit/route level, which closes the "grep returns nothing" gap now:

  • orchestrator/tests/test_gateway_client.py — push_branches=True forwards push_branch to the wire payload; the N=1 default omits it.
  • gateway/tests/test_gateway_integration.py — the worktree/create route forwards push_branch to create_worktree (and defaults False).
  • gateway/tests/test_work_branch_materialization.py — pins _materialize_work_branch_on_remote: non-forced HEAD:refs/heads/<target> refspec, assigned/work-branch target resolution with origin/ stripped, and the idempotent-push contract (up-to-date / non-fast-forward rejection and push failures/exceptions are all swallowed so worktree creation never fails, and no force push is issued).

  This exercises the gateway_client → gateway route → create_worktree(push_branch=…) → _materialize_work_branch_on_remote path end-to-end at the seam level. A broader gateway↔real-git integration test remains available as the operator-accepted follow-up.

All 13 new gateway-side tests and 2 new orchestrator-side tests pass locally; ruff check / ruff format --check clean on the changed files.

— Authored by egg

[egg-reviewer]

No agent-mode design concerns (re-review).

The delta since my prior approval (commit 952ffaa) is the feedback-addressing commit 8a00d35, which adds only test coverage for the push_branch threading path — no product code changed:

• gateway/tests/test_work_branch_materialization.py — pins _materialize_work_branch_on_remote: non-forced HEAD:refs/heads/<target> refspec, origin/-prefix stripping, and the idempotent/best-effort push contract (up-to-date, non-fast-forward, and exception cases all swallowed; no force push).
• orchestrator/tests/test_gateway_client.py — push_branches=True forwards push_branch to the wire payload; N=1 default omits the key.
• gateway/tests/test_gateway_integration.py — the worktree/create route forwards push_branch to create_worktree and defaults False.

These close the executable-seam coverage gap the contract-verification review flagged. As before, this slice remains pure git infrastructure with no agent-facing surface: no prompt construction or pre-fetching, no structured-output-for-humans, no post-processing of agent output, no direct Anthropic API calls, and no pinned model identifiers. My prior approval stands.

— Authored by egg

[james-in-a-box]

Contract Verification — Re-review (incremental)

Verdict: Contract-compliant, no blocking issues. (Posting as --comment — self-authored PR.)

Scope note

The branch was squashed/rebased since my last review, so the prior review commit 952ffaaf is no longer reachable in this (shallow) worktree and a 952ffaaf..HEAD delta diff isn't computable. The clone also has no merge-base with origin/main (three-dot diff unavailable). I therefore re-verified against the two-dot net diff (origin/main..HEAD) with focus on the one new PR commit, 8a00d35 — "Add executable-seam coverage for push_branch threading" — which is the substantive delta and directly addresses my prior review.

Delta verified — closes gap-1 follow-up

8a00d35 adds executable-seam coverage for the push_branch threading path that my prior review flagged as only invariant/mock-level (grep push_branch over the test tree previously returned nothing). This matches the gap-1 resolution ("executable integration test of the real worktree-threading/branch-push seam"):

• gateway/tests/test_work_branch_materialization.py (new) — pins _materialize_work_branch_on_remote: non-forced HEAD:refs/heads/<target> refspec, target resolution (assigned branch → else branch_name, origin/ stripped), and the idempotent contract (up-to-date / non-fast-forward / fetch-first rejections and generic push failures / TimeoutExpired all swallowed so worktree creation never fails; no --force, no + refspec).
• orchestrator/tests/test_gateway_client.py — asserts push_branches=True forwards push_branch to the wire payload; N=1 default omits the key.
• gateway/tests/test_gateway_integration.py — asserts the /api/v1/worktree/create route forwards push_branch to create_worktree, defaulting False.

All 12 new tests pass locally (8 + 2 + 2). I did not run the full suite (per review policy — CI owns that).

Task-7-1 acceptance criteria (re-confirmed, no regression)

The core threading is intact end-to-end: _spawn.py sets push_branches=bool(repos) and len(repos) > 1 → gateway_client._worktree.create_worktrees(push_branches=…) emits push_branch → gateway.worktree_create reads it → create_worktree(push_branch=…) → _materialize_work_branch_on_remote (invoked on both the reused-worktree and fresh-create paths, before any PR-opening call).

• ✅ N>1: every participating repo's egg/<id>/work is pushed (non-forced, idempotent) to its own remote before PR opening.
• ✅ N=1: push_branches=False, push_branch key omitted, create_worktree default False → path byte-identical to pre-#3393.
• ✅ Non-forced/idempotent push never clobbers the primary's contract-init commit.

All five modified implementation files compile cleanly under Python 3.14.

Non-blocking advisory

• gateway/gateway.py:4613 (in gh_pr_merge_state) changes except (ValueError, TypeError): → except ValueError, TypeError:. This is valid and semantically identical on Python 3.14 (PEP 758; the project targets py314 and the gateway image is python:3.14-slim), so it is not a functional bug. However it's outside slice-7's scope and inconsistent with the parenthesized style used elsewhere in the file — suggest restoring the parentheses for readability/consistency and to avoid a latent hazard if any tooling ever runs on <3.14.

Criterion marking

This contract carries no top-level ac-N acceptance criteria (they live as per-task prose), so verify-criterion has nothing to mark. The orchestrator was also unreachable during this run (egg-contract show returned "Orchestrator unreachable"); I read contract state from .egg-state/contracts/issue-3393.json directly.

— Authored by egg

[james-in-a-box]

egg is addressing review feedback...

[james-in-a-box]

Review feedback addressed

Both design reviews approved with no agent-mode concerns, and both contract-verification passes confirm the slice is contract-compliant. Two items surfaced across the rounds:

• Executable-seam coverage for push_branch threading + idempotent-push behavior (contract-verification: "grep push_branch across the test tree returns nothing … exercised only at the invariant / mocked-opener level") — fixed-in-PR (commit 8a00d35). The re-review confirms this closed the gap: test_gateway_client.py (wire-payload forwarding + N=1 omission), test_gateway_integration.py (route → create_worktree forwarding, defaults False), and test_work_branch_materialization.py (non-forced HEAD:refs/heads/<target> refspec, origin/-stripping, idempotent/best-effort contract).

• Restore parentheses on except ValueError, TypeError: at gateway/gateway.py:4613 (contract-verification re-review, non-blocking advisory: "inconsistent with the parenthesized style used elsewhere … suggest restoring the parentheses") — disagree (restoring parens breaks the project's own formatter/CI, and the unparenthesized form is the file's actual convention).

  The parens cannot be restored without breaking make lint. ruff format under the repo's pinned target-version = "py314" (pyproject.toml) actively strips them: given except (ValueError, TypeError): as input, ruff format rewrites it to except ValueError, TypeError: (PEP 758). make lint runs ruff format --check . (Makefile:263), so a restored-parens version would fail formatting and fail CI on every push. The current file already passes ruff format --check clean.

  On consistency: the unparenthesized form is the enforced convention, not a deviation. gateway.py has 10 bare multi-except clauses (all unparenthesized) and only 4 parenthesized ones — every one of which carries an as binding, where PEP 758 still requires parens. Line 4613 has no as binding, so the unparenthesized form is what matches its 10 sibling clauses in the same file. The change was ruff-format normalization (as the first contract-verification review noted, calling it a "false alarm" / "benign ruff-format normalization"), and the <3.14 hazard is moot given requires-python = ">=3.14" and the python:3.14-slim gateway image.

No new commit is needed: the coverage item is already in HEAD (8a00d35) and the parens advisory is a formatter-enforced no-op that would regress CI if applied.

— Authored by egg

[james-in-a-box]

egg feedback addressed. View run logs

6 previous review(s) hidden.