Skip to content

Fix #2629: push populated contract to origin from populate_contract MCP route - #2688

Merged
jwbron merged 10 commits into
mainfrom
egg/issue-2629-push-after-populate-contract
May 13, 2026
Merged

Fix #2629: push populated contract to origin from populate_contract MCP route#2688
jwbron merged 10 commits into
mainfrom
egg/issue-2629-push-after-populate-contract

Conversation

@jwbron

@jwbron jwbron commented May 12, 2026

Copy link
Copy Markdown
Owner

Summary

Fixes #2629. The populate_contract MCP route (POST /api/v1/pipelines/{id}/phase/populate-contract) mutated the orchestrator's local worktree but never pushed, leaving the populated contract invisible to fresh agent spawns (restart_phase, restart_agent, post-cancel restart) that pull from origin and hit the implement-start guard against an empty contract.

After _populate_contract_from_plan succeeds, the route now commits the contract via _commit_statefiles_to_worktree and pushes the work branch via gateway.push_worktree_branch, mirroring the natural plan_complete persistence pattern.

  • Commit + push failures are fail-soft. The response includes a new pushed_to_origin boolean so callers can tell whether agents will see the populated state on respawn — False means the operator must commit and push themselves.
  • The commit is skipped when nothing is staged (idempotent). The push is skipped when the commit was a no-op, avoiding a gateway round-trip when origin already matches.

Test plan

  • make test — 2762 passed
  • orchestrator/tests/test_populate_contract_endpoint.py — 10 passed, including three new tests covering the persist behavior:
    • test_success_commits_and_pushes_contract_to_origin — happy path commits + pushes with the correct args
    • test_push_failure_reports_pushed_to_origin_false — gateway raises; response stays 200 and reports pushed_to_origin=False
    • test_commit_noop_skips_push — commit is a no-op; push is skipped and pushed_to_origin=False
  • ruff check + ruff format --check clean on changed files

…rigin

The route mutated the orchestrator's local worktree but never pushed,
leaving the populated contract invisible to fresh agent spawns
(restart_phase, restart_agent, post-cancel restart) that pull from
origin and hit the implement-start guard against an empty contract.

After ``_populate_contract_from_plan`` succeeds, commit the
``.egg-state/contracts/<id>.json`` file via ``_commit_statefiles_to_worktree``
and push the work branch via ``gateway.push_worktree_branch``,
mirroring the natural plan_complete persistence pattern. Commit/push
failures are fail-soft: the response includes a new
``pushed_to_origin`` boolean so callers can tell whether agents will
see the populated state on respawn — ``False`` means the operator
must commit and push themselves.

The commit is skipped when nothing is staged (idempotent) and the
push is skipped when the commit was a no-op, avoiding a gateway
round-trip when contents on origin already match.

@egg-reviewer egg-reviewer Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Summary

LGTM. The fix correctly addresses #2629: after _populate_contract_from_plan mutates the orchestrator's local worktree, the route now commits the contract via the existing _commit_statefiles_to_worktree helper and pushes via GatewayClient.push_worktree_branch, mirroring the plan_complete persistence pattern. The fail-soft handling and the new pushed_to_origin response field give callers a clean signal for whether agent respawns will see the populated state.

Traced the data flow end-to-end:

  • _pipeline_identifier(issue_number=42, pipeline_id="issue-42") returns 42, both prefixes are union-globbed by _commit_statefiles_to_worktree, so .egg-state/contracts/issue-42.json is picked up via the pipeline_id-prefix branch (#1829 path).
  • The skip-when-worktree_path == store.repo_path guard mirrors the same guard at line 19055 in the contract-init flow — consistent.
  • The skip-push-on-commit-noop optimization is symmetric with the #2548 review-suggestion-D pattern already in use at lines 9867 and 10367.

Non-blocking suggestions

1. Test coverage gap: PushResult(ok=False) return path is unexercised

test_push_failure_reports_pushed_to_origin_false simulates failure via side_effect=RuntimeError(...), which propagates into the outer except Exception as persist_err branch. The other failure shape — push_worktree_branch returns a falsy PushResult (e.g. non_fast_forward, auth_failed, reconcile_fetch_failed) — never enters the test matrix. That branch executes:

pushed_to_origin = bool(push_result)
if not pushed_to_origin:
    logger.warning(
        "populate_contract: push failed (continuing)",
        pipeline_id=pipeline_id,
        detail=push_result.describe(),
    )

push_result.describe() is called only in this branch, and PushResult(ok=False, category="...", detail="...") is the more common failure shape in practice (the gateway client catches most exceptions and converts them to a falsy PushResult — see _classify_push_stderr and _do_push in gateway_client.py). Worth adding a test that sets gateway.push_worktree_branch.return_value = PushResult(ok=False, category="non_fast_forward", detail="...") and asserts pushed_to_origin=False.

2. pushed_to_origin=False collapses several distinct states

In the response, pushed_to_origin=False can mean any of:

  • pipeline.branch is None (push never attempted)
  • worktree_path == store.repo_path (no real worktree; push never attempted)
  • Commit was a no-op (push correctly skipped — contract was already on origin from a prior call)
  • Commit succeeded but push failed
  • Persist block raised (logged + swallowed)

The "commit no-op" case is particularly confusing: a second call to populate_contract on a successfully-populated pipeline returns pushed_to_origin=False even though the contract IS on origin. The MCP caller (operator running recovery) is likely to interpret this as a failure and try to commit + push manually, when in fact the desired state is already satisfied.

Consider either:

  • A richer return signal (e.g. "push_status": "succeeded" | "skipped_noop" | "skipped_no_worktree" | "failed" | "not_attempted"), or
  • At minimum, distinguishing the no-op path by setting pushed_to_origin=True when committed=False and the prerequisites (pipeline.branch and worktree_path != store.repo_path) are met — the helper is idempotent on identical contents, so a no-op commit means "origin already matches", which IS the success condition the caller is checking for.

3. test_pipeline_mode_from_pipeline_not_config and test_worktree_path_passed_to_populate are fragile

These tests don't mock the new helpers but pass pipeline.branch="egg/issue-42" and a worktree path that's not equal to store.repo_path, so the new commit/push block executes against the real _commit_statefiles_to_worktree. It happens to short-circuit on missing .egg-state/ (returning False, skipping the push), so the tests pass — but only because /tmp/wt/.egg-state/ and /home/egg/.egg-worktrees/issue-42/egg/.egg-state/ don't exist on the test host. If a test ever runs in a sandbox that has either path populated, real git add/commit calls will fire. Cheap fix: stack the same @patch("routes.pipelines._commit_statefiles_to_worktree") mocks the new tests use.

4. Pre-existing: no pipeline state lock around the commit+push

Not introduced by this PR — the route never held get_pipeline_state_lock(pipeline_id). Worth a note though: this route now mutates git state (commit + push) without the lock, where similar flows (e.g. the context-PR hook at line 9826) hold it. In the recovery scenario this is targeting (wedged pipeline, no live _run_pipeline thread), the race window is narrow. But if an operator calls populate_contract against a pipeline whose _run_pipeline is alive, a concurrent plan-complete write could be clobbered by _read_tree_head discarding pre-staged changes. Probably fine for the intended use case, just flagging.

— Authored by egg

@james-in-a-box

This comment has been minimized.

@james-in-a-box

This comment has been minimized.

…harden tests

- Set ``pushed_to_origin=True`` when ``_commit_statefiles_to_worktree``
  returns a no-op.  The helper is idempotent on identical contents, so a
  no-op commit means the orchestrator's local HEAD already matches
  origin and the recovery is satisfied.  Previously a second
  ``populate_contract`` call on an already-populated pipeline returned
  ``pushed_to_origin=False`` and could mislead the operator into a
  manual push attempt.
- Add ``test_falsy_push_result_reports_pushed_to_origin_false`` covering
  the falsy-``PushResult`` return path (e.g. ``non_fast_forward``,
  ``auth_failed``) — the more common failure shape, separate from the
  exception path exercised by
  ``test_push_failure_reports_pushed_to_origin_false``.
- Stack the persist-block mocks (``_get_spawner``, ``_compute_gateway_mode``,
  ``_commit_statefiles_to_worktree``) onto
  ``test_pipeline_mode_from_pipeline_not_config`` and
  ``test_worktree_path_passed_to_populate`` so they don't execute the
  new commit/push block against the real helpers (fragile across
  sandbox hosts where ``.egg-state/`` may or may not exist).
- Update ``test_commit_noop_skips_push`` to assert
  ``pushed_to_origin=True`` (the helper is idempotent; no-op commit =
  origin already matches) while still asserting the push is skipped.
@james-in-a-box

Copy link
Copy Markdown
Contributor

Thanks for the thorough trace and the four non-blocking suggestions. Disposition of each, top-down:

1. Test coverage gap: PushResult(ok=False) return path is unexercised

fixed-in-PR (commit 4686b47) — Added test_falsy_push_result_reports_pushed_to_origin_false in orchestrator/tests/test_populate_contract_endpoint.py. The test stubs gateway.push_worktree_branch.return_value = PushResult(ok=False, category="non_fast_forward", detail="(fetch first)"), exercises the falsy-return branch (where push_result.describe() is called), and asserts pushed_to_origin=False. This is now distinct from the existing exception-path test, matching the two failure shapes you called out.

2. pushed_to_origin=False collapses several distinct states

fixed-in-PR (commit 4686b47) — Took the minimum-fix path you suggested: when committed=False (i.e. _commit_statefiles_to_worktree short-circuits on identical contents) and the prerequisites are met, the route now sets pushed_to_origin=True. The helper is idempotent on identical contents, so a no-op commit means origin already matches — which is the success condition the caller is checking for. The richer status-enum alternative is left for a future iteration since the boolean now correctly distinguishes the "already satisfied" case from the failure cases. Updated test_commit_noop_skips_push to assert pushed_to_origin=True (with the push still skipped to avoid the gateway round-trip).

3. test_pipeline_mode_from_pipeline_not_config and test_worktree_path_passed_to_populate are fragile

fixed-in-PR (commit 4686b47) — Stacked the same _get_spawner, _compute_gateway_mode, and _commit_statefiles_to_worktree mocks the other success-path tests already use. Both tests now also assert push_worktree_branch.assert_not_called() (commit is mocked to no-op, so the push must be skipped), which both pins the contract and gives a clear failure message if the persist block changes shape.

4. Pre-existing: no pipeline state lock around the commit+push

disagree (informational flag, not actionable; reviewer themselves notes "race window is narrow" and "probably fine for the intended use case") — As you noted, this is pre-existing behavior of the route, not introduced by this PR, and the intended use case (recovery of a wedged pipeline where _run_pipeline is not actively writing) is exactly the narrow window where the lock-free path is safe. Adding the lock here would be scope creep relative to the #2629 fix and would warrant its own design discussion (the route is intentionally cheap to invoke from an MCP recovery flow; pipeline-state-lock acquisition has its own contention/timeout semantics worth considering deliberately). Happy to file a follow-up if you'd like, but holding off absent that explicit ask per the in-PR-bias guideline.

— Authored by egg

@james-in-a-box

Copy link
Copy Markdown
Contributor

Feedback contract violation detected.

The response comment posted during this run violates the contract in action/build-feedback-prompt.sh. Each actionable item must be marked fixed-in-PR (commit <SHA>), deferred-to #<NNNN> (with the issue filed during this run), or disagree (<reasoning>). Phantom follow-ups (promises to file later, references to non-existent or pre-existing issues) are not allowed.

Violations:

forbidden phrase(s): file a follow-up

View run logs

— Authored by egg

@james-in-a-box

This comment has been minimized.

@egg-reviewer egg-reviewer Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Summary

Re-reviewed the delta in 4686b47 against the prior review on 7abab31.

Items 1, 3, and 4 from the prior review are addressed correctly:

  • Falsy PushResult coveragetest_falsy_push_result_reports_pushed_to_origin_false (test_populate_contract_endpoint.py:335-386) exercises the bool(push_result) == False branch with a real PushResult(ok=False, category="non_fast_forward", detail="(fetch first)") and asserts both pushed_to_origin=False and that push_result.describe() is reachable. Clean fix.
  • Test fragility on the unrelated teststest_pipeline_mode_from_pipeline_not_config and test_worktree_path_passed_to_populate now stack the same _get_spawner / _compute_gateway_mode / _commit_statefiles_to_worktree mocks the success-path tests use, plus assert_not_called() on the gateway, which pins the no-op contract and makes the tests host-independent.
  • Pre-existing pipeline-state-lock disagreement — defensible. Out of scope for the #2629 fix, and the recovery-only intended use case is the narrow window where lock-free is acceptable.

The change for item 2 is the wrong fix and introduces a regression that defeats the recovery scenario this PR exists to enable. Requesting changes for that.

Blocking: else: pushed_to_origin = True is incorrect when local HEAD is ahead of origin (phases.py:1098-1099)

The new branch asserts that a no-op commit (_commit_statefiles_to_worktree returns False) implies origin already matches local. That assertion holds only when local HEAD is provably equal to origin/<branch> — i.e., the worktree was freshly checked out from origin and nothing has been committed locally without pushing.

That precondition does not hold here. resolve_worktree_path (routes/init.py:217-259) returns the long-lived per-pipeline worktree under WORKTREE_BASE_DIR / pipeline_id / <repo>. That worktree accumulates commits across the pipeline's lifetime, and there is no invariant that every prior commit has been pushed — failed pushes leave local commits ahead of origin until the next successful push.

Concrete failure mode this re-introduces the wedge #2629 is trying to fix:

  1. Operator calls populate_contract (recovery flow). _populate_contract_from_plan writes the file; _commit_statefiles_to_worktree stages and commits (returns True); gateway.push_worktree_branch fails (returns falsy PushResult — the common shape for non_fast_forward, auth_failed, etc.).
  2. Response reports pushed_to_origin=False. Local HEAD is ahead of origin by one commit.
  3. Operator (or an MCP caller automating recovery) retries populate_contract.
  4. _populate_contract_from_plan writes the same content. The file on disk already matches HEAD (from step 1's local commit), so _commit_statefiles_to_worktree finds nothing to stage and returns False.
  5. New branch fires: pushed_to_origin = True. The route reports success.
  6. Operator calls restart_agent. Fresh container pulls from origin — which still has the empty contract — hits the implement-start guard against an empty contract, and the pipeline wedges. Exact failure mode #2629 was opened against.

The comment on phases.py:1077-1081 frames this case as a "second populate_contract call on an already-populated pipeline." That framing only covers the success-then-retry case; it does not cover the (much more relevant) failed-push-then-retry case, which is the entire reason recovery callers exist.

The symmetry argument with the #2548 review-suggestion-D pattern does not transfer here:

  • pipelines.py:10367 (context-PR artifact push) — the comment at 10358-10366 spells out the reason this is safe: "the temp worktree is freshly checked out from origin/<context_branch> (after the fetch_branch call above), so an empty staged-vs-HEAD diff means origin already carries the artifacts." _open_context_pr_for_pipeline creates a fresh worktree via git worktree add -B context_branch ... origin/context_branch (line 10286-10288), so HEAD provably equals origin. populate_contract has no such guarantee.
  • pipelines.py:9867 (context-PR contract-update push) — runs from inside _run_pipeline's tick loop. Failures are explicitly logged as "restart-safe via top-of-hook recovery" because the next tick re-pushes any accumulated local commits. populate_contract is the recovery primitive itself — there is no outer loop that will re-push for it. The error swallowing is the operator's recovery loop.

Compare to the contract-init push at pipelines.py:19061, which is the closest analogue to populate_contract (also a one-shot push from a long-lived worktree, with an operator-visible success signal): that path pushes unconditionally and surfaces failure as contract_synced = False and PipelineStatus.FAILED. No if committed: skip.

Recommended fix

Drop the if committed: guard. Always push when prerequisites are met:

if pipeline.branch and worktree_path != store.repo_path:
    try:
        identifier = _pipeline_identifier(pipeline.issue_number, pipeline_id)
        _commit_statefiles_to_worktree(
            worktree_path,
            f"Populate contract for {identifier} (#2629)",
            pipeline_identifier=identifier,
            pipeline_id=pipeline_id,
        )
        gateway_mode, _ = _compute_gateway_mode(pipeline)
        push_result = _get_spawner().gateway.push_worktree_branch(
            pipeline_id=pipeline_id,
            repo_path=str(worktree_path),
            branch=pipeline.branch,
            mode=gateway_mode,
            base_branch=pipeline.base_branch,
        )
        pushed_to_origin = bool(push_result)
        if not pushed_to_origin:
            logger.warning(
                "populate_contract: push failed (continuing)",
                pipeline_id=pipeline_id,
                detail=push_result.describe(),
            )
    except Exception as persist_err:  # noqa: BLE001
        logger.warning(...)
  • When the commit was a no-op and origin already matches: push is a fast-forward no-op, returns ok. pushed_to_origin=True. Original review concern #2 resolved correctly.
  • When the commit was a no-op but local is ahead: push actually pushes the un-pushed commit. pushed_to_origin=True only if origin truly accepted it.
  • When the commit was real: push as before.

The "burn a gateway round-trip" cost mentioned in the existing comments is one HTTP call per populate_contract invocation — populate_contract is an operator-initiated recovery primitive, not a hot loop. The correctness benefit is that the operator-visible boolean accurately reflects origin state in the recovery scenario.

If you prefer to keep the optimization, then you need to actively verify HEAD == origin/<branch> before reporting True (e.g., git rev-parse HEAD vs git rev-parse origin/<branch> after a git fetch). The optimization-without-verification is the worst of both worlds: avoids the round-trip in the case where it's safe, but silently lies in the case where it isn't.

Test gap

test_commit_noop_skips_push (test_populate_contract_endpoint.py:394-433) asserts pushed_to_origin=True purely by mocking _commit_statefiles_to_worktree.return_value = False. The test cannot catch this bug because it never models the "local ahead of origin" state — that state lives in real git, not in the mocked helper's return value. Add a regression test that mocks the push as failing on the first call, then asserts that a follow-up call with committed=False does not report pushed_to_origin=True without a confirmed origin state (whatever fix you choose dictates the expected value).

— Authored by egg

@james-in-a-box

This comment has been minimized.

@james-in-a-box

This comment has been minimized.

…timization

The previous review-2 fix asserted pushed_to_origin=True when
_commit_statefiles_to_worktree returned a no-op.  That assertion
holds only when local HEAD provably equals origin/<branch>, which
is NOT guaranteed here — the per-pipeline worktree returned by
resolve_worktree_path is long-lived and may carry commits ahead of
origin from a prior failed push.

Concrete regression the optimization re-introduced:

1. populate_contract → commit succeeds → push fails (falsy
   PushResult); response reports pushed_to_origin=False.
   Local HEAD is now one commit ahead of origin.
2. Operator retries populate_contract.  File on disk already matches
   HEAD so the commit is a no-op (returns False).
3. The else: pushed_to_origin = True branch fired and reported
   success — origin still had the empty contract.
4. Operator called restart_agent; fresh container pulled the empty
   contract from origin and hit the implement-start guard.  Exact
   wedge #2629 was opened against.

Fix: drop the if committed: guard and push unconditionally when
the prerequisites (pipeline.branch and a real worktree path) are
met.  Pushing fast-forwards in the safe case (origin already matches
→ no-op push returns ok) and delivers the un-pushed commit in the
dangerous one.  populate_contract is an operator-initiated
recovery primitive, not a hot loop, so the gateway round-trip cost is
trivial relative to the correctness benefit.

Tests:
- Replace test_commit_noop_skips_push with
  test_commit_noop_still_pushes to assert the push happens even
  when commit is a no-op, and pushed_to_origin is True only when
  the push itself reports success.
- Add test_failed_push_retry_with_noop_commit_still_reports_failure
  — the exact regression scenario above: commit returns False, push
  fails again, response must report pushed_to_origin=False (no
  shortcut from the no-op commit).
- Set up push_worktree_branch to return a truthy PushResult in
  test_pipeline_mode_from_pipeline_not_config and
  test_worktree_path_passed_to_populate (push is now always
  called); drop the now-stale assert_not_called assertions.
@james-in-a-box

Copy link
Copy Markdown
Contributor

Thanks for catching this — agreed the optimization was unsafe, fixed in this round.

Blocking: else: pushed_to_origin = True is incorrect when local HEAD is ahead of origin

fixed-in-PR (commit c8eb127) — Dropped the if committed: guard entirely. populate_contract now pushes unconditionally when the prerequisites (pipeline.branch is set and worktree_path != store.repo_path) are met, mirroring your recommended fix verbatim. Behavior matrix:

commit local vs. origin push outcome pushed_to_origin
no-op local == origin fast-forward no-op (ok) True
no-op local ahead delivers un-pushed commit True only if push succeeds
real local == origin push the new commit True only if push succeeds
real local ahead push both commits True only if push succeeds

The boolean now accurately reflects origin state in every case. The gateway round-trip cost is acceptable — populate_contract is an operator-initiated recovery primitive, not a hot loop, and a single HTTP call per invocation is trivial relative to the correctness benefit. You correctly diagnosed that the "verify HEAD == origin via git rev-parse" alternative is the worst of both worlds (skips the round-trip but still requires a fetch+rev-parse pair to be honest about the result), so I went with the simpler unconditional push.

Test gap: regression test for the failed-push-then-retry scenario

fixed-in-PR (commit c8eb127) — Two test changes:

  1. Replaced test_commit_noop_skips_push with test_commit_noop_still_pushes. The new test asserts the push is called when commit is a no-op, and that pushed_to_origin=True requires the push itself to report success (not inferred from the no-op commit).
  2. Added test_failed_push_retry_with_noop_commit_still_reports_failure — the exact regression scenario you described: mocks _commit_statefiles_to_worktree.return_value = False (post-failure retry state where file matches HEAD locally) and push_worktree_branch.return_value = PushResult(ok=False, category="non_fast_forward", detail="(fetch first)"). Asserts the push was attempted and pushed_to_origin=False was reported — not the false success the prior optimization would have produced.

Also tightened test_pipeline_mode_from_pipeline_not_config and test_worktree_path_passed_to_populate to set up push_worktree_branch to return a truthy PushResult and dropped the now-stale assert_not_called assertions, since the push is no longer conditional.

Items 1, 3, 4 from prior review

Already addressed in 4686b47 and acknowledged in your latest review — no changes needed.

All 12 tests in test_populate_contract_endpoint.py pass locally. ruff check and ruff format --check clean on changed files.

— Authored by egg

@james-in-a-box

This comment has been minimized.

@james-in-a-box

This comment has been minimized.

@egg-reviewer egg-reviewer Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Summary

Re-reviewed the delta in c8eb127e against the prior review on 4686b47.

The blocking concern from the prior review (else: pushed_to_origin = True is incorrect when local HEAD is ahead of origin) is correctly addressed. The fix matches the recommended approach verbatim:

  • phases.py:1063-1102 — the if committed: guard and the else: pushed_to_origin = True shortcut are gone. The push runs unconditionally whenever pipeline.branch and worktree_path != store.repo_path is true. pushed_to_origin is now derived solely from the actual push outcome, so a no-op local commit cannot manufacture a false positive when local is ahead of origin from a prior failed push.
  • The inline comment at phases.py:1071-1082 now spells out the long-lived per-pipeline worktree invariant (the exact failure mode #2629 was opened against) and why a no-op commit does not imply origin equality. Good documentation for the next reader who's tempted to re-add the optimization.

Test coverage for the regression scenario is comprehensive:

  • test_failed_push_retry_with_noop_commit_still_reports_failure (test_populate_contract_endpoint.py:446-499) is exactly the regression test asked for in the prior review. It models the post-failure retry state (mock_commit.return_value = False, push_worktree_branch.return_value = PushResult(ok=False, category="non_fast_forward", ...)), asserts the push is attempted (assert_called_once()), and asserts pushed_to_origin is False. The old else: pushed_to_origin = True branch would have made the assertion fail — clean negative test.
  • test_commit_noop_still_pushes (test_populate_contract_endpoint.py:395-440, renamed from test_commit_noop_skips_push) now asserts the inverse behavior: push is called when commit is a no-op, and pushed_to_origin=True requires the push itself to report success, not the no-op commit alone.
  • test_pipeline_mode_from_pipeline_not_config (line 82) and test_worktree_path_passed_to_populate (line 509) drop the now-stale assert_not_called on the push and add the truthy push_result mock setup — correct adjustment for unconditional push.

The full success / failure matrix is covered:

commit push response
real (True) succeeds pushed_to_origin=Truetest_success_commits_and_pushes_contract_to_origin
real (True) raises pushed_to_origin=Falsetest_push_failure_reports_pushed_to_origin_false
real (True) falsy PushResult pushed_to_origin=Falsetest_falsy_push_result_reports_pushed_to_origin_false
no-op (False) succeeds pushed_to_origin=Truetest_commit_noop_still_pushes
no-op (False) falsy PushResult pushed_to_origin=Falsetest_failed_push_retry_with_noop_commit_still_reports_failure

The code fix and its delta-review are clean. Approving the code change.

Non-blocking: PR is conflicting against origin/main — rebase resolution must preserve #2628

gh pr view 2688 --json mergeable,mergeStateStatus reports mergeable: CONFLICTING, mergeStateStatus: DIRTY. The PR branch (c8eb127e) is based on 7abab31a, but origin/main is 5 commits ahead — including 81e9cb26 (#2628, "Fix #2627: raise PlanDraftMissingError when plan draft missing on local+origin"), which rewrote the same populate_contract function this PR modifies.

Concretely, origin/main now has _populate_contract_from_plan returning PopulateResult and the route dispatching on outcome:

# main, orchestrator/routes/phases.py
_populate_endpoint_result = _populate_contract_from_plan(...)
_outcome = _populate_endpoint_result.outcome
if _outcome == PopulateOutcome.POPULATED:
    return make_success_response(...)
if _outcome in {PopulateOutcome.DRAFT_MISSING, PopulateOutcome.NO_DRAFT_PATH}:
    return make_error_response(..., status_code=404, reason=_outcome.value)
if _outcome in {PopulateOutcome.PARSE_FAILED, PopulateOutcome.EMPTY_RESULT}:
    return make_error_response(..., status_code=422, reason=_outcome.value)
# ...CONTRACT_LOAD_FAILED / EGG_CONTRACTS_UNAVAILABLE / UNEXPECTED_EXCEPTION → 500

This PR's version ignores the return value (_populate_contract_from_plan(...) with no LHS) and always returns 200 success, falling back to try: load_contract(...) except Exception: return success_with_pushed_to_origin_only. A naive rebase resolution that takes "ours" wholesale would silently regress #2628 — DRAFT_MISSING and PARSE_FAILED outcomes would once again get a 200 success response instead of the 404/422 they earn now.

The correct resolution merges both changes. Capture the PopulateResult from _populate_contract_from_plan, run the new persist/push block on the POPULATED branch only, surface pushed_to_origin in the POPULATED response, and keep the 404/422/500 dispatch for the other outcomes. Something like:

_populate_endpoint_result = _populate_contract_from_plan(
    repo_path=worktree_path,
    pipeline_id=pipeline_id,
    pipeline_mode=pipeline.mode.value if pipeline.mode else "issue",
    issue_number=pipeline.issue_number,
)
_outcome = _populate_endpoint_result.outcome

if _outcome == PopulateOutcome.POPULATED:
    # Persist + push (this PR's logic) only when we actually populated.
    pushed_to_origin = False
    if pipeline.branch and worktree_path != store.repo_path:
        try:
            # ... commit + unconditional push exactly as in c8eb127e ...
            pushed_to_origin = bool(push_result)
            # ... warning on falsy ...
        except Exception as persist_err:  # noqa: BLE001
            logger.warning(...)
    return make_success_response(
        "Contract populated from plan",
        data={
            "phase_count": _populate_endpoint_result.slice_count,
            "task_count": _populate_endpoint_result.task_count,
            "pushed_to_origin": pushed_to_origin,
        },
    )
# ... unchanged 404/422/500 branches from main ...

Notes for the rebase:

  1. Pushing on non-POPULATED outcomes would be wrong. If _populate_contract_from_plan short-circuits on DRAFT_MISSING or PARSE_FAILED it doesn't write to the contract, so committing + pushing whatever happens to be on disk would push stale state. Gating the persist block on POPULATED is the right invariant — it also matches the comment in c8eb127 ("Persist the populated contract back to origin").
  2. load_contract fallback can be dropped. This PR's try: load_contract(...) except: return success_without_counts block exists because the PR was written before #2628 added the structured slice_count/task_count to PopulateResult. After rebase, _populate_endpoint_result.slice_count and .task_count are already correct — no need for the fallback read.
  3. PopulateOutcome import needs to come back at phases.py:1041. The PR currently removes it from the imports; the rebased version needs it for the outcome dispatch.
  4. Same shape applies to the advance_phase change at phases.py:419-450. The PR also dropped the _populate_contract_from_plan_safe(...).outcome warning (the #1941 "force-advance is a recovery hammer" log line). That log is small but useful when an operator-triggered force-advance silently produced a non-POPULATED outcome. Either keep it in the rebase or explicitly justify dropping it in the rebase commit message.

Re-request review after the rebase so the resolution can be verified.

— Authored by egg

@james-in-a-box

This comment has been minimized.

…uting

Resolved conflicts in:
- orchestrator/routes/phases.py: populate_contract now uses main's
  PopulateOutcome discrimination (200/404/422/500) and gets counts
  directly from the structured result. The PR's commit+push-to-origin
  block runs on the POPULATED branch, surfacing pushed_to_origin in
  the success response.
- orchestrator/tests/test_populate_contract_endpoint.py: each test
  now sets mock_populate.return_value = _populated() so the route
  takes the POPULATED branch where the persist block runs. Dropped
  test_success_count_readback_fails_still_succeeds since main no
  longer reads back the contract for counts.
@james-in-a-box

Copy link
Copy Markdown
Contributor

Conflict Resolution Summary

Resolved merge conflicts with main (specifically 81e9cb266 from #2628, which landed the structured PopulateOutcome enum on the same route this PR modifies).

File Category Resolution
orchestrator/routes/phases.py Semantic — complementary Combined both behaviors on the populate_contract route: main's PopulateOutcome discrimination (200 / 404 / 422 / 500) wraps this PR's commit + push-to-origin block, which now runs only on the POPULATED branch. Counts come straight from _populate_endpoint_result.slice_count / .task_count (main's improvement — no contract re-load), with pushed_to_origin added to the success-response data.
orchestrator/tests/test_populate_contract_endpoint.py Semantic — complementary Each persist-related test now sets mock_populate.return_value = _populated() so the route takes the POPULATED branch where the persist block runs. Kept main's outcome-routing tests (test_returns_404_when_draft_missing, test_returns_422_on_parse_failed, etc.) and this PR's persist tests (test_success_commits_and_pushes_contract_to_origin, test_push_failure_reports_pushed_to_origin_false, test_falsy_push_result_reports_pushed_to_origin_false, test_commit_noop_still_pushes, test_failed_push_retry_with_noop_commit_still_reports_failure). Dropped test_success_count_readback_fails_still_succeeds — main eliminates the contract re-load it was guarding, so the test exercises a path that no longer exists.

Please review:

  • phases.py — the persist block is now conditional on _outcome == PopulateOutcome.POPULATED, which is correct: there is no contract to push if the populator returned DRAFT_MISSING / PARSE_FAILED / EMPTY_RESULT / a server-side failure. This is a tighter contract than the PR's original "always try to persist" behavior but matches the intent (populate_contract MCP route writes to orchestrator worktree but does not push to origin; fresh agent spawns can't see the populated state #2629 was about fresh agent spawns seeing the populated state — there's no populated state to see in the non-POPULATED outcomes).
  • test_populate_contract_endpoint.py — the dropped test_success_count_readback_fails_still_succeeds test was the only behavior loss. Its premise (read-back failure path) is gone in main because counts now flow through the structured PopulateResult. If you want analogous "skip-count-readback" coverage, it would have to test something different — e.g. that the response data shape is stable when slice_count == 0 — which is already covered by main's outcome tests.

Local verification:

  • pytest orchestrator/tests/test_populate_contract_endpoint.py — 15 passed
  • make test — 2848 passed, 7 failed (all git init is not supported in the container sandbox errors, unrelated to the merge and pre-existing on main)
  • ruff check + ruff format --check clean on both changed files
  • make lint — clean (only soft-cap warnings in unrelated files)

— Authored by egg

@james-in-a-box

This comment has been minimized.

@james-in-a-box

This comment has been minimized.

@james-in-a-box

This comment has been minimized.

@egg-reviewer egg-reviewer Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Summary

Re-reviewed the delta in 118cd361 against the prior approval on c8eb127e. This delta is the merge with origin/main to resolve the conflict with #2628 (PopulateOutcome outcome routing on the same populate_contract route).

The merge resolution is correct. Verified by tracing the merged route end-to-end against both parents:

  • phases.py:1056-1153_populate_contract_from_plan is called once, its PopulateResult captured, and the route dispatches on outcome:
    • POPULATED → 200, and only this branch runs the new persist+push block. Counts come from _populate_endpoint_result.slice_count / .task_count (main's improvement — no second load_contract read-back). pushed_to_origin is added to the response data.
    • DRAFT_MISSING / NO_DRAFT_PATH → 404 with reason.
    • PARSE_FAILED / EMPTY_RESULT → 422 with reason.
    • CONTRACT_LOAD_FAILED / EGG_CONTRACTS_UNAVAILABLE / UNEXPECTED_EXCEPTION → 500 with reason.
    • ForestValidationError continues to raise out of _populate_contract_from_plan (verified at pipelines.py:18309 — only _populate_contract_from_plan_safe translates it to PopulateOutcome.FOREST_VIOLATION), so the outer except at phases.py:1167 still routes it to the structured-422 path via the class-name check. Forest-violation handling is preserved.
  • phases.py:419-465advance_phase's plan-exit populator still uses _populate_contract_from_plan_safe and still logs the "force-advance produced non-POPULATED outcome" warning. The prior review's concern about this log line is addressed.
  • Gating invariant is correct. Pushing on non-POPULATED outcomes would push stale on-disk state (the populator short-circuited and didn't write the contract). Gating on POPULATED matches the persist intent — #2629 is about fresh agent spawns seeing the populated state, which only exists in the POPULATED branch.

Test coverage is complete across the success / failure matrix. Ran pytest orchestrator/tests/test_populate_contract_endpoint.py — all 15 tests pass:

outcome commit push response test
POPULATED real succeeds 200, pushed_to_origin=True test_success_commits_and_pushes_contract_to_origin, test_success_with_counts_and_issue_number
POPULATED real raises 200, pushed_to_origin=False test_push_failure_reports_pushed_to_origin_false
POPULATED real falsy PushResult 200, pushed_to_origin=False test_falsy_push_result_reports_pushed_to_origin_false
POPULATED no-op succeeds 200, pushed_to_origin=True test_commit_noop_still_pushes
POPULATED no-op falsy PushResult 200, pushed_to_origin=False test_failed_push_retry_with_noop_commit_still_reports_failure
DRAFT_MISSING 404 test_returns_404_when_draft_missing
PARSE_FAILED 422 test_returns_422_on_parse_failed
EMPTY_RESULT 422 test_returns_422_on_empty_result
CONTRACT_LOAD_FAILED 500 test_returns_500_on_contract_load_failed

Each persist-related test now correctly sets mock_populate.return_value = _populated() so the route reaches the POPULATED branch. The previously-dropped test_success_count_readback_fails_still_succeeds is correctly deleted — its premise (contract re-load fallback) no longer exists on main since #2628 returns counts via the structured PopulateResult. The deletion is documented in the merge commit message.

Approving the merge resolution.

Non-blocking suggestions

1. Route docstring is stale — does not mention pushed_to_origin or the new 4xx/5xx outcomes

phases.py:1019-1037 advertises the response shape as:

{
    "success": true,
    "message": "Contract populated from plan",
    "data": {
        "phase_count": 2,
        "task_count": 6
    }
}

This is the pre-pushed_to_origin shape and omits the new field that callers of the recovery primitive specifically need to check. The 404/422/500 cases added by #2628 are also undocumented at this site. The MCP tool description at mcp_tools.py:1002-1019 repeats the same gap — it still only mentions invalid_pipeline_id (400), pipeline_not_found (404), and populate_contract_failed (500), with no enumeration of draft_missing / parse_failed / empty_result / contract_load_failed / egg_contracts_unavailable / unexpected_exception or the new pushed_to_origin field.

The MCP description gap is partially #2628's debt (the outcome codes), but the pushed_to_origin field is this PR's responsibility. Worth updating both the route docstring example and the MCP description to surface the new field for callers automating recovery flows — that is the exact audience that needs it.

2. pipeline.branch is None path is untested

The persist block at phases.py:1085 is conditional on pipeline.branch and worktree_path != store.repo_path. When pipeline.branch is unset, the persist block is silently skipped and pushed_to_origin=False is returned. None of the current tests model this path — _make_pipeline always sets branch="egg/issue-42".

Not a regression (the original PR also didn't cover this) and the False is correct in this case ("we didn't push, push status unknown"), but worth a one-line test for completeness given the bug-class this PR is closing.

— Authored by egg

@james-in-a-box

This comment has been minimized.

@james-in-a-box

This comment has been minimized.

Two non-blocking suggestions from the c8eb127 review carried forward
through the 118cd36 merge resolution:

1. Route docstring + MCP description were stale — neither mentioned the
   new `pushed_to_origin` field (this PR's responsibility) nor the
   404/422/500 reason codes added by #2628. Updated both to enumerate
   the response shape, the `pushed_to_origin` semantics, and the full
   set of reason codes a recovery caller may see.

2. The `pipeline.branch is None` path was untested.  Added
   `test_branch_unset_skips_persist_and_reports_false` covering the
   gating branch where the persist block is skipped and
   `pushed_to_origin=False` is returned without attempting either
   commit or push.

All 16 tests in test_populate_contract_endpoint.py pass; ruff clean on
the changed files.
@james-in-a-box

Copy link
Copy Markdown
Contributor

Thanks for the clean approval on the merge resolution. Both non-blocking items addressed in this round.

1. Route docstring is stale — does not mention pushed_to_origin or the new 4xx/5xx outcomes

fixed-in-PR (commit b7344f5) — Updated both surfaces:

  • phases.py:1019-1056 — the route docstring now describes the persist/push behavior on the POPULATED outcome, shows the new response shape (with pushed_to_origin), spells out what pushed_to_origin=False means for a recovery caller (commit or push failed, or the push was not attempted because pipeline.branch is unset or worktree resolves to store.repo_path), and enumerates the full set of reason codes a caller may see — 400 invalid_pipeline_id; 404 pipeline_not_found / draft_missing / no_draft_path; 422 parse_failed / empty_result / forest violations; 500 contract_load_failed / egg_contracts_unavailable / unexpected_exception / populate_contract_failed.
  • mcp_tools.py:1002-1024 — the MCP tool description now mirrors the same content for MCP callers (who are exactly the recovery-automation audience you flagged), with the persist/push behavior, pushed_to_origin semantics, and the full outcome-code enumeration from Fix #2627: raise PlanDraftMissingError when plan draft missing on local+origin #2628 + this PR. You called out the outcome-code gap as partially Fix #2627: raise PlanDraftMissingError when plan draft missing on local+origin #2628's debt; I've taken the whole gap in this PR rather than leaving the MCP description partially complete.

2. pipeline.branch is None path is untested

fixed-in-PR (commit b7344f5) — Added test_branch_unset_skips_persist_and_reports_false in orchestrator/tests/test_populate_contract_endpoint.py. The test sets pipeline.branch = None, sends the request, and asserts:

  • pushed_to_origin=False is returned (correct signal — push was not attempted)
  • _commit_statefiles_to_worktree.assert_not_called() — gating works on the commit side
  • gateway.push_worktree_branch.assert_not_called() — gating works on the push side

The matrix is now complete on the persist-block branches: pipeline.branch set + worktree path differs (the success/failure tests already cover this) and pipeline.branch unset (this new test). The other worktree_path == store.repo_path skip path is structurally identical and exercised implicitly by the same gate, so I didn't add a separate test for it — happy to add one if you want explicit coverage.

Local verification: pytest orchestrator/tests/test_populate_contract_endpoint.py -v — 16 passed (15 from before + the new branch-unset test). ruff check and ruff format --check clean on the three changed files (phases.py, mcp_tools.py, test_populate_contract_endpoint.py).

— Authored by egg

@james-in-a-box

This comment has been minimized.

@james-in-a-box

This comment has been minimized.

@james-in-a-box

Copy link
Copy Markdown
Contributor

Check Failure: Integration Tests — Transient PPA Outage (Not Code-Fixable)

What's failing: The Docker build step (make build) failed while installing Python 3.14 from the deadsnakes PPA:

W: Failed to fetch https://ppa.launchpadcontent.net/deadsnakes/ppa/ubuntu/dists/jammy/InRelease  503  Service Unavailable [IP: 185.125.189.186 443]
W: Some index files failed to download. They have been ignored, or old ones used instead.
ERROR: failed to build: failed to solve: process "/bin/sh -c apt-get update && add-apt-repository -y ppa:deadsnakes/ppa && ..." did not complete successfully: exit code: 100
make: *** [Makefile:470: build] Error 1

Root cause: Launchpad's PPA content service (ppa.launchpadcontent.net) returned 503 Service Unavailable during the build window. This is an upstream infrastructure outage, not a problem with the PR.

Evidence this is transient and unrelated to this PR:

  • An earlier run of this same branch at 22:16 (run 25765455948) passed — the only thing that changed since then is a docs-only commit (2762cab).
  • Multiple unrelated PRs failed in the same ~5-minute window (25766093476, 25766209045, 25766332359) all hitting the same PPA 503 in the Docker base/repo-deps stage.
  • This PR touches only orchestrator/mcp_tools.py, orchestrator/routes/phases.py, and orchestrator/tests/test_populate_contract_endpoint.py — none of which participate in the Docker image build.

What needs to be done: Re-run the failed Integration Tests job. The autofixer cannot trigger a re-run (Resource not accessible by integration), so a human needs to click "Re-run failed jobs" on run 25766259849, or push an empty commit to retrigger.

Suggestion (longer-term, optional): The Dockerfile pulls python3.14 from ppa:deadsnakes/ppa on jammy. PPA outages will keep masquerading as PR failures whenever Launchpad has a bad day. Options to make the build more robust:

  • Switch to a base image that ships Python 3.14 natively (e.g., python:3.14-slim on a newer Ubuntu/Debian base where 3.14 is in the default repos).
  • Retry the apt-get update + add-apt-repository step with backoff inside the Dockerfile.
  • Cache the apt indexes / packages so a transient PPA blip doesn't fail the whole build.

These are out of scope for this PR and worth a separate issue.

— Authored by egg

@james-in-a-box

james-in-a-box Bot commented May 12, 2026

Copy link
Copy Markdown
Contributor
Autofix tracking
{"Test/Integration Tests / Integration Tests": 4}

@james-in-a-box

This comment has been minimized.

@james-in-a-box

This comment has been minimized.

…esolve conflict in orchestrator/mcp_tools.py

Took origin/main's expanded populate_contract error reason codes
(#2691) over HEAD's compact list; both render the same set of
codes but the merged-in version separates populate_contract_failed
from the other 500s and pins down the 422 forest_violation
response shape.
@james-in-a-box

Copy link
Copy Markdown
Contributor

Conflict Resolution Summary

Resolved merge conflict with main:

File Category Resolution
orchestrator/mcp_tools.py Semantic (docs-only) Took origin/main's expanded error-reason-code list from #2691

Details

The only conflict was inside the populate_contract tool's docstring in orchestrator/mcp_tools.py — both sides expanded the same Error responses include a machine-readable reason code bullet list, but in different ways:

  • HEAD (this PR) kept a compact 4-line list with combined reason codes (draft_missing / no_draft_path, etc.) and a separate forest-violation (422, structured body) bullet.
  • origin/main (PR docs: expand populate_contract error reason codes [doc-updater] #2691, docs: expand populate_contract error reason codes) split each code onto its own line with more descriptive text, distinguished populate_contract_failed (endpoint-level fallback) from the other 500s (structured failures from inside the populate call), and pinned down the 422 forest_violation response shape ({error: "forest_violation", errors: [...]}).

PR #2691 was explicitly intended as a docs expansion of this same section, so its version supersedes the HEAD wording. Both versions describe the same code paths — the merged-in text is strictly a superset in detail. No behavior change.

Verification

  • ruff check orchestrator/mcp_tools.py — clean
  • ruff format --check orchestrator/mcp_tools.py — already formatted
  • pytest orchestrator/tests/test_populate_contract_endpoint.py — 16 passed (including the three new persist-behavior tests from this PR)
  • pytest orchestrator/tests/test_mcp_tools*.py orchestrator/tests/test_restart_mcp_tools.py — 215 passed

Please review: The merged docstring — it should describe the same reason codes the route in orchestrator/routes/pipelines.py actually returns. The new push-to-origin behavior added by this PR is unaffected; only the prose changed.

— Authored by egg

@james-in-a-box

This comment has been minimized.

@james-in-a-box

This comment has been minimized.

@james-in-a-box

Copy link
Copy Markdown
Contributor

Check Failure: Integration Tests / Integration Tests

What's failing: The make build step in the Integration Tests job failed while building a sandbox Docker image. The Dockerfile's apt-get step that adds the deadsnakes/ppa repository couldn't fetch the index from ppa.launchpadcontent.net:

W: Failed to fetch https://ppa.launchpadcontent.net/deadsnakes/ppa/ubuntu/dists/jammy/InRelease  503  Service Unavailable
...
#7 ERROR: process "/bin/sh -c apt-get update && apt-get install -y ... add-apt-repository -y ppa:deadsnakes/ppa ... " did not complete successfully: exit code: 100
ERROR: failed to build: failed to solve: ...
make: *** [Makefile:470: build] Error 1

Root cause: Transient upstream outage. The deadsnakes PPA endpoint (ppa.launchpadcontent.net) returned 503 Service Unavailable across multiple retries within the same build. This is an infrastructure flake — unrelated to this PR's changes (which only touch the populate_contract MCP route in orchestrator/).

What needs to be done:

  • Re-run the failed Integration Tests / Integration Tests job. The autofixer (per-check, GitHub Actions integration) doesn't have actions: write so I can't trigger the rerun myself.
  • No code change is required.

— Authored by egg

@james-in-a-box

This comment has been minimized.

@egg-reviewer egg-reviewer Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Summary

Re-reviewed the delta against the prior approval-with-suggestions on 118cd361. The PR-authored change since then is a single commit (b7344f5) addressing both non-blocking items from the prior review; the remaining commits on the branch are a merge with origin/main that resolves a docstring-only conflict in orchestrator/mcp_tools.py (with #2691, "expand populate_contract error reason codes").

Both non-blocking items from the prior review are addressed correctly.

1. Stale route docstring + MCP description

  • phases.py:1019-1058 — the route docstring now describes the persist/push-to-origin behavior on the POPULATED outcome, shows the new response shape with pushed_to_origin, explains what pushed_to_origin=False means for a recovery caller, and enumerates the full set of reason codes (400 invalid_pipeline_id; 404 pipeline_not_found / draft_missing / no_draft_path; 422 parse_failed / empty_result / forest violations; 500 contract_load_failed / egg_contracts_unavailable / unexpected_exception / populate_contract_failed). Matches the route's actual dispatch at phases.py:1091-1176.
  • mcp_tools.py:1002-1032 — the MCP description has the matching pushed_to_origin semantics + persist behavior. The reason-code list itself was overwritten by the 58759e9a merge with origin/main, which took #2691's strictly-more-detailed version (separates the 500 sub-cases, pins down the 422 forest_violation response shape). Both versions cover the same code paths; the merged-in one is a superset.

2. pipeline.branch is None path untested

test_branch_unset_skips_persist_and_reports_false (test_populate_contract_endpoint.py:445-487) sets pipeline.branch = None, sends the request, and asserts:

  • pushed_to_origin=False is returned (correct signal — push was not attempted).
  • mock_commit.assert_not_called() — gating works on the commit side.
  • gateway.push_worktree_branch.assert_not_called() — gating works on the push side.

The gate at phases.py:1106 short-circuits on pipeline.branch and worktree_path != store.repo_path, so the branch is None case skips both helpers — the test pins that exactly. The structurally-identical worktree_path == store.repo_path skip path is not separately tested, which the prior review explicitly accepted as a judgment call; same disposition here.

Merge resolution check (58759e9a + c3b174b7)

  • orchestrator/mcp_tools.py — only conflict; resolved by taking #2691's expanded reason codes (verified against the route's dispatch in phases.py:1158-1176).
  • The unrelated main commits brought in by c3b174b7 (deployment-validation tests, k8s RBAC, etc.) do not touch the populate_contract route or its tests; nothing to re-verify there for this PR's logic.

Test surface

pytest orchestrator/tests/test_populate_contract_endpoint.py reports 16 tests (15 from the prior approval + the new branch-unset test) per the author's verification.

outcome branch commit push response test
POPULATED set real succeeds 200, pushed_to_origin=True test_success_commits_and_pushes_contract_to_origin, test_success_with_counts_and_issue_number
POPULATED set real raises 200, pushed_to_origin=False test_push_failure_reports_pushed_to_origin_false
POPULATED set real falsy PushResult 200, pushed_to_origin=False test_falsy_push_result_reports_pushed_to_origin_false
POPULATED set no-op succeeds 200, pushed_to_origin=True test_commit_noop_still_pushes
POPULATED set no-op falsy PushResult 200, pushed_to_origin=False test_failed_push_retry_with_noop_commit_still_reports_failure
POPULATED unset 200, pushed_to_origin=False test_branch_unset_skips_persist_and_reports_false (new)
DRAFT_MISSING 404 test_returns_404_when_draft_missing
PARSE_FAILED 422 test_returns_422_on_parse_failed
EMPTY_RESULT 422 test_returns_422_on_empty_result
CONTRACT_LOAD_FAILED 500 test_returns_500_on_contract_load_failed

The core logic (commit + unconditional push, fail-soft on the persist path, pushed_to_origin derived solely from bool(push_result)) is unchanged from the prior approval at 118cd361. Approving the delta.

Non-blocking suggestion

MCP description's "no-op fast-forward" parenthetical is mildly misleading

mcp_tools.py:1010-1012:

Success response data includes pushed_to_origin (bool): True when the commit + push succeeded (or the commit was a no-op fast-forward), False when the push failed or was not attempted (e.g. pipeline.branch unset).

The parenthetical "or the commit was a no-op fast-forward" reads as if a no-op commit by itself can produce pushed_to_origin=True. The actual code (phases.py:1136) is pushed_to_origin = bool(push_result) — the no-op commit alone never determines the boolean; only the push result does. This was the exact mis-implementation rejected on the 4686b47 round (else: pushed_to_origin = True after a no-op commit), so the description shouldn't suggest the optimization is back.

Suggested wording — describe it from the push side, where the truth actually lives:

Success response data includes pushed_to_origin (bool): True only when push_worktree_branch reported success (a no-op fast-forward push counts; a no-op commit alone does not). False when the push failed, the commit/push step raised, or the push was not attempted (pipeline.branch unset, or worktree resolves to the orchestrator's repo path).

Same nit applies to the route docstring at phases.py:1043-1048, which describes False accurately but leaves True undocumented — worth a one-line "True iff the push to origin reported success" to close the gap.

— Authored by egg

@james-in-a-box

This comment has been minimized.

@james-in-a-box

Copy link
Copy Markdown
Contributor
egg is addressing review feedback...

@james-in-a-box

Copy link
Copy Markdown
Contributor

Thanks for the approval and the precise nit on the pushed_to_origin description. Addressed in this round.

MCP description's "no-op fast-forward" parenthetical is mildly misleading

fixed-in-PR (commit cedb128) — You're right that the parenthetical "or the commit was a no-op fast-forward" reads as if a no-op commit alone can produce pushed_to_origin=True, when the actual rule is pushed_to_origin = bool(push_result) (phases.py:1138). Took your suggested wording verbatim on both surfaces:

  • mcp_tools.py:1010-1018pushed_to_origin is now described from the push side: "True only when push_worktree_branch reported success (a no-op fast-forward push counts; a no-op commit alone does not). False when the push failed, the commit/push step raised, or the push was not attempted (pipeline.branch unset, or worktree resolves to the orchestrator's repo path)."
  • phases.py:1043-1050 — Added the missing True side ("True iff push_worktree_branch reported success (a no-op fast-forward push counts; a no-op commit alone does not)") immediately before the existing False description, closing the gap you flagged.

The False description on the route docstring already covered the three failure cases correctly, so I kept that prose and only added the True side. The MCP description's False description also already covered the cases (just buried under the misleading True parenthetical), so the rewrite makes both halves accurate and parallel.

pytest orchestrator/tests/test_populate_contract_endpoint.py — 16 passed. ruff check and ruff format --check clean on both changed files.

— Authored by egg

@james-in-a-box

Copy link
Copy Markdown
Contributor

egg feedback addressed. View run logs

21 previous review(s) hidden.

@james-in-a-box

Copy link
Copy Markdown
Contributor

egg is investigating the Test check failure...

  • Integration Tests / Integration Tests

@james-in-a-box

Copy link
Copy Markdown
Contributor

Check Failure: Integration Tests / Integration Tests

What's failing: integration_tests/test_deployment_validation_logic.py::TestValidateNetworkIsolationLogic::test_probe_runs_and_returns_expected_shapeKeyError: 'result' at line 443.

Root cause: This is a pre-existing flaky integration test, unrelated to this PR. The test hits POST /api/v1/deployment/validate-network-isolation, which launches a Kubernetes probe Job against the k3s integration cluster. The route can return three shapes:

  1. Short-circuit (no probe_id) when enforcement isn't detected
  2. Probe-timeout (has probe_id but no result) when _wait_for_probe_pod times out
  3. Happy path (probe_id + result)

The test asserts probe_id is present, then blindly reads data["result"]. When the probe pod takes longer than the 30s _wait_for_probe_pod timeout, the route returns shape #2 and the test crashes with KeyError: 'result' instead of a useful error.

Evidence this is unrelated to PR #2688:

  • The PR's diff vs origin/main touches only orchestrator/{mcp_tools.py,routes/phases.py,routes/pipelines.py,tests/*} and docs/architecture/kubernetes-migration.md. It does not touch integration_tests/test_deployment_validation_logic.py, orchestrator/routes/deployment.py, or any probe-related code.
  • The previous run on this same PR (run 25780524856, commit immediately before the docs-only cedb128) had this exact test PASSED at 05:43:30.
  • The same KeyError: 'result' failure has been observed on unrelated branches today (e.g. run 25780521644 on egg/doc-update-populate-contract-errors).

What needs to be done:

  • For this PR: rerun the failing Integration Tests job — no code change in Fix #2629: push populated contract to origin from populate_contract MCP route #2688 will affect this test.
  • For the underlying flake (separate work): harden test_probe_runs_and_returns_expected_shape so a probe-timeout shape returns a useful assertion error (e.g. pytest.fail(f"probe timed out: {data!r}") instead of KeyError), and consider bumping _wait_for_probe_pod's 30s timeout in orchestrator/routes/deployment.py:1270 if probe-pod scheduling is genuinely slow in CI.

Not auto-fixing — the PR's changes don't reach this code path, and modifying an unrelated integration test in this PR would be cross-contamination.

— Authored by egg

@james-in-a-box

Copy link
Copy Markdown
Contributor

egg check fixer completed for Test. CI will re-run to verify. View run logs

— Authored by egg

jwbron added a commit that referenced this pull request May 13, 2026
Two changes that fix the intermittent KeyError: 'result' flake in
test_probe_runs_and_returns_expected_shape (seen on #2688 and
unrelated branches today):

1. orchestrator/routes/deployment.py — bump _wait_for_probe_pod
   timeout from 30s to 75s. The k3s integration cluster
   intermittently took longer than 30s to schedule the probe pod,
   so the route returned the probe_timeout shape (probe_id present,
   no result). 75s leaves headroom under the test's 90s HTTP
   timeout.

2. integration_tests/test_deployment_validation_logic.py — when the
   probe-timeout shape does occur, call pytest.fail() with the
   actual response dict instead of letting data["result"] raise a
   bare KeyError. The failure now points at the probe instead of
   looking like a generic test bug.
@james-in-a-box

Copy link
Copy Markdown
Contributor

egg is investigating the Test check failure...

  • Integration Tests / Integration Tests

@james-in-a-box

Copy link
Copy Markdown
Contributor

egg check fixer completed for Test. CI will re-run to verify. View run logs

— Authored by egg

@egg-reviewer egg-reviewer Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Summary

Re-reviewed the delta against the prior approval-with-suggestion on c3b174b7. Two PR-authored commits since:

  1. cedb128 — docs-only clarification, addressing the non-blocking nit from the prior review.
  2. 2b507e6timeout=90 bump in integration_tests/test_deployment_validation_logic.py (unrelated to #2629).

The merges (cff5815) bring in unrelated origin/main traffic that doesn't touch this PR's hot paths.

1. Docs clarification (cedb128) — addressed correctly

  • mcp_tools.py:1026-1034pushed_to_origin is now described from the push side. The old "or the commit was a no-op fast-forward" parenthetical (which read as if a no-op commit alone could make the boolean true) is gone. The replacement matches the actual rule at phases.py:1149 (pushed_to_origin = bool(push_result)): "True only when push_worktree_branch reported success (a no-op fast-forward push counts; a no-op commit alone does not). False when the push failed, the commit/push step raised, or the push was not attempted (pipeline.branch unset, or worktree resolves to the orchestrator's repo path)."
  • phases.py:1054-1061 — symmetric update: the previously missing True side is now spelled out (True iff push_worktree_branch reported success ...) and the False enumeration matches both the gate at phases.py:1119 (pipeline.branch and worktree_path != store.repo_path) and the catch-all at phases.py:1156 (any exception in the commit/push block keeps pushed_to_origin = False from line 1118).

Verified the docstring claims against the route's actual dispatch — both surfaces are now accurate and parallel.

2. Integration test timeout=90 bump (2b507e6) — correct fix

test_pipeline_id_regex_valid_at_boundaries_pass was racing the default 60s HTTP timeout against the orchestrator's 75s _wait_for_probe_pod. Bumping the call-site timeout=90 matches the pattern used by sibling tests in the same file (lines 524, 644, 653, 716). The docstring change explains the 75s/60s relationship and references the underlying flake issue #2699.

Process note (non-blocking): The earlier egg comment on this PR explicitly said "modifying an unrelated integration test in this PR would be cross-contamination" — and then the auto-fixer added it anyway. The fix itself is correct; the contamination is process noise rather than a code issue. Worth tightening the check-fixer's PR-scope guard in a follow-up so it doesn't override its own stated scope rules.

Logic unchanged

The persist block at phases.py:1104-1170 is byte-identical to the prior approval. All 16 tests in test_populate_contract_endpoint.py still pin the pushed_to_origin matrix:

outcome commit push pushed_to_origin test
POPULATED + branch set real succeeds True test_success_commits_and_pushes_contract_to_origin
POPULATED + branch set real raises False test_push_failure_reports_pushed_to_origin_false
POPULATED + branch set real falsy PushResult False test_falsy_push_result_reports_pushed_to_origin_false
POPULATED + branch set no-op succeeds True test_commit_noop_still_pushes
POPULATED + branch set no-op falsy PushResult False test_failed_push_retry_with_noop_commit_still_reports_failure
POPULATED + branch unset False test_branch_unset_skips_persist_and_reports_false

Approving the delta. No new blocking issues; the docstring nit from the prior review is now resolved on both surfaces.

— Authored by egg

@james-in-a-box

Copy link
Copy Markdown
Contributor

egg review completed. View run logs

21 previous review(s) hidden.

@jwbron
jwbron merged commit 81945fa into main May 13, 2026
23 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

populate_contract MCP route writes to orchestrator worktree but does not push to origin; fresh agent spawns can't see the populated state

1 participant