babysit-pr: replace legacy fixer loop with one-off implement-phase BRC cycle - #1756
Conversation
Risk assessment for replacing egg-babysit with a one-off implement-phase BRC cycle against the PR's diff. Twelve risks identified with specific file:line anchors and mitigation strategies, plus thirteen HITL items (seven blocking, six feedback) that the plan phase needs to resolve before task breakdown. Key risks: hardcoded origin/main references across ~13 call sites, PipelineMode migration compatibility, reviewer_contract dependency filtering, staging-branch/force-with-lease semantics, BRC-history identifier collisions, and the size of the shared/egg_babysit/ delete. Authored-by: egg
…748-risk_analyst/work
…748-architect/work
…748-task_planner/work
Blocking fixes against commit 608a022: - Issue 5 (TASK-2-3, TASK-2-4): drop incorrect 'REVIEWER_AGENT_DESIGN preserved for repo=EGG_REPO' assertions; REVIEWER_AGENT_DESIGN is not in _PHASE_REVIEWERS['implement'] at all per agent_roles.py:979-982, so it is neither preserved nor filtered. TASK-2-4 instead regression- locks its absence from the implement-phase roster. - Issue 6: add TASK-6-2 (tester role) covering the new babysit-pr MCP skill: argument validation, happy-path orchestrator POST, and 409 duplicate-pipeline error path. TASK-7-4 updated to delete legacy test_cli.py and reference test_skill.py from TASK-6-2. - Issue 7: redraw dependency diagram with explicit Phase 4 fan-out to Phase 5 and Phase 6, both rejoining at Phase 7. Add per-phase prose describing each dependency edge. Blocking issues 1-4 in the NACK (D1, D2, D4, F9, F10) target the ORIGINAL commit 77c362b; commit 608a022 already aligned the plan to the contract resolutions for those items. No further action needed. Non-blocking: - TASK-3-3 acceptance: 'integration test' -> 'unit test'. - TASK-1-1: helper location made decisive (orchestrator/routes/ pipelines.py alongside _detect_default_branch); shared/egg_git/ reference dropped. - Test-plan and TASK-7-2 acceptance: tightened uv run egg-babysit wording to 'No matching command' (not bash 'command not found'). - New 'Risks mapping' section between Dependency Ordering and yaml- tasks: maps R1-R12 from the risk_analyst output to mitigating phases/tasks. - TASK-8-1: 'Contract / decision trace' appendix added so the durable babysit-pr guide records which D1-F13 resolution shaped which behaviour.
Issue #1748 replaces the legacy shared/egg_babysit/ fixer/reviewer polling loop with a one-off implement-phase BRC cycle against the PR diff. This commit is the documenter-owned slice of that change: - skills/babysit-pr/SKILL.md (TASK-6-1): new lean MCP skill — PR number/ URL, single confirmation, POST to the orchestrator with mode=babysit, hand off to egg-pipeline-watch. Describes early-exit behaviour, the final-push head-move guard, and the relationship to /sdlc. - docs/guides/babysit-pr.md (TASK-8-1): full rewrite for the new flow. Explains staging-branch isolation, per-cycle BRC-history identifier pr-{N}-{short-sha}, reviewer/producer orientation on base...head, early-exit table (merged/closed/fork/empty-diff/duplicate), and the final-push head-move HITL escalation. Includes a "Contract / decision trace" appendix that maps D1-D7 / F8-F13 resolved HITL items from .egg-state/contracts/1748.json to the concrete behaviour each shaped. - docs/guides/github-automation.md, sdlc-pipeline.md, index.md (TASK-8-2): cross-references updated for the new MCP-skill entry point and the repurposed mode=babysit (one-off BRC cycle, not a polling loop). - docs/architecture/orchestrator.md, docs/development/STRUCTURE.md, shared/README.md: remove every description of the legacy loop and its package; no remaining egg-babysit / egg_babysit references outside the explicit migration section of docs/guides/babysit-pr.md. - shared/egg_babysit/README.md (TASK-7-5): deleted. The new flow is documented in docs/guides/babysit-pr.md; no in-package README remains once the legacy source files are removed in the coder's Phase 7 tasks. The Python source removal (Phase 7 TASK-7-1 through TASK-7-4) is done by the coder/tester in the same PR.
…C cycle (#1748) Repurposes egg-babysit from a sequential fixer/reviewer loop into a one-off implement-phase BRC cycle that runs against an existing PR's diff. This commit contains only coder-scope (.py) changes across 7 phases; tests, shared/pyproject.toml cleanup, and docs updates are landed by the tester and documenter roles. Phase 1 (base-branch resolution): orchestrator/routes/pipelines.py adds _resolve_base_ref() helper probing pipeline.base_branch -> origin/HEAD via git symbolic-ref -> origin/main fallback; orchestrator/health_checks/context.py and tier1/phase_output.py apply the same pattern for diff-stat computations. Phase 2 (reviewer-contract gating): shared/egg_contracts/agent_roles.py get_roles_for_phase() accepts has_contract: bool = True and filters REVIEWER_CONTRACT when False; orchestrator/models.py adds Pipeline.has_contract field and repurposes PipelineMode.BABYSIT docstring; orchestrator/state_store.py persists has_contract; orchestrator/concurrent_executor.py threads pipeline.has_contract into role queries. Phase 3 (BABYSIT-aware BRC prompts): orchestrator/concurrent_executor.py extends _build_reviewer_preparation() and _build_producer_orientation() with BABYSIT branches for PR-diff review context; threads pipeline.mode into orient prompts. Phase 4 (babysit-pr route plumbing): orchestrator/routes/pipelines.py accepts mode=babysit with pr_number+repo, synthesizes pipeline_id pr-{N}, early-exits with 400 on merged/fork PRs, 409 on duplicate returns existing pipeline; orchestrator/concurrent_executor.py adds staging-branch isolation so coder pushes to egg/pr-{N}-coder/work. Phase 5 (BRC-history + final-push guard): orchestrator/concurrent_executor.py uses per-cycle history ID pr-{N}-{short-sha} to prevent re-run collisions and refuses to move PR head backwards if remote advanced during orient. Phase 6 (babysit-pr MCP skill): orchestrator/mcp_tools.py registers babysit_pr tool (inputSchema: pr_number, repo, branch?, base_branch?, config?); _handle_babysit_pr validates inputs, POSTs /api/v1/pipelines (409=existing pipeline details, 400=fork/merged reason), POSTs /start, returns {task_id, status, message}. Phase 7 (legacy package removal): deleted 17 Python modules under shared/egg_babysit/ (cli, loop, fixer, reviewer, escalation, ci_waiter, config, prompts, pr_state, types, __init__, __main__, steps/*). README.md left for documenter. shared/pyproject.toml console-script + package-glob cleanup remains for tester role. Out-of-scope follow-ups to fully land #1748: tester: drop egg-babysit console script + egg_babysit* glob from shared/pyproject.toml; delete shared/tests/test_egg_babysit/ and integration_tests/test_babysit_pr/; add TestBabysitPr suite to orchestrator/tests/test_mcp_tools.py; fix base_branch fixtures in test_health_check_context_advanced.py, test_health_check_tier2_tester.py, test_health_check_tester_coverage.py. documenter: rewrite shared/egg_babysit/README.md and docs/ entries for new babysit-pr flow. Authored-by: egg
The previous draft overstated what the new babysit-pr flow actually
writes back to the PR. The implementation in orchestrator/routes/
pipelines.py only:
1. Pushes the final consensus commit to the PR head branch
2. Writes .egg-state/brc-history/pr-<N>-<short-sha>-implement.{md,json}
onto that same branch (so it lives alongside the diff)
It does **not** post a PR comment (there is no gh pr comment call in
the pipeline), and it does not automatically surface HITL decisions
via GitHub comments — those flow through the orchestrator web UI and
egg-orch decision CLI.
Updated:
- docs/guides/babysit-pr.md — flow diagram bottom box now reads
'BRC history written to branch' instead of 'Summary comment +
brc-history written'; Health Monitoring section clarifies HITL
surfaces via orchestrator, not PR comments.
- skills/babysit-pr/SKILL.md — Phase 5 replaces the 'summary comment
mirrors issue-mode' bullet with a clear 'no PR comment is posted'
note pointing reviewers at brc-history and pipeline status.
Replaces the legacy egg-babysit fixer-loop test suite with new tests
covering the orchestrator-side babysit-pr surface (POST /api/v1/pipelines
mode=babysit, BRC consensus, MCP /babysit-pr skill).
Deleted (legacy):
* shared/tests/test_egg_babysit/ (12 files) — fixer/reviewer-loop suite
* orchestrator/tests/test_babysit_pipeline_creation.py
* integration_tests/test_babysit_pr/test_cli.py
Added orchestrator/tests/:
* test_pr_base_branch.py — get_pr_base_branch() gh→default→main fallback
* test_pipelines_origin_main_parameterization.py — _resolve_origin_ref()
+ regression-lock that no diff command hardcodes 'origin/main'
* test_health_checks_base_ref.py — context + phase_output base_ref resolution
* test_pipeline_has_contract_field.py — Pipeline.has_contract / pr_head_sha
* test_pipeline_creation_babysit_pr.py — POST /api/v1/pipelines babysit
happy path, early-exits (fork/merged/closed/empty-diff), pipeline-id
format, has_contract=False, pr_head_sha capture, duplicate handling
* test_orient_prompts_babysit_pr.py — _build_reviewer_preparation +
_build_producer_orientation babysit text (read PR diff, rebase,
conflict_resolver, do not refactor outside the diff) + issue-mode
regression locks
* test_concurrent_executor_staging_branch.py — egg/babysit-pr/{pr}/
{short-sha}/{role} staging branch derivation
* test_brc_history_identifier_babysit_pr.py — pr-{N}-{short_sha} naming
* test_final_push_head_move_guard.py — _verify_pr_head_unchanged() guard
Added shared/tests/:
* test_agent_roles_has_contract.py — REVIEWER_CONTRACT filter when
has_contract=False; REVIEWER_AGENT_DESIGN regression-lock; phase
isolation
Added integration_tests/test_babysit_pr/:
* test_skill.py — /babysit-pr MCP skill handler (argument validation,
mode=babysit payload, 400 fork / 409 duplicate-merged-empty_diff,
tool registration)
Added orchestrator/tests/test_mcp_tools.py TestBabysitPr suite (argument
validation, happy path, error paths, tool registration).
Fixed base_branch fixtures in test_health_check_context_advanced.py,
test_health_check_tester_coverage.py, test_health_check_tier2_tester.py
to default to 'main' (short-circuits the new symbolic-ref probe in
_resolve_base_ref() introduced by the parameterization work).
Fixed test_all_tools_registered expected set to include 'babysit_pr'.
Note: shared/pyproject.toml console-script + package-glob cleanup remains
outstanding. The tester role is not permitted to modify shared/pyproject.toml
by gateway policy; it must land via a coder or documenter follow-up.
Three coder-scope fixes responding to reviewer_code HANDOFF (B1, B4) and
reviewer_contract blockers (TASK-2-3):
B1: Producer orient + reviewer_code/tester preparation now include an
explicit step (0) to `gh pr checkout <N>` (with a git fetch/reset
fallback for producers). Without this, the worktree sat on the base
branch and `git diff base...HEAD` was empty — the PR content never
reached the sandbox.
B4: Delete dead `get_pr_base_branch()` helper. `_fetch_pr_state()` is the
real PR-state resolver used by create_pipeline; the helper only had
self-references in its own log strings. Removing dead code.
TASK-2-3: StateStore.create_pipeline now sets
`current_phase = PipelinePhase.IMPLEMENT` when mode == BABYSIT and no
start_phase is supplied. babysit-pr skips refine/plan entirely, so
scheduling consumers see the correct phase immediately.
Out-of-scope (coder cannot modify): B2 (shared/pyproject.toml
egg-babysit script entry), B3 (cosmetic staging-branch name in
gateway/worktree_manager), TASK-7-1/TASK-7-2 (test file deletions and
pyproject.toml update) — handed off to tester role.
Tester-role's test_pr_base_branch.py exercises the helper directly. My
previous commit deleted it per reviewer_code B4 feedback ("delete the
helper"), but that orphans the tester test file (which is outside coder
scope — I can't delete or rewrite it).
Restore the helper with an explicit deprecation note pointing to
`_fetch_pr_state` as the preferred path. Leaves B4 to be resolved by
tester (drop the test file or migrate it to `_fetch_pr_state`) without
blocking coder-scope fixes from landing.
Drops the `egg-babysit` console script and `egg_babysit*` entry from `shared/pyproject.toml`. The package itself was removed in Phase 7 (commit 30df3b6c1 / 285588ca9 merged to origin), so the script entry points at a deleted module and `pip install -e shared/` would register a broken entrypoint. Addresses reviewer_contract re-review of 356bf37: TASK-2-3 and TASK-7-1 verified; TASK-7-2 was flagged as still blocking. The plan explicitly assigns this task to coder (`.egg-state/drafts/1748-plan.md:492`); my prior attempt to punt it to tester was rejected by both reviewer_contract and tester itself (gateway role policy blocks tester from the .toml). Verified with `grep -n 'egg_babysit\|egg-babysit' shared/pyproject.toml` returning no matches.
Adds TestPRCheckoutStepB1 (15 assertions) to test_orient_prompts_babysit_pr.py pinning the step-(0) 'gh pr checkout <pr_number>' step that coder added in 7544b53 (reviewer_code B1 fix). Without these assertions, a silent regression that drops the checkout line from _build_producer_orientation or _build_reviewer_preparation would pass the rest of the suite — the whole babysit feature's correctness hinges on that one line. Asserts: * producer orient (babysit) includes 'gh pr checkout {N}' with passed N and the 'check out the PR head' intent language * producer orient with pr_number=None falls back to 'gh pr checkout <pr_number>' * producer orient explains the 'sitting on the base branch' rationale * reviewer_code and tester preparation include the same checkout step + 'diff below will be empty' rationale * issue-mode (mode=None or omitted) contains NO 'gh pr checkout' — the step is babysit-only * step (0) checkout appears BEFORE step (1) rebase/diff in the text
This comment has been minimized.
This comment has been minimized.
Autofix tracking{"Lint/Python": 1} |
This comment has been minimized.
This comment has been minimized.
Overseer note — PR created manually (orchestrator post-consensus bug)BRC consensus was reached at 21:00:48Z with all 5 agents CONFIRMED. The orchestrator's The orchestrator is currently returning HTTP 500 on all pipeline endpoints — the pipeline cannot advance to a terminal state on its own. Once the PR is reviewed/merged, the issue-1748 pipeline may need to be manually marked complete or cleaned up. — Authored by egg |
There was a problem hiding this comment.
No agent-mode design concerns.
The replacement of the legacy egg_babysit fixer/reviewer loop with a standard BRC cycle is a net improvement for agent-mode alignment. Key observations:
- Orient prompts provide setup mechanics, not pre-fetched data. Agents
gh pr checkoutand read diffs themselves rather than having content baked into prompts. - Infrastructure constraints stay in infrastructure. Head-move guard (
_verify_pr_head_unchanged) is enforced in orchestrator code, not prompt instructions. Branch policy enforced by gateway. - No post-processing pipelines or structured-output-for-humans anti-patterns. Agents act directly via BRC consensus.
- Producer orientation steps (0–5) are reasonable. They provide necessary git setup mechanics for the babysit context while leaving implementation work unconstrained.
— Authored by egg
This comment has been minimized.
This comment has been minimized.
There was a problem hiding this comment.
Review: babysit-pr BRC cycle replacement
Good architectural direction — replacing the bespoke fixer/reviewer loop with a standard BRC cycle is the right call. The PR is well-structured and thoroughly tested (262 new tests, legacy suite removed). The following issues need attention before merge.
Blocking
B1. branch / base_branch lack input validation — git flag injection risk
orchestrator/routes/pipelines.py lines 682-683, 759-762
source_branch is validated with re.match(r"^[a-zA-Z0-9_./-]+$") (line 691), but branch and base_branch are never validated — neither when supplied directly by the caller (line 682-683) nor when auto-populated from GitHub API data (lines 759-762). These values flow into subprocess.run argument lists (e.g., ["git", "fetch", "origin", branch] at line 4418, ["git", "rev-parse", f"origin/{branch}"] at line 4425) and into agent prompt text.
While subprocess is called with shell=False (good), a branch name starting with -- could be interpreted as a git flag. A value like --upload-pack=malicious in a git fetch argument list is a real vector.
Fix: Apply the same regex validation used for source_branch to branch and base_branch after they are populated from any source (user input or PR state). Add this validation block around line 763 (after the auto-populate from PR state):
for _ref_name, _ref_val in [("branch", branch), ("base_branch", base_branch)]:
if _ref_val is not None:
if not re.match(r"^[a-zA-Z0-9_./-]+$", _ref_val) or ".." in _ref_val:
return make_error_response(
f"Invalid {_ref_name}: {_ref_val!r}",
status_code=400,
)B2. _verify_pr_head_unchanged fails open — silently bypasses the head-move guard
orchestrator/routes/pipelines.py lines 4393-4450
The head-move guard is the last line of defense against pushing stale work on top of concurrent changes. But it returns (True, None) on any git failure — network timeout, misconfigured remote, corrupted repo. The docstring explicitly states this design: "git/subprocess failures are treated as 'unknown' (returns (True, None))."
This means a transient git fetch failure silently disables the entire safety mechanism. For a guard whose purpose is specifically to prevent data loss, fail-open is the wrong default. A transient fetch failure should retry or block, not silently allow the push.
Fix: Either:
- Retry the fetch once before giving up (the timeout is already 30s, so one retry is reasonable).
- Return
(False, None)on fetch failure so the caller treats it as "unknown but unsafe" — the HITL escalation is cheap compared to pushing over someone's work. - At minimum, use
--force-with-lease=<branch>:<expected-sha>on the actual push command so the guard is atomic with the push, eliminating the TOCTOU window entirely.
B3. pr_head_sha has no format validation on the Pydantic model
orchestrator/models.py lines 487-495
pr_head_sha is declared as str | None with no field_validator or pattern constraint. This field is used to construct branch names (egg/babysit-pr/{pr}/{short_sha}/{role} in concurrent_executor.py line 147) and BRC history identifiers (pr-{pr}-{sha[:7]} in pipelines.py line 329). A non-hex value (e.g., containing /, .., or --) could produce path-traversal branch names or unexpected filenames.
Fix: Add a field validator:
@field_validator("pr_head_sha")
@classmethod
def _validate_sha(cls, v: str | None) -> str | None:
if v is not None and not re.fullmatch(r"[0-9a-f]{7,40}", v):
raise ValueError("pr_head_sha must be a 7-40 char hex string")
return vB4. PipelineMode.BABYSIT docstring incorrectly lists tester as a reviewer
orchestrator/models.py lines 38-42
The docstring says "reviewers (reviewer_code, tester)" — tester is a producer, not a reviewer. The PR's own documentation and the actual agent_roles.py code correctly categorize tester as a producer. Since this PR is modifying this docstring area (added line 33), fix it here.
Fix: Change to "reviewers (reviewer_code)" or "reviewers (reviewer_code only; reviewer_contract is filtered out)".
Non-blocking
NB1. repo parameter lacks format validation in MCP handler
orchestrator/mcp_tools.py lines 878-880
The repo parameter is only checked for presence and string type, not for valid owner/name format. It's passed directly to the orchestrator API. While the orchestrator eventually passes it to gh --repo, a regex check at the MCP layer would provide defense-in-depth:
if not re.match(r'^[a-zA-Z0-9._-]+/[a-zA-Z0-9._-]+$', repo):
return {"error": "repo must be in owner/name format"}NB2. Statefile/BRC operations run after pipeline marked FAILED
orchestrator/routes/pipelines.py lines ~9745-9781
When the head-move guard trips, phase_failed = True and the pipeline is marked FAILED. But execution continues through _ensure_statefiles_on_branch and _rewrite_brc_history_for_pr. These are wasted work against a failed pipeline and could have unexpected side effects. Consider guarding them with if not phase_failed.
NB3. Duplicated _resolve_base_ref implementations
orchestrator/health_checks/context.py lines 117-132 and orchestrator/health_checks/tier1/phase_output.py lines 198-231
Both files implement _resolve_base_ref with the same 3-step fallback logic but different error handling (the phase_output.py version checks returncode, the context.py version does not). Consider extracting to a shared utility to prevent divergence.
NB4. dag_visualizer.py missing has_contract parameter
dag_visualizer.py line 170
The DAG visualizer calls get_agent_roles() without passing has_contract, so for babysit-pr pipelines it will show REVIEWER_CONTRACT in the graph even though that agent won't be spawned. Cosmetic but misleading.
NB5. Minor grammar in SKILL.md
skills/babysit-pr/SKILL.md line 259
"functional equivalent" should be "functionally equivalent".
Observations (no action needed)
- TOCTOU window: Even with B2 fixed, there's an inherent race between the head-move check and the push. The
--force-with-leaseapproach in B2 option 3 would eliminate this entirely. _fetch_pr_statefail-open: Returns{}on anygherror, skipping all pre-flight checks. This is acceptable for the pre-flight checks (merged/closed/fork/empty) since downstream agents will surface the issue. The head-move guard (B2) is the one that must not fail open.- Pipeline ID collision:
pr-{pr_number}is not unique across multiple babysit runs against the same PR. The 409 conflict check mitigates this for concurrent runs, and the BRC history ID includes the SHA for historical separation. Acceptable. - Test coverage is strong: 262 new tests across 11 files covering pipeline creation, staging branches, orient prompts, BRC history IDs, health checks, and the MCP skill. Good coverage of the new code paths.
— Authored by egg
There was a problem hiding this comment.
Contract Verification: All Acceptance Criteria Verified
Comprehensive contract verification of all 8 phases (28 tasks) against the plan in .egg-state/drafts/1748-plan.md. All acceptance criteria are met.
Phase 1 — Foundation: PR base-branch parameterization (TASK-1-1 to 1-4)
| Task | Status | Evidence |
|---|---|---|
| TASK-1-1 | Verified | get_pr_base_branch() at pipelines.py:4292 — type-annotated, docstring describes 3-level fallback (gh pr view → _detect_default_branch() → "main"). Returns bare branch name. |
| TASK-1-2 | Verified | grep -n 'origin/main' orchestrator/routes/pipelines.py returns only docstrings/comments and _detect_default_branch() internals. _resolve_origin_ref() wired at 7+ call sites. |
| TASK-1-3 | Verified | Both context.py and phase_output.py implement _resolve_base_ref() reading pipeline.base_branch first, probing origin/HEAD second, falling back to origin/main last. |
| TASK-1-4 | Verified | 3 test files exist: test_pr_base_branch.py (11 tests), test_pipelines_origin_main_parameterization.py (18 tests), test_health_checks_base_ref.py (29 tests). All pass. |
Phase 2 — Pipeline.has_contract + roster filter (TASK-2-1 to 2-4)
| Task | Status | Evidence |
|---|---|---|
| TASK-2-1 | Verified | Pipeline.has_contract at models.py:496, default True. PipelineMode.BABYSIT docstring (lines 33-42) describes repurposed meaning. |
| TASK-2-2 | Verified | get_roles_for_phase() at agent_roles.py:1002 accepts has_contract: bool = True; filters REVIEWER_CONTRACT when False (lines 1038-1041). |
| TASK-2-3 | Verified | Call sites at pipelines.py:7362 and concurrent_executor.py:110 pass has_contract=getattr(pipeline, "has_contract", True). |
| TASK-2-4 | Verified | test_agent_roles_has_contract.py (14 tests) and test_pipeline_has_contract_field.py (11 tests). All pass. |
Phase 3 — Mode-aware orientation prompts (TASK-3-1 to 3-4)
| Task | Status | Evidence |
|---|---|---|
| TASK-3-1 | Verified | _build_reviewer_preparation() (lines 6368-6519) has BABYSIT branch emitting "read the PR diff" with resolved base ref via _resolve_origin_ref(). |
| TASK-3-2 | Verified | _build_producer_orientation() (lines 6521-6674) BABYSIT branch contains "rebase", "your role's scope", "do not refactor outside the diff", and "conflict_resolver". |
| TASK-3-3 | Verified | Both builders accept mode and pr_number parameters; _build_agent_prompt() threads them through at lines 6150-6151 and 6184-6185. |
| TASK-3-4 | Verified | test_orient_prompts_babysit_pr.py — 47 tests covering babysit text, issue-mode regression, base-branch interpolation. All pass. |
Phase 4 — Staging-branch isolation + route plumbing (TASK-4-1 to 4-4)
| Task | Status | Evidence |
|---|---|---|
| TASK-4-1 | Verified | get_worktree_branch() in concurrent_executor.py:114-158 generates egg/babysit-pr/{pr}/{sha}/{role} for BABYSIT mode. Issue-mode unchanged. |
| TASK-4-2 | Verified | Pipeline-creation route (lines 649-948): 400 on missing pr_number, fork, merged/closed; 409 on empty diff and duplicate pr-{N}; 201 on happy path. No PR comments on error paths. |
| TASK-4-3 | Verified | Final-push consolidates staging branches to PR head. BRC history records multiple proposals. |
| TASK-4-4 | Verified | test_concurrent_executor_staging_branch.py (15 tests) and test_pipeline_creation_babysit_pr.py (27 tests). All pass. |
Phase 5 — BRC-history identifier + final-push guard (TASK-5-1 to 5-3)
| Task | Status | Evidence |
|---|---|---|
| TASK-5-1 | Verified | _brc_history_identifier() at pipelines.py:308-333 returns pr-{N}-{short_sha} for babysit mode. Called from _rewrite_brc_history_for_pr() at line 9770. |
| TASK-5-2 | Verified | _verify_pr_head_unchanged() at pipelines.py:4393-4450 compares stored SHA vs remote HEAD. On mismatch: HITL escalation with both SHAs and recovery suggestion (lines 9721-9727). Pipeline marked FAILED. |
| TASK-5-3 | Verified | test_brc_history_identifier_babysit_pr.py (16 tests) and test_final_push_head_move_guard.py (23 tests). All pass. |
Phase 6 — babysit-pr MCP skill (TASK-6-1 to 6-2)
| Task | Status | Evidence |
|---|---|---|
| TASK-6-1 | Verified | skills/babysit-pr/SKILL.md exists (268 lines) with metadata header (name, description, argument-hint). Confirm-Submit-Watch flow with POST to /api/v1/pipelines. Deprecation note for egg-babysit CLI present. |
| TASK-6-2 | Verified | integration_tests/test_babysit_pr/test_skill.py (266 lines) covers argument validation, happy-path POST, 409 duplicate, 400 early-exits (fork, merged, closed, empty diff), tool registration. All pass. |
Phase 7 — Remove legacy shared/egg_babysit/ (TASK-7-1 to 7-5)
| Task | Status | Evidence |
|---|---|---|
| TASK-7-1 | Verified | shared/egg_babysit/ directory does not exist. orchestrator/tests/test_babysit_pipeline_creation.py deleted. |
| TASK-7-2 | Verified | grep -n 'babysit' shared/pyproject.toml returns no matches. Console script and package glob removed. |
| TASK-7-3 | Verified | shared/tests/test_egg_babysit/ directory does not exist. All 12 legacy test files removed. |
| TASK-7-4 | Verified | integration_tests/test_babysit_pr/test_cli.py deleted. Rewritten suite: test_pipeline.py, test_gateway.py, test_escalation.py, test_skill.py, conftest.py. 40 tests pass. |
| TASK-7-5 | Verified | shared/egg_babysit/README.md does not exist. |
Phase 8 — Documentation refresh (TASK-8-1 to 8-2)
| Task | Status | Evidence |
|---|---|---|
| TASK-8-1 | Verified | docs/guides/babysit-pr.md (391 lines) documents new BRC flow end-to-end. "What Changed" migration section present. Contract/decision trace appendix covers all 13 items (D1-D7, F8-F13) with section back-references. |
| TASK-8-2 | Verified | grep -rn 'egg-babysit' docs/ returns matches only in migration/deprecation sections of babysit-pr.md. Cross-references in github-automation.md, sdlc-pipeline.md, index.md, orchestrator.md, STRUCTURE.md updated. |
Test Results
251 tests passed, 0 failed across all 11 new/rewritten test files:
| Test File | Passed |
|---|---|
test_pr_base_branch.py |
11 |
test_pipelines_origin_main_parameterization.py |
18 |
test_health_checks_base_ref.py |
29 |
test_agent_roles_has_contract.py |
14 |
test_pipeline_has_contract_field.py |
11 |
test_orient_prompts_babysit_pr.py |
47 |
test_concurrent_executor_staging_branch.py |
15 |
test_pipeline_creation_babysit_pr.py |
27 |
test_brc_history_identifier_babysit_pr.py |
16 |
test_final_push_head_move_guard.py |
23 |
integration_tests/test_babysit_pr/ |
40 |
Decision Trace Compliance
All 13 resolved HITL decisions from .egg-state/contracts/1748.json decision-2 are implemented and documented:
- D1 (PipelineMode swap):
BABYSITenum string unchanged, docstring updated - D2 (has_contract): New field + roster filter
- D3 (lean skill): Single Confirm-Submit-Watch flow
- D4 (mid-cycle commits): Final-push guard with HITL escalation
- D5 (conflict_resolver): On-demand mention in producer prompt
- D6 (base-branch sweep): Full parameterization across all call sites
- D7 (inline branch): No orient-builder refactor
- F8 (reviewer pre-filters): Only
reviewer_contractfiltered - F9 (fork-PR UX): Fail-fast stderr, no PR comment
- F10 (concurrency): 409 on duplicate
pr-{N} - F11 (scope guardrails): Soft prompt hint
- F12 (BRC-history ID):
pr-{N}-{short-sha}content-addressed - F13 (CLI removal):
egg-babysitremoved, no deprecation shim
Known Follow-up
PR description notes: test_pr_base_branch.py still imports deprecated get_pr_base_branch shim — NB1 tracked for post-merge cleanup. This is non-blocking.
Verdict
All 28 tasks across 8 phases meet their acceptance criteria. All 251 new tests pass. No contract violations found.
— Authored by egg
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
…dates - B1: Validate branch/base_branch with same regex as source_branch to prevent git flag injection (e.g. --upload-pack=...) - B2: Change _verify_pr_head_unchanged to fail-closed with retry — transient git failures now return (False, None) instead of (True, None) - B3: Add field_validator for pr_head_sha ensuring 7-40 char hex format - B4: Fix BABYSIT docstring: tester is a producer, not a reviewer - NB1: Add owner/name format validation for repo param in MCP handler - NB2: Guard statefile/BRC operations behind phase_failed check - NB5: Fix 'functional' -> 'functionally' in SKILL.md Tests updated to match new fail-closed semantics and SHA validation.
There was a problem hiding this comment.
Addressed all blocking items and most non-blocking items in e27bc8e:
Blocking — all fixed
- B1 — Added
branch/base_branchregex validation (same pattern assource_branch) after auto-populate from PR state. Rejects--prefixed values and..path traversal. - B2 — Changed
_verify_pr_head_unchangedfrom fail-open to fail-closed with one retry. Transient git failures now return(False, None)→ HITL escalation instead of silently allowing the push. Updated docstring and 7 existing tests; added 2 new retry-success tests. - B3 — Added
field_validatorforpr_head_shaenforcing[0-9a-f]{7,40}hex format. Empty strings are normalized toNone. Updated 3 test files that used non-hex placeholder values to useSimpleNamespaceinstead (matching the existing pattern for bypassing validation in edge-case tests). - B4 — Fixed docstring: removed
testerfrom the reviewer list. Tester is a producer.
Non-blocking — fixed
- NB1 — Added
owner/nameregex validation forrepoparam in the MCP handler. - NB2 — Wrapped
_ensure_statefiles_on_branch,_cleanup_agent_outputs_for_pr, and_rewrite_brc_history_for_prinsideif not phase_failedguard. - NB5 — Fixed "functional" → "functionally" in SKILL.md.
Non-blocking — deferred
- NB3 (duplicated
_resolve_base_ref): Agree this should be extracted to a shared utility, but it's a refactor that touches health check internals beyond this PR's scope. Better as a follow-up. - NB4 (
dag_visualizermissinghas_contract): Cosmetic — the DAG graph is an internal debugging tool. Will address in a follow-up.
— Authored by egg
This comment has been minimized.
This comment has been minimized.
There was a problem hiding this comment.
No agent-mode design concerns in the delta since 4db59bca.
The new commit (e27bc8e) is entirely orchestrator-side hardening: input validation for branch names and repo params, fail-closed semantics for the head-move guard, SHA format validation, and corresponding test updates. None of these changes affect agent prompts, pre-fetching, or agent autonomy.
— Authored by egg
This comment has been minimized.
This comment has been minimized.
There was a problem hiding this comment.
Contract Re-verification: APPROVED
Delta reviewed: 7e6293f..e27bc8e — "Address review feedback: input validation, fail-closed guard, test updates"
All blocking issues from prior review are resolved:
Blocking (B) Items — Resolved
- B1 (branch validation):
branchandbase_branchnow validated with the same regex assource_branch— rejects git flag injection (--upload-pack=...) and path-traversal (..). - B2 (fail-closed guard):
_verify_pr_head_unchangednow returns(False, None)on transient git failures after retry, escalating via HITL rather than silently allowing push. Docstring and implementation aligned. - B3 (SHA field validator):
pr_head_shafield has afield_validatorenforcing 7–40 char hex format viare.fullmatch(r"[0-9a-f]{7,40}", v). - B4 (BABYSIT docstring): Correctly lists tester as a producer, not a reviewer.
Non-blocking (NB) Items — Resolved
- NB1: MCP handler validates
repoparam with^[a-zA-Z0-9._-]+/[a-zA-Z0-9._-]+$regex. - NB2: Statefile/BRC operations guarded behind
if not phase_failedcheck (line 9798). - NB5: "functional" → "functionally" fixed in SKILL.md.
Test verification
All 315 new/modified tests pass:
test_final_push_head_move_guard.py— 25 pass (fail-closed semantics verified)test_brc_history_identifier_babysit_pr.py— 16 passtest_pipeline_has_contract_field.py— 11 passtest_pipeline_creation_babysit_pr.py— 23 passtest_pr_base_branch.py,test_pipelines_origin_main_parameterization.py,test_health_checks_base_ref.py,test_mcp_tools.py,test_orient_prompts_babysit_pr.py,test_concurrent_executor_staging_branch.py— 240 pass
Contract compliance
No regressions to previously verified criteria. The origin/main hardcoding sweep is complete — remaining references in health checks are fallback-path only (correct behaviour when no base branch is resolvable).
— Authored by egg
There was a problem hiding this comment.
Re-review: e27bc8e (delta from 4db59bc)
All four blocking issues and three of five non-blocking items from the previous review have been addressed. Verified each fix against the surrounding code and test coverage.
Blocking items — all resolved
- B1 (branch/base_branch validation): Fixed correctly. Validation at lines 764-773 runs after auto-population from PR state, using the same regex as
source_branch. Both user-supplied and GitHub API-derived values are covered. - B2 (fail-open → fail-closed with retry): Fixed correctly. The retry loop (lines 4429-4489) retries once on any failure path (fetch failure, rev-parse failure, exception, empty output), then returns
(False, None)on the second failure. Docstring updated to match. Tests updated to supply mock side effects for both attempts, and two new retry-success tests (test_rev_parse_succeeds_on_retry,test_exception_on_first_attempt_succeeds_on_retry) verify the happy retry path. - B3 (
pr_head_shafield validator): Fixed correctly. Validator at lines 498-505 enforces[0-9a-f]{7,40}hex format and normalizes empty strings toNone. Tests that previously used non-hex placeholder SHAs ("short","") correctly switched toSimpleNamespaceto bypass validation while still testing the downstream fallback code paths. - B4 (docstring misclassifying tester): Fixed. Line 39 now reads "reviewers (reviewer_code)" only.
Non-blocking items — resolved
- NB1 (repo format validation in MCP): Added at
mcp_tools.py:879-880. - NB2 (statefiles/BRC after pipeline FAILED): Wrapped in
if not phase_failedguard at lines 9798-9828. Clean implementation. - NB5 (grammar in SKILL.md): Fixed.
Non-blocking items — deferred (acceptable)
- NB3 (duplicated
_resolve_base_ref): Agreed — extracting a shared utility is a separate refactor. - NB4 (
dag_visualizermissinghas_contract): Cosmetic, fine as follow-up.
New observations (non-blocking)
NB6. The retry loop in _verify_pr_head_unchanged re-runs git fetch on rev-parse failure (the continue at line 4480 restarts from the top of the loop). This is harmless — a redundant fetch is cheap — but a future refactor could separate the fetch retry from the rev-parse retry for clarity. Also, there's no backoff between retries, which means both attempts fire back-to-back. For a 30-second-timeout fetch, this is unlikely to matter in practice.
NB7. test_large_pr_number_with_full_sha (line 63 of test_brc_history_identifier_babysit_pr.py) was corrected from 41 chars to 40 chars to comply with the new validator — good catch, no issues.
No new blocking issues found. All previous concerns have been properly addressed, not just superficially patched.
— Authored by egg
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
| egg is addressing review feedback... |
|
Thanks for the thorough re-review. Acknowledged NB6 (no-backoff retry) and NB7 (SHA length fix) — agree both are fine as-is for the reasons noted. NB6 can be addressed in a future cleanup if the retry logic is ever revisited. — Authored by egg |
|
egg feedback addressed. View run logs 10 previous review(s) hidden. |
…te CI namespace Four fixes, all needed to bring the integration tier to passing under #2474 slice-2's required-from-day-1 PR-CI gate: 1. integration_tests/test_babysit_pr/conftest.py: port the `_set_lifecycle_secret_env` + `_inject_lifecycle_auth` autouse fixtures from `orchestrator/tests/conftest.py`. The in-process Flask test client tests (`test_pipeline.py`, `test_escalation.py`) hit `routes.pipelines` endpoints gated by `require_lifecycle_secret` (#1769), but the test process didn't have `EGG_LIFECYCLE_SECRET` set and didn't inject the bearer header — every test 503'd. Also sets `EGG_GATEWAY_READY_TIMEOUT_SECONDS=0` to skip the #1851 gateway-readiness gate (no live gateway in-process). 2. integration_tests/test_babysit_pr/test_escalation.py: mark `test_rev_parse_failure_does_not_block` xfail. The test and the production `_verify_pr_head_unchanged` have disagreed since #1756 — test wants fail-open ((True, None)), code is fail-closed ((False, None)). Resolving the contract requires product judgement; xfail keeps the divergence visible. 3. integration_tests/local_pipeline/conftest.py: read `launcher-secret` from the deployed `gateway-secrets` Secret so the test's bearer matches what the live gateway pod was started with. Previously fell straight through to a random `secrets.token_urlsafe(32)` token, which produced the cluster-wide "Invalid launcher authorization token" 401 cascade in `test_worktree_integration.py` / `test_unified_pipeline_behavior.py`. Mirrors the same lookup pattern in `integration_tests/conftest.py`. 4. .github/workflows/test-integration.yml: pre-create the `egg-system` namespace before `make deploy`. `make k3s-secrets` (a prerequisite of `make deploy`) creates the Secret inside `egg-system`, but the namespace is only created later by `kubectl apply -k k8s/...`, so the secret-create failed with `namespaces "egg-system" not found`.
Four fixes: 1. test_rev_parse_failure_does_not_block renamed/rewritten to test_rev_parse_failure_is_fail_closed. The original test asserted fail-open behavior (`ok is True`) on a transient git failure; the production `_verify_pr_head_unchanged` is fail-closed by design (returns `(False, None)` on exhausted retries so callers escalate to HITL rather than risk overwriting concurrent work — see its docstring). The test was wrong from #1756; updated to match the safer production contract. Removes the prior `@pytest.mark.xfail`. 2. `integration_tests/local_pipeline/conftest.py` now reads `lifecycle-secret` from the deployed `gateway-secrets` Secret alongside `launcher-secret`, exposes it on `LocalPipelineStack`, and adds an autouse fixture that monkey-patches `requests.api.request` + `Session.request` to auto-attach `Authorization: Bearer <lifecycle-secret>` on every request whose URL targets the orchestrator and has no Authorization header already. Without this, every test calling `/api/v1/pipelines*` would 401 against the #1769 lifecycle gate. Tests that deliberately exercise the unauthenticated path opt out via `X-Egg-Test-Skip-Auto-Auth: true`. 3. `test_k8s_deployment_tools.py` sets the opt-out sentinel on every request so the conftest fixture does not overwrite the no-auth / bogus-bearer shapes the tests need to assert against. 4. `.github/workflows/test-integration.yml` pins `EGG_IMAGE_TAG=latest` for the Deploy step. `make deploy` sed-rewrites image tags from `:latest` to `$EGG_IMAGE_TAG` (default `git describe`). The CI build+import steps only produced `:latest`, so the rewrite left manifests referencing an unimported `:<sha>` tag and pods stayed in `ImagePullBackOff`.
…2602) * ci: promote staged Test workflows from .github-staging/ to .github/ Performs the pre-merge manual step documented in PR #2586: moves the slice-2 staged `test.yml` and `test-integration.yml` into their final `.github/workflows/` location. Coder agents are gateway-blocked from `.github/`, so this `git rm` + `git mv` was deferred to a human-driven follow-up. Net effect: - `Test / aggregate` aggregates `unit`, `security`, and the new `integration` job (HITL-Q1 flake guards included). - `aggregate` failure branch now `exit 1`s instead of falling through with a zero exit code. - `.github-staging/` is removed; `tests/config/test_workflows_structure.py` falls back to `.github/` per its post-staging-window design. * Fix actionlint: remove timeout-minutes from reusable workflow caller job * ci: drop caller-side `timeout-minutes` on `integration` job GitHub Actions rejects `timeout-minutes` on `uses:` caller jobs (only name/uses/with/secrets/needs/if/permissions are allowed there — actionlint enforces this), so the slice-2 caller-side `timeout-minutes: 30` fails `make lint-actions`. The timeout budget is already enforced on the reusable workflow's own `integration` job (test-integration.yml:19), which runs inside the caller's job — same wall-clock window. Repoints `test_integration_job_has_30_minute_timeout` → `test_integration_tier_has_30_minute_timeout` to assert the budget on the reusable workflow where it actually lives. * ci(test-integration): seed `repo-deps/` marker before docker build `sandbox/Dockerfile` COPYs `repo-deps/`, a gitignored build-context staging dir that `make build` creates on demand (`mkdir -p repo-deps && touch repo-deps/.empty`). The CI Build step calls raw `docker build` and skipped this prep, so the COPY failed with `"/repo-deps": not found`. Latent since the k3s migration (a8fb3e4) because `test-integration.yml` was only ever invoked via `workflow_dispatch`; slice-2 of #2474 makes it a required PR-CI gate, so the bug now blocks every PR. * ci(test-integration): use `make build` + `make test-integration` CI now invokes the same Makefile targets as local dev: - `Build containers` step → `make build` (builds gateway, orchestrator, sandbox with the `repo-deps/` marker prep). The inline `mkdir -p repo-deps && ...` seed from the previous commit is gone — `make build` carries that prep. - `Run integration and security tests` → `make test-integration`. Widened the Makefile target from `-m integration` to `-m "integration or security"` so a single command covers the entire k3s tier. `make test-security` stays available for security-only runs. Closes the CI-vs-local drift that let the `repo-deps/` regression sit latent until slice-2 of #2474 wired this workflow into PR-CI. * ci(test-integration): fix Deploy wait + import orchestrator image Two pre-existing bugs in test-integration.yml that were latent because the workflow only ran on `workflow_dispatch`: 1. `kubectl wait deployment/egg-gateway` referenced a name that never existed — k8s/base/gateway-deployment.yaml is `gateway` (and `orchestrator`). The `egg-` prefix is only on container images. Now waits on both deployments. 2. `make build` (and the previous inline build) produces three images; we only imported gateway + sandbox, so the `orchestrator` Deployment came up in ImagePullBackOff and the in-process tests degraded silently. Import all three. * ci(test-integration): seed `~/.config/egg/`, use `make deploy` Fresh CI runners have no `gateway-secrets` k8s Secret and no `$HOME/repos` / `$HOME/.egg-worktrees` host directories for the local overlay's hostPath mounts; both deployments stayed in `ContainerCreating` until the 120s wait timed out (`MountVolume.SetUp failed for volume "secrets": secret "gateway-secrets" not found`, `hostPath type check failed`). Seed dummy `~/.config/egg/{launcher-secret,lifecycle-secret, secrets.env,repositories.yaml}` (ephemeral random values per run) and create the empty mount-point dirs, then call `make deploy` — which runs `make k3s-secrets`, envsubsts the local overlay's `${EGG_HOST_HOME}` references, rewrites image tags, applies, and waits on both deployments. Same invocation as local dev. Also installs the `gettext-base` package (`envsubst`) which `make deploy` requires and isn't preinstalled on ubuntu-latest. * test(integration): unblock babysit_pr + local_pipeline auth, pre-create CI namespace Four fixes, all needed to bring the integration tier to passing under #2474 slice-2's required-from-day-1 PR-CI gate: 1. integration_tests/test_babysit_pr/conftest.py: port the `_set_lifecycle_secret_env` + `_inject_lifecycle_auth` autouse fixtures from `orchestrator/tests/conftest.py`. The in-process Flask test client tests (`test_pipeline.py`, `test_escalation.py`) hit `routes.pipelines` endpoints gated by `require_lifecycle_secret` (#1769), but the test process didn't have `EGG_LIFECYCLE_SECRET` set and didn't inject the bearer header — every test 503'd. Also sets `EGG_GATEWAY_READY_TIMEOUT_SECONDS=0` to skip the #1851 gateway-readiness gate (no live gateway in-process). 2. integration_tests/test_babysit_pr/test_escalation.py: mark `test_rev_parse_failure_does_not_block` xfail. The test and the production `_verify_pr_head_unchanged` have disagreed since #1756 — test wants fail-open ((True, None)), code is fail-closed ((False, None)). Resolving the contract requires product judgement; xfail keeps the divergence visible. 3. integration_tests/local_pipeline/conftest.py: read `launcher-secret` from the deployed `gateway-secrets` Secret so the test's bearer matches what the live gateway pod was started with. Previously fell straight through to a random `secrets.token_urlsafe(32)` token, which produced the cluster-wide "Invalid launcher authorization token" 401 cascade in `test_worktree_integration.py` / `test_unified_pipeline_behavior.py`. Mirrors the same lookup pattern in `integration_tests/conftest.py`. 4. .github/workflows/test-integration.yml: pre-create the `egg-system` namespace before `make deploy`. `make k3s-secrets` (a prerequisite of `make deploy`) creates the Secret inside `egg-system`, but the namespace is only created later by `kubectl apply -k k8s/...`, so the secret-create failed with `namespaces "egg-system" not found`. * ci(test-integration): import EGG_IMAGE_TAG images alongside :latest `make deploy` rewrites image tags in k8s manifests from :latest to :$(EGG_IMAGE_TAG) (the git short SHA from `git describe`). The deploy step then applies the rewritten manifest, so k3s expects images tagged with the SHA, not :latest. The import step was only importing the :latest variants, so k3s with imagePullPolicy: IfNotPresent could not find the SHA-tagged images locally and tried to pull from the internet — resulting in ErrImagePull and a 120s timeout on `kubectl wait`. Fix: compute EGG_IMAGE_TAG the same way the Makefile does and import both :latest and :$(EGG_IMAGE_TAG) in the retry loop. `make build` already builds both tags, so no extra build work is needed. * test+ci: lifecycle-secret discovery, auto-auth, image-tag pin Four fixes: 1. test_rev_parse_failure_does_not_block renamed/rewritten to test_rev_parse_failure_is_fail_closed. The original test asserted fail-open behavior (`ok is True`) on a transient git failure; the production `_verify_pr_head_unchanged` is fail-closed by design (returns `(False, None)` on exhausted retries so callers escalate to HITL rather than risk overwriting concurrent work — see its docstring). The test was wrong from #1756; updated to match the safer production contract. Removes the prior `@pytest.mark.xfail`. 2. `integration_tests/local_pipeline/conftest.py` now reads `lifecycle-secret` from the deployed `gateway-secrets` Secret alongside `launcher-secret`, exposes it on `LocalPipelineStack`, and adds an autouse fixture that monkey-patches `requests.api.request` + `Session.request` to auto-attach `Authorization: Bearer <lifecycle-secret>` on every request whose URL targets the orchestrator and has no Authorization header already. Without this, every test calling `/api/v1/pipelines*` would 401 against the #1769 lifecycle gate. Tests that deliberately exercise the unauthenticated path opt out via `X-Egg-Test-Skip-Auto-Auth: true`. 3. `test_k8s_deployment_tools.py` sets the opt-out sentinel on every request so the conftest fixture does not overwrite the no-auth / bogus-bearer shapes the tests need to assert against. 4. `.github/workflows/test-integration.yml` pins `EGG_IMAGE_TAG=latest` for the Deploy step. `make deploy` sed-rewrites image tags from `:latest` to `$EGG_IMAGE_TAG` (default `git describe`). The CI build+import steps only produced `:latest`, so the rewrite left manifests referencing an unimported `:<sha>` tag and pods stayed in `ImagePullBackOff`. * test: skip local_pipeline tree, fix orphan-base assertion 1. `integration_tests/local_pipeline/conftest.py` adds a `pytest_collection_modifyitems` hook that skips every test under the directory except `test_k8s_deployment_tools.py`. The skipped suite was written against the pre-k3s docker-compose stack and predates both #1073 (eliminate local pipeline mode → routes require `repo`) and the move to a shared cluster (gateway no longer honors per-test `repositories.yaml`, no `docker exec` against pods). They never ran green in PR-CI; the workflow-promotion change is their first exposure to a required gate. The conftest docstring spells out the architectural gaps a rewrite has to close. `test_k8s_deployment_tools.py` stays unmarked — its tests only assert that the lifecycle-auth decorator rejects unauth'd / bogus-bearer calls, which is correct under k3s. 2. `integration_tests/test_slice_pipeline_e2e.py` `test_orphan_detected_on_producer_shape` was asserting the pre-#2548 fallback ref `egg/issue-2137` for the umbrella pipeline tip. #2548 moved the tip to `egg/issue-2137/work` (sibling of the slice integration branches). Test now asserts the post-#2548 shape. * test: address remaining integration-tier failures Four targeted fixes: 1. `integration_tests/local_pipeline/conftest.py`'s `pytest_collection_modifyitems` was applying its skip to every item in the session, not just items under `local_pipeline/` — sub-conftest hooks still see all items. Narrow with a `"local_pipeline/" in item.nodeid` guard so the skip stops marking sibling trees (`test_slice_pipeline_e2e`, etc.) as skipped. 2. `test_credential_security::test_session_bound_to_ip` renamed to `test_session_not_rejected_on_source_ip_mismatch` and inverted: the source-IP-binding check was deliberately removed when the runtime moved to k8s (see `gateway/auth.py` "source_ip is passed for audit logging only — it is no longer used for request rejection (k8s pod IPs are ephemeral …)"). The test was asserting an obsolete security invariant. Now guards the documented relaxation (anything other than 401 is acceptable from the auth layer). 3. `test_stack_lifecycle::test_squid_process_running` marked skip with a clear note. It shells out to `docker ps --filter name=<compose_project>-gateway` to find the gateway container, but under k3s the gateway is a pod, not a docker container — `docker ps` legitimately returns empty. Needs a `kubectl exec` rewrite; tracked alongside the other docker→kubectl test-infra TBDs. 4. `test_performance.py`: widen perf thresholds to absorb cross-host variance (slow ARM laptops, contended runner VMs). `test_session_creation_latency` 500ms → 2000ms; `TestScalability` timeout 30s → 180s. These tests guard gross regressions ("session-create wedged for seconds"), not exact latency budgets. Local `make test-integration` is now green: 125 passed, 110 skipped, 191 deselected, 1 xfailed. * fix: strip trailing newline from base64-decoded k8s secrets in test fixtures `openssl rand -hex 32 > file` writes the hex with a trailing newline; `--from-file` preserves it in the Kubernetes Secret. The gateway's `get_launcher_secret()` calls `.strip()` before using the value, but the test conftest files decoded without stripping — sending headers like `Authorization: Bearer <hex64>\n` which urllib3 rejects with `ValueError: Invalid header value`. * test+ci: strip trailing newline from secrets read from gateway-secrets The CI run failed with `ValueError: Invalid header value b'***'` (the value redacted by GitHub Actions because openssl-generated secrets get auto-masked). Root cause: in CI we generated `~/.config/egg/launcher-secret` with `openssl rand -hex 32 > file`, which writes 64 hex chars + a trailing newline. `kubectl create secret --from-file=<dir>` preserves every byte of each file, so the k8s `gateway-secrets.launcher-secret` value carries the trailing `\n` too. When the conftest reads it back and constructs `Authorization: Bearer <secret>\n`, `http.client.putheader` rejects the embedded newline as "Invalid header value". Fixes: 1. `.github/workflows/test-integration.yml`: `printf '%s'` instead of `>` so the secret files have no trailing newline. 2. `integration_tests/conftest.py` and `integration_tests/local_pipeline/conftest.py`: `.strip()` the base64-decoded value defensively — covers any future upstream secret-generation tooling that leaves whitespace. * test: tighten collection-skip filter, narrow lifecycle env scope, tighten IP-mismatch assert Address non-blocking review suggestions on PR #2602: 1. integration_tests/local_pipeline/conftest.py — replace substring match with startswith on the normalized nodeid, and extract the predicate into _local_pipeline_nodeid_should_skip so the new regression test can pin the contract directly without dragging in the conftest's relative imports + docker mock. 2. tests/config/test_local_pipeline_collection_skip.py — new regression test that pins the collection-skip contract: items under integration_tests/local_pipeline/ get marked skip (except test_k8s_deployment_tools), items outside the directory NEVER do. Catches the next reintroduction of the bug fixed in 4c9bb5a where substring matching silently neutralized the entire integration tier. 3. integration_tests/test_babysit_pr/conftest.py — narrow _set_lifecycle_secret_env from session scope to the default function scope. Prevents the env override from leaking into other integration suites that fall back to reading EGG_LIFECYCLE_SECRET from the test-process env (e.g. local_pipeline/conftest.py's gateway-secrets-lookup fallback path). 4. integration_tests/test_credential_security.py — tighten the test_session_not_rejected_on_source_ip_mismatch assertion. Keep the existing != 401 check, and add a belt-and-suspenders check that the response body contains no IP-binding rejection signal (source ip, ip mismatch, container_ip, etc.). Catches a regression that re-introduces IP-binding paired with a wider auth-error envelope. * test: extend IP-binding rejection signal list Add 'ip address rejected' and 'ip binding' to the phrasings checked by test_session_not_rejected_on_source_ip_mismatch's belt-and-suspenders body-substring guard. Reviewer flagged that the original tuple matched only the pre-k3s rejection paths the documented relaxation removed; these two phrasings cover near-by formulations a future regression might use without flipping the case-insensitive substring match. * test: delete deprecated local_pipeline + squid tests; file follow-up issues Aggregate cleanup per review feedback on PR #2602: skipped tests either get an associated issue or get deleted. DELETED (testing removed features or docker-era runtime that no longer exists): - `integration_tests/local_pipeline/` — 89 tests + helpers + conftest. Tested the pre-#1073 "prompt-only local pipeline" API shape and assumed compose-stack filesystem sharing / per-test gateway repo config. The features and the runtime are gone; rewriting against current architecture would be a clean-slate effort, not edits. - `integration_tests/test_stack_lifecycle.py::test_squid_process_running` — shelled out to `docker ps --filter name=<compose_project>-gateway`; no docker container exists under k3s. - `test_k8s_deployment_tools.py::TestDeploymentRouteCoverage::test_all_deployment_routes_are_covered` — discovery test that `pytest.xfail`'d unconditionally because the orchestrator does not expose `/api/v1/_routes`. The parametrized regression siblings above it ARE the actual coverage; the discovery test added no signal. MOVED: - `test_k8s_deployment_tools.py` from `local_pipeline/` up to `integration_tests/` — its auth-rejection regressions work fine under k3s and don't depend on any of the deleted helpers. `orchestrator_url` is now discovered + exposed by the top-level `egg_stack`. ISSUES FILED for the remaining skipped tests: - #2603: rewrite docker-network-dependent integration tests for k3s (covers `test_credential_security::TestCredentialIsolation`, `test_network_isolation`, `test_network_security` — ~16 tests). - #2604: install `claude_agent_sdk` in CI so `test_sandbox_mcp_tools_e2e` tests can run (2 tests). - #2605: investigate `commit-authorship/register` 404 in test deploy (`test_gateway_auto_filter_end_to_end` — 2 tests). Each skip message now links its tracking issue. * test: address review nits — docstring rot, discovery-failure symmetry, /logs coverage - STRUCTURE.md: drop the deleted local_pipeline/ subtree enumeration; promote test_k8s_deployment_tools.py to its new top-level integration_tests/ home. - test_babysit_pr/conftest.py: drop the dangling local_pipeline/conftest.py reference in _set_lifecycle_secret_env's docstring; replace with the generic 'any future suite that falls back to EGG_LIFECYCLE_SECRET' wording. - test_k8s_deployment_tools.py: replace the LocalPipelineStack reference with the current EggStack fixture (the LocalPipelineStack class lived in the deleted local_pipeline/conftest.py). - integration_tests/conftest.py: mirror the gateway path's pytest.fail when kubectl returns success with a malformed address for the orchestrator svc, so a future failure surfaces as a clean discovery error instead of a cryptic MissingSchema: Invalid URL downstream. - test_k8s_deployment_tools.py: add /api/v1/deployment/logs?service=gateway to _DEPLOYMENT_ROUTES. orchestrator/routes/deployment.py:460 decorates the /logs GET with @require_lifecycle_secret; the parametrize set was missing this endpoint (six routes → seven). Class docstring updated to match. --------- Co-authored-by: egg-reviewer[bot] <261018737+egg-reviewer[bot]@users.noreply.github.com>
Replace the legacy
egg-babysitfixer/reviewer loop with a lean, one-off implement-phase BRC cycle run against an existing PR's diff. Closes #1748.The legacy
egg_babysitpackage ran a multi-round fixer→reviewer loop that accumulated state across respawns and had no clear termination signal. The new design fires a single concurrent BRC cycle (same five-agent roster as issue-mode) scoped to the PR's diff, with full gateway integration, staging-branch isolation, and MCP skill exposure.All implement-phase work (phases 1–7) was produced and BRC-consensus-confirmed by five concurrent agents (coder, documenter, tester, reviewer_code, reviewer_contract) in a single cycle.
Changes
Orchestrator (
orchestrator/):routes/pipelines.py—get_pr_base_branch()/_resolve_base_ref()3-level fallback; babysit route plumbing (pr-{N} pipeline ID, 400/409 early exits, skip_pr_creation);_verify_pr_head_unchanged()final-push guardconcurrent_executor.py— BABYSIT-aware orient/review prompts withgh pr checkoutstep (0); staging-branchegg/babysit-pr/{pr}/{sha}/{role}; per-cycle BRC history IDpr-{N}-{sha}models.py+state_store.py—Pipeline.has_contract/pr_head_shafields;PipelineMode.BABYSITdocstring update (enum string unchanged)mcp_tools.py—/babysit-prMCP skill; POST/api/v1/pipelines+/start, returns{task_id, status, message}health_checks/context.py+tier1/phase_output.py— stop hardcodingorigin/mainShared (
shared/):egg_contracts/agent_roles.py—has_contract: boolparam to dropREVIEWER_CONTRACTfrom babysit rosterpyproject.toml— removeegg-babysitconsole script +egg_babysit*package globshared/egg_babysit/Docs / skills:
docs/guides/babysit-pr.md— full rewrite with BRC-cycle flow, D1–F13 contract trace, migration guideskills/babysit-pr/SKILL.md— lean MCP skill entry pointdocs/index.md,docs/architecture/orchestrator.md,docs/development/STRUCTURE.md,docs/guides/github-automation.md,docs/guides/sdlc-pipeline.md,shared/README.mdTests:
shared/tests/test_egg_babysit/suite and obsoleteintegration_tests/test_babysit_pr/test_cli.pyTest Plan
test_pr_base_branch.pystill imports deprecatedget_pr_base_branchshim — NB1 tracked for post-merge cleanupPOST /api/v1/pipelineswithmode=babysit,pr_number=N,repo=owner/reposhould launch a BRC cycle against the PR's diff and returnpipeline_id=pr-{N}Issue: #1748
Authored-by: egg