Migrate claude --print to Agent SDK - #1088
Conversation
Create shared/egg_agent package wrapping claude_agent_sdk.query() for in-process use and build_agent_command() for container command construction. Replace 13 duplicate command-building sites in the orchestrator with a single function call, gut the sandbox runner to delegate to the SDK client, and migrate egg-health-inspect from subprocess to the SDK. Issue: #1080
This comment has been minimized.
This comment has been minimized.
Autofix tracking{"Lint/Python": 3, "Test/Unit Tests": 1, "Lint/Custom Checks": 1} |
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
…egg-exec-20260314-071628-3143/work
Conflict Resolution SummaryResolved merge conflicts with
Resolution rationale for the spawn logic conflict: Main added a new feature (consensus protocol reminder appended to Verification:
— Authored by egg |
This comment has been minimized.
This comment has been minimized.
There was a problem hiding this comment.
No agent-mode design concerns.
The consolidation into egg_agent follows the "shared infrastructure" good pattern from the design guide. The orchestrator continues to use claude --print via container commands (build_agent_command), preserving the security boundary. The SDK client (claude_agent_sdk.query()) is only used inside sandbox containers — it's the programmatic equivalent of claude --print, not a direct API call. Model aliases used throughout.
— Authored by egg
This comment has been minimized.
This comment has been minimized.
There was a problem hiding this comment.
Review: Migrate claude --print to Agent SDK
Clean, well-scoped refactoring that eliminates 13 duplicate command-building blocks and replaces subprocess-based Claude invocation with the Agent SDK. Net -91 lines is good. No blocking issues found.
Issues
1. runner.py wrapper silently drops new AgentResult fields — sandbox/llm/claude/runner.py:60-68
llm.result.AgentResult is now a re-export of egg_agent.result.AgentResult — they are the same class. But the wrapper reconstructs a new instance copying only the original 6 fields, silently discarding cost_usd, num_turns, duration_ms, and session_id:
# Current — drops 4 fields
return AgentResult(
success=sdk_result.success,
stdout=sdk_result.stdout,
stderr=sdk_result.stderr,
returncode=sdk_result.returncode,
error=sdk_result.error,
metadata=sdk_result.metadata,
)Since both types are the same class, just return the SDK result directly:
return sdk_resultOr if you want to keep the explicit mapping for documentation purposes, include the new fields too.
2. Dead stderr_parts variable — shared/egg_agent/client.py:84
stderr_parts: list[str] = [] is initialized but never appended to. It's referenced in the TimeoutError handler and success path as "".join(stderr_parts), which always evaluates to "". Remove the variable and use "" directly.
3. Docstring claims python3 -m egg_agent but code builds claude command — shared/egg_agent/command.py:1-6,21-23
Module docstring says:
"run
python3 -m egg_agent(or theclaudeCLI as a fallback)"
Function docstring says:
"Inside the container this is executed as
python3 -m egg_agent ..."
The actual code builds a claude CLI command. The docstrings describe behavior that doesn't exist yet. Fix to match reality or add a note that this is planned future behavior.
4. Missing noqa: E402 on mid-file import — orchestrator/routes/pipelines.py:145
The from egg_agent import build_agent_command import is placed after the Blueprint(...) definition at line 142, making it a module-level-import-not-at-top. The adjacent import at line 146 already has # noqa: E402. If the linter is configured to check this, the new import needs the same annotation.
Observations (no action required)
- The
allowed_toolsremoval fromClaudeConfigis safe — confirmed unused anywhere. - All private functions removed from
runner.py(_read_lines_unbuffered,_classify_error, etc.) are confirmed to have no external consumers. - The
max_turnsvalues at all call sites match the old inline commands exactly (200 default, 100 for checker, 50 for reviewer). egg-health-inspectmigration from subprocess torun_agent()is clean. The retry logic correctly handles the new error surface.
— Authored by egg
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
There was a problem hiding this comment.
No agent-mode design concerns with the new changes.
The delta since my last review is a single commit addressing code review feedback (dropped fields fix, dead code removal, docstring corrections, lint suppression). None of these affect agent-mode design — the architecture reviewed and approved previously is unchanged.
— Authored by egg
This comment has been minimized.
This comment has been minimized.
There was a problem hiding this comment.
Re-review: Feedback Addressing Commit
All four issues from the previous review are properly fixed:
1. runner.py dropped AgentResult fields — Fixed. Returns sdk_result directly instead of reconstructing with only 6 fields. Since llm.result.AgentResult is a re-export of egg_agent.result.AgentResult (same class), this is correct and now preserves cost_usd, num_turns, duration_ms, and session_id.
2. Dead stderr_parts variable — Fixed. Variable removed, "".join(stderr_parts) replaced with "" in both the TimeoutError handler and the success return path.
3. Inaccurate docstrings in command.py — Fixed. Module and function docstrings now accurately describe "claude CLI" instead of claiming python3 -m egg_agent.
4. Missing noqa: E402 — Fixed. pipelines.py:145 now has the annotation, consistent with the adjacent import.
No new issues introduced. The changes are mechanical and well-scoped.
— Authored by egg
This comment has been minimized.
This comment has been minimized.
| egg is addressing review feedback... |
|
egg feedback addressed. View run logs 18 previous review(s) hidden. |
The egg_agent shared module was added in #1088 but the orchestrator Dockerfile was not updated to copy it into the container image, causing an ImportError crash loop on startup. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…#1100) * Fix orchestrator crash: add missing egg_agent module to Dockerfile The egg_agent shared module was added in #1088 but the orchestrator Dockerfile was not updated to copy it into the container image, causing an ImportError crash loop on startup. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * Update Dockerfile comment to include egg_agent in shared modules list * Remove spawn dependency enforcement to allow concurrent agents The dependency blocking added in #1085 prevented agents from spawning until their declared dependencies completed. This is wrong — agents should run concurrently and communicate via the messaging system. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * Fix stale container/agent status after pipeline cancellation When a pipeline is cancelled, cleanup_pipeline() removes Docker containers but the pipeline state store still shows them as "running". This adds _mark_pipeline_records_cancelled() which syncs the persisted state after container cleanup — marking containers as REMOVED, agents as FAILED, and coordinator spawn records as cancelled. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * Fix cancel state sync: reload pipeline before marking records The previous approach mutated the stale pipeline object returned by update_pipeline(), which could overwrite coordinator state changes made between the status update and container cleanup. Now reloads the latest state from the store after containers are killed, ensuring we mark the most up-to-date records and don't clobber concurrent writes. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * Address review feedback: fix error message, rename function, clean up docs * Fix spawned agents missing EGG_ORCHESTRATOR_URL env var container_spawner.py set EGG_PIPELINE_ID but never set EGG_ORCHESTRATOR_URL, causing the sandbox entrypoint to bail out with "Orchestrator mode enabled but missing URL or pipeline_id" (exit 124). The other two spawn paths (sandbox_template.py and routes/pipelines.py) both set it correctly — this was the only one missing it. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * Increase cancel_task MCP endpoint timeout from 30s to 120s Cancellation involves stopping containers which regularly exceeds the default 30s request timeout, causing spurious "timed out" errors. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * Fix test to expect timeout=120 in cancel_task * Align orchestrator URL construction across spawn paths Use ORCHESTRATOR_EXTERNAL_IP (static IP) instead of ORCHESTRATOR_CONTAINER_NAME (hostname) in container_spawner.py for public mode, matching the pattern in routes/pipelines.py. --------- Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com> Co-authored-by: egg-reviewer[bot] <261018737+egg-reviewer[bot]@users.noreply.github.com>
* Fix orchestrator crash: add missing egg_agent module to Dockerfile The egg_agent shared module was added in #1088 but the orchestrator Dockerfile was not updated to copy it into the container image, causing an ImportError crash loop on startup. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * Update Dockerfile comment to include egg_agent in shared modules list * Remove spawn dependency enforcement to allow concurrent agents The dependency blocking added in #1085 prevented agents from spawning until their declared dependencies completed. This is wrong — agents should run concurrently and communicate via the messaging system. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * Fix stale container/agent status after pipeline cancellation When a pipeline is cancelled, cleanup_pipeline() removes Docker containers but the pipeline state store still shows them as "running". This adds _mark_pipeline_records_cancelled() which syncs the persisted state after container cleanup — marking containers as REMOVED, agents as FAILED, and coordinator spawn records as cancelled. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * Fix cancel state sync: reload pipeline before marking records The previous approach mutated the stale pipeline object returned by update_pipeline(), which could overwrite coordinator state changes made between the status update and container cleanup. Now reloads the latest state from the store after containers are killed, ensuring we mark the most up-to-date records and don't clobber concurrent writes. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * Replace implicit consensus READY with agent restart The consensus wrapper was blindly auto-signaling READY when an agent exited cleanly without participating in consensus. This masked the real problem: the agent didn't follow the consensus protocol. Instead of faking consensus, the wrapper now restarts Claude with a recovery prompt that explains what happened and instructs the agent to assess state and explicitly signal READY or continue working. Restarts are capped at 2 (configurable). After exhausting restarts, the wrapper enters a passive wait loop but does NOT auto-signal READY. Also removes the orchestrator-level implicit READY registration in the concurrent phase poll loop (pipelines.py) — agents must explicitly participate in consensus. Issue: #1081 * Fix checks: apply automated formatting fixes * Exit with failure after max restarts instead of passive wait After exhausting restart attempts, the wrapper now exits with code 1 instead of entering a passive wait loop. There's nothing to wait for — no process inside the container can make progress. The non-zero exit triggers the orchestrator's agent failure path (HITL decision). * Update docs for restart-based consensus wrapper Update concurrent-execution.md to reflect the new behavior: the wrapper restarts the agent with a recovery prompt instead of auto-signaling READY. Remove references to EGG_CONSENSUS_WRAPPER_TIMEOUT (no longer used) and implicit READY (removed from orchestrator). * Address review feedback on consensus restart PR * Fix consensus wrapper to use pipeline status endpoint All consensus checks were querying egg-orch message status, which returns message bus statistics — not consensus data. Changed to egg-orch pipeline status with the correct nested path (data.concurrent.consensus) matching the real API response. Also addressed: - Added MAX_READY_POLL_CYCLES (default 10) as a separate constant for the READY polling loop instead of reusing MAX_RESTARTS (default 2) - Extracted claude mock creation to shared _make_mock_claude() helper - Updated all test mocks to use pipeline status and the correct response structure * Add behavioral test for READY polling path --------- Co-authored-by: James Wiesebron <jameswiesebron@khanacademy.org> Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com> Co-authored-by: egg-reviewer[bot] <261018737+egg-reviewer[bot]@users.noreply.github.com> Co-authored-by: egg <egg@localhost> Co-authored-by: jwbron <8340608+jwbron@users.noreply.github.com>
Summary
All headless Claude Code sessions used
claude --printwith subprocess spawning across 15+ independent sites that constructed identical command lists. This PR consolidates all invocations into a sharedegg_agentpackage.Changes:
shared/egg_agent/package withbuild_agent_command()for container commands andrun_agent_async()/run_agent()wrappingclaude_agent_sdk.query()for in-process executionorchestrator/routes/pipelines.py,coordinator.py, andconcurrent_executor.pywith singlebuild_agent_command()callssandbox/llm/claude/runner.pyto a thin wrapper delegating to the SDK clientegg-health-inspectfromsubprocess.run(["claude", ...])toegg_agent.client.run_agent()_read_lines_unbuffered, version checking,allowed_toolsconfig fieldNot changed: Interactive mode (
os.execvpe), shell alias,shared/egg_contracts/orchestrator.py:AgentResult(different class).Issue: #1080
Test plan:
build_agent_command()output verified to match original command format exactlypython3 -m egg_agent --model sonnet --max-turns 1 "Say hello"works in sandbox containerAuthored-by: egg