Skip to content

Fix orchestrator crash on host paths in egg-state dirs - #806

Merged
jwbron merged 2 commits into
mainfrom
egg/fix-egg-state-host-path-crash
Feb 17, 2026
Merged

Fix orchestrator crash on host paths in egg-state dirs#806
jwbron merged 2 commits into
mainfrom
egg/fix-egg-state-host-path-crash

Conversation

@james-in-a-box

Copy link
Copy Markdown
Contributor

Fix orchestrator crash creating .egg-state dirs on host paths

The orchestrator receives host-translated worktree paths from the gateway
(e.g. /home/jwies/.egg-worktrees/...), but the orchestrator container
only sees these at /home/egg/... via Docker volume mounts. Two bugs:

  1. ensure_egg_state_dirs() crashed with PermissionError: [Errno 13]
    trying to mkdir on inaccessible host paths
  2. phase_readonly_mounts() silently produced no mounts because
    is_dir() returned False on host paths, leaving .egg-state/
    subdirs writable during implement phase

Adds _host_to_local_volumes() helper to translate host paths using the
existing HOST_HOME env var, and a local_volumes parameter to
phase_readonly_mounts() so it checks local paths for existence while
keeping host paths as Docker mount sources.

Issue: Pipeline issue-732 PermissionError

Test plan:

  • pytest tests/shared/egg_container/test_phase_mounts.py — 32 tests pass (3 new)
  • pytest orchestrator/tests/test_container_spawner.py — 38 tests pass (6 new)
  • New tests cover: host→local translation, prefix-only replacement, passthrough when HOST_HOME matches or is unset, local_volumes for is_dir checks with host mount sources

Authored-by: egg

The orchestrator receives host-translated paths from the gateway (e.g.
/home/jwies/.egg-worktrees/...) but can only access them at /home/egg/
via Docker volume mounts. ensure_egg_state_dirs crashed with
PermissionError trying to mkdir on inaccessible host paths, and
phase_readonly_mounts silently failed because is_dir() returned False.

Add _host_to_local_volumes() to translate host paths using HOST_HOME env
var. Pass translated paths to ensure_egg_state_dirs for filesystem ops
and add local_volumes parameter to phase_readonly_mounts so it uses
local paths for is_dir() checks while keeping host paths for Docker
mount sources.

@egg-reviewer egg-reviewer Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review: Fix orchestrator crash on host paths in egg-state dirs

Reviewed all 4 changed files. The fix is correct and well-scoped — it addresses a real crash (PermissionError) and a silent bug (empty readonly mounts) caused by the orchestrator trying to operate on host-absolute paths it can't access.

Approach

The solution is clean: translate host paths to orchestrator-local paths using the existing HOST_HOME env var, then pass the local paths for filesystem operations while keeping host paths as Docker mount sources. This mirrors the inverse of the gateway's translate_to_host_path() in gateway/gateway.py:249.

Minor issues

1. HOST_HOME with trailing slash
orchestrator/container_spawner.py:109path.replace(host_home, container_home, 1) will produce a double-slash if HOST_HOME has a trailing slash (e.g., /home/jwies//home/egg//...). The gateway's translate_to_host_path() has the same latent issue, so this is consistent with existing behavior, but both could be tightened with host_home.rstrip("/"). Not blocking.

2. Test for "HOST_HOME not set" is unnecessarily convoluted
orchestrator/tests/test_container_spawner.py:626-631 — The double-nested patch.dict with __import__("os") is hard to read. A simpler approach:

with patch.dict("os.environ", {}, clear=True):
    result = _host_to_local_volumes(repo_volumes)

If the test needs to preserve some env vars but remove HOST_HOME, use patch.dict("os.environ", {"HOST_HOME": ""}) since the function already treats empty string as unset. Not blocking — the test is correct, just noisy.

No issues found

  • Security: No path traversal concern — HOST_HOME is set by the deployment operator (via .env / docker-compose), not by user input. The startswith + replace(..., 1) pattern correctly handles the prefix-only replacement.
  • Correctness: ensure_egg_state_dirs now receives local paths it can actually mkdir/chown on. phase_readonly_mounts correctly uses local paths for is_dir() checks while keeping host paths as Docker mount sources. The check_volumes.get(repo_name, host_path) fallback is safe — an inaccessible path just returns is_dir() == False, skipping the mount.
  • Backward compatibility: The local_volumes parameter defaults to None, preserving existing behavior for all callers outside the orchestrator.
  • Test coverage: Good — 9 new tests covering translation, passthrough, prefix-only replacement, multiple repos, and the local_volumes/repo_volumes separation in phase_readonly_mounts.

Looks good to merge.

— Authored by egg

@james-in-a-box

This comment has been minimized.

@james-in-a-box

This comment has been minimized.

@james-in-a-box

Copy link
Copy Markdown
Contributor Author

Addressed both suggestions from the review:

  1. HOST_HOME trailing slash — Added .rstrip("/") to _host_to_local_volumes so a trailing slash on HOST_HOME (e.g. /home/jwies/) no longer produces double-slashes in translated paths. Added a test for this case.

  2. Simplified test_passthrough_when_host_home_empty — Replaced the double-nested patch.dict with the simpler patch.dict("os.environ", {"HOST_HOME": ""}) approach, since the function already treats empty string as unset.

All tests passing (39 container_spawner + 32 phase_mounts).

— Authored by egg

@james-in-a-box

This comment has been minimized.

@egg-reviewer egg-reviewer Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Re-review: Fix orchestrator crash on host paths in egg-state dirs

Reviewed delta 4878b65..df748f0 (1 commit, 2 files changed).

Previous feedback verification

Both suggestions from the initial review have been properly addressed:

1. HOST_HOME trailing slashcontainer_spawner.py:104 now applies .rstrip("/") before any path operations. This correctly prevents double-slashes when HOST_HOME ends with /. Edge cases are safe: HOST_HOME="/" or "///" strip to empty string, which hits the if not host_home early return.

2. Simplified testtest_passthrough_when_host_home_empty replaced the double-nested patch.dict with patch.dict("os.environ", {"HOST_HOME": ""}). Clean and correct — the function treats empty string as unset.

3. New test coveragetest_trailing_slash_on_host_home (line 663) directly covers the trailing-slash scenario.

No new issues found

The changes are minimal and focused. No correctness, security, or design concerns.

Phase mounts tests pass (32/32). Container spawner tests cannot run in this sandbox (missing docker SDK) but the PR author reports 39/39 passing.

— Authored by egg

@james-in-a-box

This comment has been minimized.

@james-in-a-box

Copy link
Copy Markdown
Contributor Author
egg is addressing review feedback...

@james-in-a-box

Copy link
Copy Markdown
Contributor Author

Re-review confirmed no new issues. All previous feedback was already addressed in df748f0. No further changes needed.

— Authored by egg

@james-in-a-box

Copy link
Copy Markdown
Contributor Author

egg feedback addressed. View run logs

4 previous review(s) hidden.

@jwbron
jwbron merged commit b4b1ecd into main Feb 17, 2026
12 checks passed
github-actions Bot pushed a commit that referenced this pull request Feb 17, 2026
Update orchestrator architecture doc to explain the HOST_HOME-based
path translation introduced in #806 and add HOST_HOME to the
environment variables table.

Authored-by: egg
jwbron added a commit that referenced this pull request Feb 17, 2026
)

Update orchestrator architecture doc to explain the HOST_HOME-based
path translation introduced in #806 and add HOST_HOME to the
environment variables table.

Authored-by: egg

Co-authored-by: jwbron <8340608+jwbron@users.noreply.github.com>
james-in-a-box Bot added a commit that referenced this pull request Feb 17, 2026
* Fix orchestrator crash on host paths in ensure_egg_state_dirs

The orchestrator receives host-translated paths from the gateway (e.g.
/home/jwies/.egg-worktrees/...) but can only access them at /home/egg/
via Docker volume mounts. ensure_egg_state_dirs crashed with
PermissionError trying to mkdir on inaccessible host paths, and
phase_readonly_mounts silently failed because is_dir() returned False.

Add _host_to_local_volumes() to translate host paths using HOST_HOME env
var. Pass translated paths to ensure_egg_state_dirs for filesystem ops
and add local_volumes parameter to phase_readonly_mounts so it uses
local paths for is_dir() checks while keeping host paths for Docker
mount sources.

* Address review: strip HOST_HOME trailing slash, simplify test

---------

Co-authored-by: egg <egg@localhost>
Co-authored-by: egg-reviewer[bot] <261018737+egg-reviewer[bot]@users.noreply.github.com>
jwbron pushed a commit that referenced this pull request Feb 17, 2026
* Initialize SDLC contract for issue #732

* Add analysis for issue #732: parallel phase-level dispatch

* Add agent-design review verdict for issue #732 refine phase

* Add refine review verdict for issue #732

* Persist statefiles after refine phase

* Add architect analysis for Tier 3 phase-level dispatch (#732)

* Fix orchestrator crash on host paths in egg-state dirs (#806)

* Fix orchestrator crash on host paths in ensure_egg_state_dirs

The orchestrator receives host-translated paths from the gateway (e.g.
/home/jwies/.egg-worktrees/...) but can only access them at /home/egg/
via Docker volume mounts. ensure_egg_state_dirs crashed with
PermissionError trying to mkdir on inaccessible host paths, and
phase_readonly_mounts silently failed because is_dir() returned False.

Add _host_to_local_volumes() to translate host paths using HOST_HOME env
var. Pass translated paths to ensure_egg_state_dirs for filesystem ops
and add local_volumes parameter to phase_readonly_mounts so it uses
local paths for is_dir() checks while keeping host paths for Docker
mount sources.

* Address review: strip HOST_HOME trailing slash, simplify test

---------

Co-authored-by: egg <egg@localhost>
Co-authored-by: egg-reviewer[bot] <261018737+egg-reviewer[bot]@users.noreply.github.com>

* WIP: auto-commit uncommitted work (architect) [issue-732]

Container 5b0b993dd1ae67f73d4e1d90bf5422e1963957629c1d28d04ac2973b9daaba18 exited with uncommitted changes.
This commit preserves the agent's work-in-progress.

Authored-by: egg

* Add task planner implementation plan for Tier 3 phase-level dispatch (#732)

* Add risk assessment for Tier 3 phase-level dispatch (#732)

* Add plan review verdict for #732

* Persist statefiles after plan phase

* Add Phase/AgentExecution model extensions for Tier 3 dispatch

Add dependencies field to Phase model, phase_id to AgentExecutionModel,
complexity_tier/enable_parallel_phases to PipelineConfig, and complexity_tier
to Pipeline. Update JSON schema with new fields and expanded role enums.
Update plan parser to propagate phase dependencies from parsed plans.

* Add 3-tier complexity assessment to refine phase

Update refine prompt to instruct LLM to signal complexity_tier (low/mid/high)
and parallel_phases in YAML metadata. Add _check_high_complexity_signal()
to parse the tier. Set pipeline.complexity_tier from detected signal during
refine-to-plan transition. Update test to match new prompt format.

* Add composite execution tracking and phase dependency graph

Extend OrchestrationState with (phase_id, role) composite keys via
phase_executions dict. Add phase-scoped can_agent_run/get_runnable_agents.
Update Orchestrator with phase_id support. Create PhaseDependencyGraph
class for computing phase execution waves from Phase.dependencies.

* Add Tier 3 sequential phase cycling for implement phase

Add _run_tier3_implement() that loops through plan phases in dependency
order, running coder -> tester -> agentic review per phase. Add
_build_phase_scoped_prompt() to scope coder prompts to a single phase's
tasks. Wire Tier 3 dispatch into _run_pipeline when complexity_tier is
HIGH. Add _read_review_verdict and _read_last_review_feedback helpers.

* Add conditional integrator write access for Tier 3

Make INTEGRATOR_ROLE file access dynamic: in Tier 3 (high complexity),
integrator gains write access to source, tests, and docs to fix integration
issues. In Tier 2, it remains read-only. Add INTEGRATOR_TIER3_PATTERNS to
gateway agent_restrictions. Update validate_agent_push and
check_agent_restrictions to accept complexity_tier parameter.

* Add per-phase worktrees and parallel dispatch for Tier 3

Add create_phase_worktree() and cleanup_phase_worktrees() to
WorktreeManager for sub-worktree lifecycle. Refactor _run_tier3_implement
to support both sequential and parallel phase execution. When
enable_parallel_phases is True, independent phases in the same wave
execute concurrently via ThreadPoolExecutor.

* Add tests for Tier 3 dispatch and fix can_agent_run bug

Add comprehensive test coverage for Tier 3 phase-level dispatch:
- PhaseDependencyGraph: wave computation, cycle detection, topo sort (16 tests)
- Composite execution tracking: creation, lookup, marking, compat (19 tests)
- Plan parser dependency propagation: formats, normalization (7 tests)
- Tier 3 dispatch: complexity detection, phase-scoped prompts (17 tests)
- Integrator Tier 3 write access: patterns, check, validate (27 tests)
- Phase worktree lifecycle: create, cleanup, sanitization (10 tests)

Fix can_agent_run() to return False when execution is None (unregistered
role) instead of falling through to dependency check. CODER has no deps
so previously returned True for any unregistered phase.

* Update documentation for Tier 3 phase-level dispatch

Document the new 3-tier complexity assessment, phase-level dispatch
model, composite execution tracking, phase dependency graphs, and
Tier 3 integrator write access across the SDLC pipeline guide,
orchestrator architecture, and component READMEs.

* Add tests for Tier 3 parallel dispatch feature

Test coverage for:
- get_role_definition with complexity_tier (22 tests)
- Orchestrator class with phase_id parameter (18 tests)
- check_agent_restrictions with complexity_tier (16 tests)
- _run_tier3_implement execution flow (14 tests)

* Fix Tier 3 test task ID format and lint issues

* Fix lint errors, formatting, and broken test references

- Auto-fix 10 ruff lint errors (unused imports, import ordering)
- Reformat 19 Python files to pass ruff format check
- Fix test_dag_visualizer tests referencing non-existent AgentRole.CHECKER
  and AgentRole.REVIEWER_UNIFIED enum values (replaced with REFINER and
  REVIEWER_CONTRACT)
- Update implement-results.json with check results

* Add code review verdict for #732 implement phase

* Add contract review verdict for #732 implement phase

* Persist statefiles after implement phase

* Address review feedback for Tier 3 phase-level dispatch

Critical fixes:
- Add plan_phase_id to verdict file paths to prevent race conditions
  between parallel phase reviewers
- Remove duplicate _read_review_verdict definition (shadowing bug)
- Return non-zero exit code when review retries exhausted without
  approval (was silently returning success)
- Cancel remaining futures on parallel wave failure instead of blocking
- Thread complexity_tier through gateway session model so integrator
  Tier 3 write permissions are enforced at the gateway level

Medium fixes:
- Reset enable_parallel_phases before complexity re-detection on HITL
  revision downgrade
- Make all_complete()/any_failed() check both executions and
  phase_executions dicts for Tier 3 visibility
- Remove gateway/ and sandbox/ from integrator Tier 3 write access
  (defense-in-depth: don't grant write to security infrastructure)
- Add phase-scoped tester prompt in Tier 3 so testers focus on
  current phase's code
- Add Tier 3-specific integrator prompt with merge/fix/test
  responsibilities

Minor fixes:
- Move import bisect to module level in dependency_graph.py
- Move import yaml to module level in pipelines.py
- Remove redundant import re in plan_parser.py
- Fix cleanup prefix mismatch in cleanup_phase_worktrees()
- Use deque.popleft() for O(1) dequeue in topological_sort()
- Fix ambiguous number extraction in dependency normalization
  (prefer "phase N" pattern over bare first number)
- Align allowed_write between gateway and agent_roles (both now
  block gateway/ and sandbox/)

* Fix double reviewer execution and parallel cancellation in Tier 3

Skip outer reviewer loop for Tier 3 pipelines since per-phase reviewers
already run inside _run_tier3_implement(). Without this fix, the outer
loop spawns redundant review containers and can trigger a full pipeline
retry when the outer reviewer requests changes.

Add threading.Event-based cancellation for parallel phase execution.
The existing f.cancel() calls are ineffective for already-running futures;
the cancel_event is checked before each container spawn so sibling phases
abort promptly when one fails.

* Fix ReviewVerdict type mismatches and add test coverage

Fix 10 pre-existing test failures in test_tier3_execute.py where mocks
returned plain dicts instead of ReviewVerdict objects. The production
code accesses .verdict and .feedback as attributes, so dict mocks cause
AttributeError when tests actually run.

Add test coverage for two previously untested behaviors flagged by the
reviewer:
- TestRetryExhaustion: verify exhausted review retries return exit code 1
- TestCancelEventParallelCancellation: verify cancel_event aborts sibling
  phases when one parallel phase fails

* Fix thread-safety race in cancel_event parallel test

---------

Co-authored-by: egg-orchestrator <egg@localhost>
Co-authored-by: james-in-a-box[bot] <246424927+james-in-a-box[bot]@users.noreply.github.com>
Co-authored-by: egg-reviewer[bot] <261018737+egg-reviewer[bot]@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant