[issue-2817][slice-1/28] Docs and seam-table scaffolding

docs/guides/decomposition-pattern.md

@@ -2,10 +2,12 @@
 
 > Canonical pattern for decomposing oversized Python source files
 > ratcheted by `scripts/file-size-allowlist.yaml`. This is the shape
-> every slice in issue #2261 follows; future contributors who need to
-> decompose another file should follow the same recipe so the codebase
-> grows one consistent seam shape rather than a handful of bespoke
-> ones.
+> every slice in issue #2817 follows (which is a refresh of #2261 —
+> #2261 originated the pattern via PR #2335 but was closed after six
+> follow-up pipelines failed to land any further slice; the program
+> resumed under #2817). Future contributors who need to decompose
+> another file should follow the same recipe so the codebase grows one
+> consistent seam shape rather than a handful of bespoke ones.
 
 ## Why we decompose
 
@@ -23,9 +25,9 @@ grow it. The decomposition program drives those baselines down to
 zero.
 
 The complete acceptance contract for this program lives in issue
-[#2261](https://github.com/jwbron/egg/issues/2261); the HITL decisions
-that locked the pattern shape (decisions 1, 5, 6, 7, 8) are
-referenced inline below.
+[#2817](https://github.com/jwbron/egg/issues/2817); the HITL decisions
+that locked the pattern shape (decisions 1, 5, 6, 7, 8) are referenced
+inline below.
 
 ## (a) Sub-package layout
 
@@ -258,7 +260,7 @@ pre-allocate the canonical further-splits expected per cluster, e.g.
 
 A fresh allowlist entry for "this new submodule is also over the cap"
 is **not allowed** — it directly contradicts the
-"allowlist empty" acceptance criterion of the parent issue (#2261).
+"allowlist empty" acceptance criterion of the parent issue (#2817).
 If the further-split is non-obvious, escalate via a HITL
 question rather than adding the entry.
 
@@ -273,7 +275,7 @@ Decompositions are pure refactors. If a slice surfaces a latent bug
 or a test-coverage gap, **file a follow-up issue** rather than
 bundling the fix:
 
-- Reference the parent issue (`Part of #2261`) in the follow-up.
+- Reference the parent issue (`Part of #2817`) in the follow-up.
 - Note the slice that discovered the gap (`surfaced in slice-7
   during decomposition of orchestrator/overseer/monitor.py`).
 - Don't bundle the fix into the slice's PR — bundling makes the
@@ -286,6 +288,37 @@ mechanical patch-path updates: the slice may fix a one-line
 `patch("routes.pipelines._foo")` rewrite (feedback Q1), but
 anything more invasive is a follow-up.
 
+## (i) Test-file layout: guidance, not a 1:1 gate
+
+When a source file `foo.py` becomes the sub-package `foo/`, tests for
+the new sub-package live alongside the source in a matching
+`tests/.../foo/` sub-package. The pattern keeps a `test_package_shape.py`
+(exercises the barrel) and a `test_main.py` (exercises `__main__.py`);
+where one source `_<submodule>.py` maps cleanly to one behavioral
+cluster, prefer one `test_<submodule>.py` per submodule so tests are
+easy to find from the source.
+
+**This is guidance, not a hard 1:1 gate** (refresh contract decision
+cq-4 under #2817). Suites that are organized by behavioral scenario,
+and whose scenarios legitimately span multiple submodules, may stay
+topically grouped rather than being force-fit into per-submodule
+buckets. The load-bearing requirement is **not** filename mirroring —
+it is the decoupling guarantee that test files keep using the barrel
+surface (`from foo import _bar`, `patch("foo._bar")`) so they don't
+move when the source's internal layout shifts. As long as that
+decoupling holds, the suite's topical organization is fine.
+
+`scripts/select_tests/`'s test suite is the canonical example of a
+legitimately scenario-organized suite (`fallbacks`, `monorepo`,
+`baseline`, `lkg`, …). Several of its files span multiple submodules
+and don't have a natural single-submodule home; it is **not** required
+to be retrofitted to per-submodule mirroring.
+
+When in doubt: pick the layout that minimises future churn. If a
+single behavioral scenario will keep touching the same N submodules
+together, a topical file is correct. If a submodule maps cleanly to a
+discrete set of tests, the 1:1 mirror is correct.
+
 ## Pre-merge checklist (per slice)
 
 - [ ] Step-0 baseline commit lands cleanly (`git mv` only, no cluster
@@ -299,13 +332,17 @@ anything more invasive is a follow-up.
 - [ ] Slice's CLAUDE.md seam table is updated with the new
       submodule layout (see `orchestrator/CLAUDE.md` and
       `gateway/CLAUDE.md`).
+- [ ] Test layout follows section (i) — 1:1 mirroring is **guidance**,
+      not a gate. Scenario-organized suites may stay topical so long
+      as test files use the barrel surface
+      (`from foo import _bar`, `patch("foo._bar")`).
 - [ ] `make lint` and `make test-all` are green.
 - [ ] No behaviour change in the diff; behaviour-adjacent fixes are
       filed as follow-ups (section (h)).
 
 ## See also
 
-- [Issue #2261](https://github.com/jwbron/egg/issues/2261) — full
+- [Issue #2817](https://github.com/jwbron/egg/issues/2817) — full
   scope and acceptance contract for the decomposition program.
 - [Issue #2248](https://github.com/jwbron/egg/issues/2248) and
   [#2250](https://github.com/jwbron/egg/issues/2250) — the file-size
@@ -314,5 +351,5 @@ anything more invasive is a follow-up.
   [gateway/CLAUDE.md](../../gateway/CLAUDE.md) — submodule seam
   tables that downstream slices populate.
 - [docs/architecture/slice-dag.md](../architecture/slice-dag.md) —
-  the slice-DAG implement phase that drives the 15 slices in
+  the slice-DAG implement phase that drives the 28 slices in
   parallel.

gateway/CLAUDE.md

@@ -14,32 +14,45 @@ Run `make test` from the repo root — it's changeset-aware and selects only the
 
 Several of the gateway's largest source files are being decomposed into sub-packages so each module fits the 1,500-line / 100 KB cap from `scripts/file-size-allowlist.yaml`. The canonical pattern (sub-package + explicit per-symbol re-export barrel + underscore-prefixed submodules) is documented in [../docs/guides/decomposition-pattern.md](../docs/guides/decomposition-pattern.md). The tables below map each decomposed module to its current submodule layout so contributors can find code without scanning the barrel.
 
-The barrel `__init__.py` is the **stable public API** (HITL decision-7 of #2261). External consumers — tests, production importers, mocks — keep importing through the barrel (`from gateway.gateway import get_anthropic_client`); the submodule paths below are package-private and may move between releases. `gateway.py`'s routes still register via `@app.route(...)` decorators on thin wrappers in `__init__.py`; submodules hold the implementation bodies (HITL decision-8, refine feedback Q5).
-
-### `gateway/gateway/` — TBD (#2261 slice-14)
-
-Placeholder. Slice-14 of #2261 lands the decomposition of
-`gateway/gateway.py` (~9,890 lines). Pre-allocated submodule
-clusters per the plan:
+The barrel `__init__.py` is the **stable public API** (HITL decision-7 of #2817). External consumers — tests, production importers, mocks — keep importing through the barrel (`from gateway.gateway import get_anthropic_client`); the submodule paths below are package-private and may move between releases. `gateway.py`'s routes still register via `@app.route(...)` decorators on thin wrappers in `__init__.py`; submodules hold the implementation bodies (HITL decision-8, refine feedback Q5).
+
+### `gateway/gateway/` — TBD (#2817 slices 8–12)
+
+Placeholder. Slices 8–12 of #2817 land the decomposition of
+`gateway/gateway.py` (~10,690 lines) as a linear chain. Slice-8 lands
+the step-0 baseline + flat clusters (`_app_factory.py`, `_auth.py`,
+`_checkpoint_routes.py`, `_gh_routes.py`); slices 9–10 each extract one
+sub-sub-package (`_git_routes/`, then `_jira_routes/`); slice-11
+extracts one sub-sub-package (`_confluence_routes/`) plus two flat
+modules (`_worktree_routes.py`, `_anthropic_proxy.py`); slice-12 is
+terminal — it extracts `_sessions.py`
+and drops the allowlist entry. Pre-allocated submodule clusters per
+the plan (trailing `/` denotes a sub-sub-package, `.py` denotes a flat
+module):
 
 | Submodule | Owned symbols |
 |-----------|---------------|
-| `_git_routes/` | TBD — `git_push`, `git_execute`, `git_fetch` (+ named helpers split out of the security-critical `git_push` mega-handler, R5 mitigation) |
+| `_git_routes/` | TBD — `git_push`, `git_execute`, `git_fetch` (+ named helpers split out of the security-critical `git_push` mega-handler) |
 | `_jira_routes/` | TBD — Jira reads, writes, validators |
+| `_confluence_routes/` | TBD — Confluence reads, writes |
+| `_worktree_routes.py` | TBD — worktree lifecycle endpoints |
+| `_anthropic_proxy.py` | TBD — Anthropic API proxy + UpstreamRegistry routing |
+| `_checkpoint_routes.py` | TBD — checkpoint endpoints |
+| `_gh_routes.py` | TBD — GitHub passthrough endpoints |
 | `_auth.py` | TBD — credential injection / token refresh |
 | `_sessions.py` | TBD — session lifecycle |
 | `_app_factory.py` | TBD — Flask app construction, middleware wiring |
 
-The terminal slice (#2261 slice-14) replaces the TBD rows with the
+The terminal slice (#2817 slice-12) replaces the TBD rows with the
 concrete submodule layout once the decomposition lands.
 
 ### Other in-flight decompositions
 
 The following gateway-side files are also under decomposition in
-#2261; rows will be filled in as each slice lands:
+#2817; rows will be filled in as each slice lands:
 
 | File | Current size | Slice |
 |------|--------------|-------|
-| `gateway/worktree_manager.py` | ~2,090 lines | slice-8 (#2261) |
-| `gateway/git_client.py` | ~2,032 lines | slice-6 (#2261) |
-| `gateway/checkpoint_handler.py` | ~1,655 lines | slice-3 (#2261) |
+| `gateway/worktree_manager.py` | ~2,087 lines | slice-20 (#2817) |
+| `gateway/git_client.py` | ~2,068 lines | slice-21 (#2817) |
+| `gateway/checkpoint_handler.py` | ~1,777 lines | slice-26 (#2817) |

orchestrator/CLAUDE.md

@@ -15,14 +15,16 @@ Run `make test` from the repo root — it's changeset-aware and selects only the
 
 Several of the orchestrator's largest source files are being decomposed into sub-packages so each module fits the 1,500-line / 100 KB cap from `scripts/file-size-allowlist.yaml`. The canonical pattern (sub-package + explicit per-symbol re-export barrel + underscore-prefixed submodules) is documented in [../docs/guides/decomposition-pattern.md](../docs/guides/decomposition-pattern.md). The tables below map each decomposed module to its current submodule layout so contributors can find code without scanning the barrel.
 
-The barrel `__init__.py` is the **stable public API** (HITL decision-7 of #2261). External consumers — tests, production importers, mocks — keep importing through the barrel (`from routes.pipelines import _foo`); the submodule paths below are package-private and may move between releases.
+The barrel `__init__.py` is the **stable public API** (HITL decision-7 of #2817). External consumers — tests, production importers, mocks — keep importing through the barrel (`from routes.pipelines import _foo`); the submodule paths below are package-private and may move between releases.
 
-### `orchestrator/routes/pipelines/` — TBD (#2261 slice-15)
+### `orchestrator/routes/pipelines/` — TBD (#2817 slices 2–7)
 
-Placeholder. Slice-15 of #2261 lands the decomposition of
-`orchestrator/routes/pipelines.py` (~16,400 lines) plus the per-phase
-refactor of `_run_pipeline`. Pre-allocated submodule clusters per the
-plan:
+Placeholder. Slices 2–7 of #2817 land the decomposition of
+`orchestrator/routes/pipelines.py` (~24,578 lines) as a linear chain.
+Slice-2 lands the step-0 baseline + five flat clusters; slices 3–6
+each extract one sub-sub-package; slice-7 is terminal — it carves
+`_run_pipeline` into per-phase handlers and drops the allowlist entry.
+Pre-allocated submodule clusters per the plan:
 
 | Submodule | Owned symbols |
 |-----------|---------------|
@@ -37,19 +39,21 @@ plan:
 | `_status_helpers.py` | TBD — phase status / wait helpers |
 | `_brc_verdicts.py` | TBD — BRC verdict aggregators |
 
-The terminal slice (#2261 slice-15) replaces the TBD rows with the
+The terminal slice (#2817 slice-7) replaces the TBD rows with the
 concrete submodule layout once the decomposition lands.
 
 ### Other in-flight decompositions
 
 The following orchestrator-side files are also under decomposition in
-#2261; rows will be filled in as each slice lands:
+#2817; rows will be filled in as each slice lands:
 
 | File | Current size | Slice |
 |------|--------------|-------|
-| `orchestrator/mcp_tools.py` | ~2,820 lines | slice-12 (#2261) |
-| `orchestrator/gateway_client.py` | ~2,357 lines | slice-11 (#2261) |
-| `orchestrator/overseer/monitor.py` | ~2,050 lines | slice-7 (#2261) |
-| `orchestrator/peer_consensus.py` | ~2,013 lines | slice-5 (#2261) |
-| `orchestrator/routes/signals.py` | ~1,986 lines | slice-4 (#2261) |
-| `orchestrator/routes/deployment.py` | ~1,604 lines | slice-2 (#2261) |
+| `orchestrator/mcp_tools.py` | ~2,820 lines | slice-15 (#2817) |
+| `orchestrator/gateway_client.py` | ~3,713 lines | slice-14 (#2817) |
+| `orchestrator/routes/signals.py` | ~2,557 lines | slice-16 (#2817) |
+| `orchestrator/peer_consensus.py` | ~2,215 lines | slice-18 (#2817) |
+| `orchestrator/overseer/monitor.py` | ~2,024 lines | slices 22–23 (#2817) — slice-22 PREP, slice-23 TERMINAL |
+| `orchestrator/kubernetes_spawner.py` | ~1,873 lines | slice-24 (#2817) — TBD |
+| `orchestrator/routes/phases.py` | ~1,654 lines | slice-27 (#2817) — TBD |
+| `orchestrator/routes/deployment.py` | ~1,650 lines | slice-28 (#2817) |

orchestrator/tests/test_populate_contract_audit_events.py

@@ -1214,7 +1214,7 @@ class TestPlanCompleteCallSiteWireUp:
     ``_run_pipeline`` integration setup just for the plan-complete
     branch.
 
-    Fragility note (#2261 slice-15): when ``_run_pipeline`` is
+    Fragility note (#2817 slice-7): when ``_run_pipeline`` is
     decomposed into per-phase handlers (``_run_plan.py``,
     ``_run_implement.py``, etc. — see ``orchestrator/CLAUDE.md``), the
     plan-complete branch body will live in a different function and
@@ -1338,7 +1338,7 @@ class TestSafetyNetForestViolationLandsOnEmptyContractHitl:
     the inner call in ``try: ... except ForestValidationError:`` and
     synthesizes the same ``PopulateResult`` so both paths converge.
 
-    Fragility note (#2261 slice-15): same caveat as
+    Fragility note (#2817 slice-7): same caveat as
     :class:`TestPlanCompleteCallSiteWireUp` — the safety-net branch will
     move out of ``_run_pipeline`` when it is decomposed into per-phase
     handlers; ``inspect.getsource(_run_pipeline)`` will no longer cover

sandbox/CLAUDE.md

@@ -9,3 +9,22 @@ Untrusted agent container. Provides the isolated execution environment where Cla
 ## Testing
 
 Run `make test` from the repo root — it's changeset-aware and selects only the tests reachable from your diff. `make test-all` runs the full suite. See [docs/guides/testing.md](../docs/guides/testing.md) and the root [CLAUDE.md](../CLAUDE.md#quick-reference). Avoid invoking `pytest` directly: you'll skip the narrowing and may hit venv-PATH issues.
+
+## Submodule seam tables
+
+Several of the sandbox's largest source files are being decomposed into sub-packages so each module fits the 1,500-line / 100 KB cap from `scripts/file-size-allowlist.yaml`. The canonical pattern (sub-package + explicit per-symbol re-export barrel + underscore-prefixed submodules) is documented in [../docs/guides/decomposition-pattern.md](../docs/guides/decomposition-pattern.md). The table below maps each decomposed module to its current submodule layout so contributors can find code without scanning the barrel.
+
+The barrel `__init__.py` is the **stable public API** (HITL decision-7 of #2817). External consumers — tests, production importers, mocks — keep importing through the barrel (`from egg_lib.orch_cli import main`); the submodule paths below are package-private and may move between releases.
+
+### In-flight decompositions
+
+The following sandbox-side files are under decomposition in #2817;
+rows will be filled in as each slice lands:
+
+| File | Current size | Slice |
+|------|--------------|-------|
+| `sandbox/egg_lib/orch_cli.py` | ~4,034 lines | slice-13 (#2817) — TBD |
+| `sandbox/entrypoint.py` | ~2,210 lines | slice-19 (#2817) — TBD |
+
+The terminal slice for each file replaces the TBD row with the
+concrete submodule layout once the decomposition lands.

scripts/file-size-allowlist.yaml

@@ -56,15 +56,17 @@ files:
   # additions, pushing the file to ~1,530 lines. The slice-1 branch alone
   # is 1,388 lines (under the 1,500-line hard cap), but the merged
   # work-branch state breaches the cap. Allowlisting under #2548 so the
-  # BRC implement-phase lint passes; decompose under #2569.
+  # BRC implement-phase lint passes; decompose under #2569 (the broader
+  # decomposition work now tracked under #2817 — see issue: field).
   shared/egg_contracts/plan_parser.py:
-    issue: "2548"
+    issue: "2817"
   # #2777 slice-1: the new plan-pre-flight validator wiring and the
   # restructured pre-state-lock plan-exit block (validator → populate →
   # opener) add ~199 lines to advance_phase, pushing this file from
   # ~1,442 lines to ~1,641 — over the 1,500-line hard cap. Slice-1
   # cannot also land a phases.py decomposition; the slice-15 cluster
-  # in #2261 tracks the broader decomposition work, so the allowlist
-  # entry follows the same pattern as the other slice-15 targets.
+  # in #2261 tracks the broader decomposition work (now refreshed under
+  # #2817 — see issue: field), so the allowlist entry follows the same
+  # pattern as the other slice-15 targets.
   orchestrator/routes/phases.py:
-    issue: "2261"
+    issue: "2817"

shared/CLAUDE.md

@@ -0,0 +1,30 @@
+# Shared Libraries
+
+Reusable Python libraries shared between the gateway sidecar, orchestrator, and sandbox container. Each top-level directory under `shared/` is an installable Python package consumed by the other subsystems.
+
+- **[README.md](README.md)** — per-package architecture, public API, and usage examples
+- **[../docs/index.md](../docs/index.md)** — full documentation index
+- **[../docs/guides/decomposition-pattern.md](../docs/guides/decomposition-pattern.md)** — canonical sub-package + barrel re-export pattern used by the in-flight decompositions tracked below
+
+## Testing
+
+Run `make test` from the repo root — it's changeset-aware and selects only the tests reachable from your diff. `make test-all` runs the full suite. See [docs/guides/testing.md](../docs/guides/testing.md) and the root [CLAUDE.md](../CLAUDE.md#quick-reference). Avoid invoking `pytest` directly: you'll skip the narrowing and may hit venv-PATH issues.
+
+## Submodule seam tables
+
+Several of the largest shared source files are being decomposed into sub-packages so each module fits the 1,500-line / 100 KB cap from `scripts/file-size-allowlist.yaml`. The canonical pattern (sub-package + explicit per-symbol re-export barrel + underscore-prefixed submodules) is documented in [../docs/guides/decomposition-pattern.md](../docs/guides/decomposition-pattern.md). The table below maps each decomposed module to its current submodule layout so contributors can find code without scanning the barrel.
+
+The barrel `__init__.py` is the **stable public API** (HITL decision-7 of #2817). External consumers — tests, production importers, mocks — keep importing through the barrel (`from egg_contracts.checkpoint_cli import main`, `from egg_contracts.plan_parser import parse_plan`); the submodule paths below are package-private and may move between releases.
+
+### In-flight decompositions
+
+The following shared-side files are under decomposition in #2817;
+rows will be filled in as each slice lands:
+
+| File | Current size | Slice |
+|------|--------------|-------|
+| `shared/egg_contracts/checkpoint_cli.py` | ~2,233 lines | slice-17 (#2817) — TBD |
+| `shared/egg_contracts/plan_parser.py` | ~1,835 lines | slice-25 (#2817) — TBD (was previously tracked under the closed #2569, folded into #2817 on 2026-05-30) |
+
+The terminal slice for each file replaces the TBD row with the
+concrete submodule layout once the decomposition lands.