From c5612a06eb22c64e1ec461a52a484f4b82579901 Mon Sep 17 00:00:00 2001 From: jwbron <8340608+jwbron@users.noreply.github.com> Date: Mon, 27 Apr 2026 20:57:53 +0000 Subject: [PATCH 1/3] docs: update consult-advisor docs for configurable model Document that egg-orch overseer consult-advisor now reads PipelineConfig.overseer_advisor_model from the orchestrator status endpoint when a pipeline ID is provided, rather than always using the Opus default. Triggered by: ab840a084 (overseer: wire overseer_advisor_model through consult-advisor CLI (#2113) (#2158)) --- docs/guides/pipeline-health-monitoring.md | 4 ++-- docs/reference/orchestrator-cli.md | 2 +- 2 files changed, 3 insertions(+), 3 deletions(-) diff --git a/docs/guides/pipeline-health-monitoring.md b/docs/guides/pipeline-health-monitoring.md index b53f2d664a..ca524d39d1 100644 --- a/docs/guides/pipeline-health-monitoring.md +++ b/docs/guides/pipeline-health-monitoring.md @@ -430,7 +430,7 @@ The intersection gate keeps the heavy-tier model out of every poll cycle while s | `alert` | Emit an `OVERSEER_ALERT` carrying the advisor's `alert_summary`, `alert_detail`, and translated `priority`. The advisor returns `priority` as `p0..p3`; `egg_overseer.priority.label_to_alert` maps to the alert verb's `low|medium|high` dimension. | | `file_issue` | Emit an `OVERSEER_ALERT` whose `recommendation=file_issue` carries a fully composed `issue_title` + `issue_body` + `priority` + `anomaly_signature` in `recommendation_payload`. The CLI verb is **not** invoked here — see [Auto-Issue Filing (Shadow vs Live)](#auto-issue-filing-shadow-vs-live). | -The advisor is exposed to the sandbox as a CLI verb (`egg-orch overseer consult-advisor`); the handler at `sandbox/egg_lib/orch_cli.py::cmd_overseer_consult_advisor` calls `consult_advisor` from `shared/egg_overseer/advisor.py` directly. The underlying `run_agent_async` Opus call therefore runs sandbox-side and stays on the LLM-execution side of the EGG200 boundary documented in [agent-mode-design.md](agent-mode-design.md) — the orchestrator pod never holds Anthropic credentials. The CLI verb reads the keyword arguments (`classification`, `health_alerts`, `progress_events`, `recent_log_lines`) that comprise the executor → advisor prompt contract from a JSON file passed via `--inputs-file`. +The advisor is exposed to the sandbox as a CLI verb (`egg-orch overseer consult-advisor`); the handler at `sandbox/egg_lib/orch_cli.py::cmd_overseer_consult_advisor` calls `consult_advisor` from `shared/egg_overseer/advisor.py` directly. The underlying `run_agent_async` call therefore runs sandbox-side and stays on the LLM-execution side of the EGG200 boundary documented in [agent-mode-design.md](agent-mode-design.md) — the orchestrator pod never holds Anthropic credentials. The model used is resolved from `PipelineConfig.overseer_advisor_model` (read via the orchestrator status endpoint) when a pipeline ID is available; falls back to the `opus` default when absent or the lookup fails. The CLI verb reads the keyword arguments (`classification`, `health_alerts`, `progress_events`, `recent_log_lines`) that comprise the executor → advisor prompt contract from a JSON file passed via `--inputs-file`. **No advisor cap is enforced in this PR** — the existing `max_llm_cost_per_hour=$5` envelope at `sandbox/agent-config/rules/overseer.md` remains the only budget control. A follow-up issue tracks an `overseer_advisor_max_uses_per_phase` (or equivalent) knob if production data shows the cap is needed. @@ -533,7 +533,7 @@ The sandbox-side CLI verb is the one and only way the overseer invokes the advis | `progress_events` | `list[dict]` | Recent structured progress events | | `recent_log_lines` | `list[str]` | Tail of agent container log lines | -The handler at `sandbox/egg_lib/orch_cli.py::cmd_overseer_consult_advisor` calls `egg_overseer.advisor.consult_advisor()` directly. Output is the JSON-serialized `AdvisorVerdict` written to `--output-file` (or stdout when omitted). The Opus `run_agent_async` call runs sandbox-side, keeping the LLM call on the LLM-execution side of the EGG200 boundary; the orchestrator pod never holds Anthropic credentials. +The handler at `sandbox/egg_lib/orch_cli.py::cmd_overseer_consult_advisor` calls `egg_overseer.advisor.consult_advisor()` directly. Output is the JSON-serialized `AdvisorVerdict` written to `--output-file` (or stdout when omitted). The `run_agent_async` call runs sandbox-side, keeping the LLM call on the LLM-execution side of the EGG200 boundary; the orchestrator pod never holds Anthropic credentials. The model alias is resolved from `PipelineConfig.overseer_advisor_model` when a pipeline ID is provided (positional arg or `EGG_PIPELINE_ID`), falling back to `opus` when absent or the orchestrator is unreachable. **Backwards compatibility — top-level optional fields with omit-when-unset serialization.** The `OVERSEER_ALERT` schema gains three first-class optional fields on the `Message` envelope (`orchestrator/message_store.py`): `recommendation: str | None`, `recommendation_payload: dict | None`, and `schema_version: int = 1`. The `Message.to_dict()` serializer **omits** each of the three fields when they hold their defaults, so legacy callers that don't set them produce JSON byte-identical to the pre-#1962 shape. New consumers branch on the presence of `recommendation` (or, equivalently, `schema_version >= 2`); old consumers see no envelope change. The `egg-orch overseer alert` CLI gains `--recommendation file_issue --recommendation-payload-file /path/to/payload.json` flags that populate the new fields; the sandbox handler at `sandbox/egg_agent_tools/handlers/progress.py::progress_overseer_alert` enforces a 50 KB cap on the payload and validates that `recommendation` is one of the legal values (`file_issue` is currently the only accepted value). diff --git a/docs/reference/orchestrator-cli.md b/docs/reference/orchestrator-cli.md index dd98e77d21..b38b9515c9 100644 --- a/docs/reference/orchestrator-cli.md +++ b/docs/reference/orchestrator-cli.md @@ -39,7 +39,7 @@ Run `egg-orch --help` for full usage. All commands support `--json` for machine- | `egg-orch message send [] --to --type --subject "..." --body "..."` | Send directed or broadcast message. Types: `HANDOFF`, `STATUS`, `PROGRESS`, `HEARTBEAT`. (`QUESTION` was removed in [#1897](https://github.com/jwbron/egg/issues/1897).) | | `egg-orch overseer alert [] --anomaly --priority --summary "..." [--detail "..."] [--recommend "..."] [--recommendation file_issue] [--recommendation-payload-file ]` | Broadcast `OVERSEER_ALERT` to human operator (overseer use only — always sets `message_type=OVERSEER_ALERT` and `to_role=all`). `--recommendation file_issue` attaches a structured advisor verdict; requires `--recommendation-payload-file` (JSON path with `issue_title`/`issue_body`/`priority`/`anomaly_signature`). | | `egg-orch overseer file-issue [] --anomaly-type --priority --agent-role --anomaly-signature --issue-title-file --issue-body-file [--parent-alert-message-id ] [--dry-run] [--json]` | File a GitHub issue from the overseer role (advisor-gated). Checks `filed-issues.jsonl` + `gh issue list` for deduplication; skips filing if a matching open issue exists. Default stdout is plain text (`Filed issue #N (...)` or `Existing issue #N already covers ...`); with `--json`, prints JSON `{"issue_number": int, "filed": bool, "dedup_match": int\|null}`; with `--dry-run` (no `gh` invocation either way): if a dedup match is found, prints the same `--json` shape (with `issue_number`/`dedup_match` populated); otherwise prints `{"issue_number": null, "filed": false, "dedup_match": null, "dry_run": true, "argv": [...], "title": str, "body_bytes": int}`. Requires `EGG_PIPELINE_REPO` env var. | -| `egg-orch overseer consult-advisor [] --inputs-file [--output-file ] [--json]` | Consult the Opus advisor for a structured `AdvisorVerdict` (sandbox-side LLM call). Reads a JSON inputs file with `classification`, `health_alerts`, `progress_events`, and `recent_log_lines`. Returns `decision` (`alert`\|`file_issue`\|`watch`), `priority`, `alert_summary`, `alert_detail`, `issue_title`, `issue_body`, and `reasoning`. Without `--output-file`, the JSON verdict is written to stdout; with `--output-file`, the verdict is written to that path and stdout shows a confirmation message (pass `--json` to additionally echo the verdict to stdout). | +| `egg-orch overseer consult-advisor [] --inputs-file [--output-file ] [--json]` | Consult the advisor for a structured `AdvisorVerdict` (sandbox-side LLM call). Reads a JSON inputs file with `classification`, `health_alerts`, `progress_events`, and `recent_log_lines`. Returns `decision` (`alert`\|`file_issue`\|`watch`), `priority`, `alert_summary`, `alert_detail`, `issue_title`, `issue_body`, and `reasoning`. Without `--output-file`, the JSON verdict is written to stdout; with `--output-file`, the verdict is written to that path and stdout shows a confirmation message (pass `--json` to additionally echo the verdict to stdout). When `[]` (or `EGG_PIPELINE_ID`) is set, the verb reads `PipelineConfig.overseer_advisor_model` from the orchestrator status endpoint and passes the configured model alias to `consult_advisor`; falls back to the `opus` default if the pipeline ID is absent or the lookup fails. | | `egg-orch message poll [] [--since ] [--limit ]` | Poll for messages from other agents (concurrent mode) | | `egg-orch message wait [] --for ... [--timeout N] [--since ]` | Block until a **new** typed BRC event arrives (cursor-less calls start at stream tip — already-seen events are skipped). Pass `--since ` for zero-drop send→wait semantics. Exit 0 = matched, 1 = timeout, 2 = transient (retry-safe), 3 = permanent. See [Agent Wait Patterns §3](agent-wait-patterns.md#3-exit-code-contract-for-egg-orch-message-wait) | | `egg-orch message wait-loop [] --for ... [--since ]` | **Canonical STAY ALIVE idiom** — loops `message wait` server-side until a new matching event arrives (defaults to stream-tip; pass `--since ` for zero-drop send→wait). Do not wrap in an outer shell loop. See [Agent Wait Patterns §1](agent-wait-patterns.md#1-the-canonical-idiom) | From cdd922290daf6c5b19ba1fb152152966488fd997 Mon Sep 17 00:00:00 2001 From: "egg-reviewer[bot]" <261018737+egg-reviewer[bot]@users.noreply.github.com> Date: Mon, 27 Apr 2026 21:08:58 +0000 Subject: [PATCH 2/3] docs: generalize advisor model reference at line 420 Address reviewer's non-blocking observation: replace 'an Opus 4.6 advisor' with 'the configured advisor model (PipelineConfig.overseer_advisor_model, defaulting to the opus alias)' to match the configurable model wiring this PR is documenting elsewhere in the same file. --- docs/guides/pipeline-health-monitoring.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/guides/pipeline-health-monitoring.md b/docs/guides/pipeline-health-monitoring.md index ca524d39d1..4077bee6dd 100644 --- a/docs/guides/pipeline-health-monitoring.md +++ b/docs/guides/pipeline-health-monitoring.md @@ -417,7 +417,7 @@ When a Tier 1 `infrastructure_error` alert reaches the overseer monitor, it is r ### Advisor Gate -Issue [#1962](https://github.com/jwbron/egg/issues/1962) introduces the **advisor strategy** for the overseer's decision tier: Haiku continues to drive every cycle (`max_turns=1`), and an Opus 4.6 advisor is invoked **only when both** of these conditions hold simultaneously: +Issue [#1962](https://github.com/jwbron/egg/issues/1962) introduces the **advisor strategy** for the overseer's decision tier: Haiku continues to drive every cycle (`max_turns=1`), and the configured advisor model (`PipelineConfig.overseer_advisor_model`, defaulting to the `opus` alias) is invoked **only when both** of these conditions hold simultaneously: 1. **Haiku flags an anomaly** with classification confidence ≥ 0.8. 2. **A Tier-1 orchestrator health alert is currently active** (the same intersection precedent shipped in [#2012](https://github.com/jwbron/egg/issues/2012)). From 53efb9fae6e01ba526487b4d991ae277369c7375 Mon Sep 17 00:00:00 2001 From: "egg-reviewer[bot]" <261018737+egg-reviewer[bot]@users.noreply.github.com> Date: Mon, 27 Apr 2026 21:24:11 +0000 Subject: [PATCH 3/3] Harmonize line 537 fallback wording to 'lookup fails' MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Reviewer's carry-over non-blocking nit: line 537 said 'falls back to opus when absent or the orchestrator is unreachable' — narrower than lines 42 and 434 which both use 'lookup fails', encompassing all three fallback branches (ImportError, malformed pipeline ID, OrchestratorError). --- docs/guides/pipeline-health-monitoring.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/guides/pipeline-health-monitoring.md b/docs/guides/pipeline-health-monitoring.md index ea84139737..5c6c020712 100644 --- a/docs/guides/pipeline-health-monitoring.md +++ b/docs/guides/pipeline-health-monitoring.md @@ -534,7 +534,7 @@ The sandbox-side CLI verb is the one and only way the overseer invokes the advis | `progress_events` | `list[dict]` | Recent structured progress events | | `recent_log_lines` | `list[str]` | Tail of agent container log lines | -The handler at `sandbox/egg_lib/orch_cli.py::cmd_overseer_consult_advisor` calls `egg_overseer.advisor.consult_advisor()` directly. Output is the JSON-serialized `AdvisorVerdict` written to `--output-file` (or stdout when omitted). The `run_agent_async` call runs sandbox-side, keeping the LLM call on the LLM-execution side of the EGG200 boundary; the orchestrator pod never holds Anthropic credentials. The model alias is resolved from `PipelineConfig.overseer_advisor_model` when a pipeline ID is provided (positional arg or `EGG_PIPELINE_ID`), falling back to `opus` when absent or the orchestrator is unreachable. +The handler at `sandbox/egg_lib/orch_cli.py::cmd_overseer_consult_advisor` calls `egg_overseer.advisor.consult_advisor()` directly. Output is the JSON-serialized `AdvisorVerdict` written to `--output-file` (or stdout when omitted). The `run_agent_async` call runs sandbox-side, keeping the LLM call on the LLM-execution side of the EGG200 boundary; the orchestrator pod never holds Anthropic credentials. The model alias is resolved from `PipelineConfig.overseer_advisor_model` when a pipeline ID is provided (positional arg or `EGG_PIPELINE_ID`), falling back to `opus` when absent or the lookup fails. **Backwards compatibility — top-level optional fields with omit-when-unset serialization.** The `OVERSEER_ALERT` schema gains three first-class optional fields on the `Message` envelope (`orchestrator/message_store.py`): `recommendation: str | None`, `recommendation_payload: dict | None`, and `schema_version: int = 1`. The `Message.to_dict()` serializer **omits** each of the three fields when they hold their defaults, so legacy callers that don't set them produce JSON byte-identical to the pre-#1962 shape. New consumers branch on the presence of `recommendation` (or, equivalently, `schema_version >= 2`); old consumers see no envelope change. The `egg-orch overseer alert` CLI gains `--recommendation file_issue --recommendation-payload-file /path/to/payload.json` flags that populate the new fields; the sandbox handler at `sandbox/egg_agent_tools/handlers/progress.py::progress_overseer_alert` enforces a 50 KB cap on the payload and validates that `recommendation` is one of the legal values (`file_issue` is currently the only accepted value).