Skip to content

docs: update consult-advisor docs for configurable model [doc-updater] - #2171

Merged
jwbron merged 4 commits into
mainfrom
egg/doc-update-overseer-advisor-model
Apr 27, 2026
Merged

docs: update consult-advisor docs for configurable model [doc-updater]#2171
jwbron merged 4 commits into
mainfrom
egg/doc-update-overseer-advisor-model

Conversation

@james-in-a-box

Copy link
Copy Markdown
Contributor

Update documentation to reflect changes from ab840a0 (overseer: wire overseer_advisor_model through consult-advisor CLI (#2113) (#2158)):

  • docs/reference/orchestrator-cli.md: note that [<id>]/EGG_PIPELINE_ID triggers resolution of PipelineConfig.overseer_advisor_model; falls back to opus default
  • docs/guides/pipeline-health-monitoring.md: update two places that called the advisor call an "Opus call" — it now uses the configured model alias

Triggered by: ab840a084

Test Plan

  • Automated: doc-only change, no code paths affected
  • Manual: verify CLI description matches egg-orch overseer consult-advisor --help

Authored-by: egg

Document that egg-orch overseer consult-advisor now reads
PipelineConfig.overseer_advisor_model from the orchestrator status
endpoint when a pipeline ID is provided, rather than always using
the Opus default.

Triggered by: ab840a0 (overseer: wire overseer_advisor_model through
consult-advisor CLI (#2113) (#2158))

@egg-reviewer egg-reviewer Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review

Doc-only PR updating docs/guides/pipeline-health-monitoring.md and docs/reference/orchestrator-cli.md to reflect the runtime wiring landed in ab840a0. Verified the technical claims against the implementation:

  • PipelineConfig.overseer_advisor_model exists at orchestrator/models.py:420 with default="opus".
  • ✅ The status endpoint at orchestrator/routes/pipelines.py:2722-2723 exposes config.overseer_advisor_model exactly as the CLI handler expects.
  • sandbox/egg_lib/orch_cli.py:1813-1862 resolves pipeline_id from the positional arg or EGG_PIPELINE_ID, calls OrchClient().get_pipeline_status(...), reads status["config"]["overseer_advisor_model"], and constructs a duck-typed SimpleNamespace for consult_advisor. Falls back to config=None (→ "opus" in shared/egg_overseer/advisor.py:238) on missing/malformed pipeline-id, OrchestratorError, or ImportError.
  • ✅ The CLI table entry's wording ("falls back to the opus default if the pipeline ID is absent or the lookup fails") matches the three explicit fallback branches in code.
  • ✅ Removing the bare "Opus" qualifier in the two call sites is correct — those sentences described the underlying run_agent_async invocation, which is now model-agnostic.

Non-blocking observations

  • docs/guides/pipeline-health-monitoring.md:420 still says "an Opus 4.6 advisor is invoked only when both ..." This is technically still true for the default config (the "opus" alias resolves to Opus 4.6), but now misleading for callers who override overseer_advisor_model (e.g., to claude-opus-4-7 as one of the tests does). Since the PR is already in this file rephrasing the same concept, "Opus-tier advisor" or "configured advisor model" would have been a natural one-line follow-on.
  • The other "Opus" mentions in the file (lines 29, 245, 286, 294-296, 339-340, 425) are tier-class descriptions ("Sonnet/Opus decision-maker") or accurate default="opus" references — fine to leave.

No blocking issues.

— Authored by egg

@james-in-a-box

This comment has been minimized.

@james-in-a-box

This comment has been minimized.

Address reviewer's non-blocking observation: replace 'an Opus 4.6 advisor'
with 'the configured advisor model (PipelineConfig.overseer_advisor_model,
defaulting to the opus alias)' to match the configurable model wiring this
PR is documenting elsewhere in the same file.
@james-in-a-box

Copy link
Copy Markdown
Contributor Author

Agreed. Pushed cdd9222 generalizing line 420 from "an Opus 4.6 advisor" to "the configured advisor model (PipelineConfig.overseer_advisor_model, defaulting to the opus alias)" — consistent with the rephrasing this PR already does at lines 433 and 536. Left the other Opus mentions you flagged (29, 245, 286, 294-296, 339-340, 425) untouched since they're tier-class descriptions or accurate default="opus" references.

— Authored by egg

@james-in-a-box

This comment has been minimized.

@james-in-a-box

This comment has been minimized.

… docs/reference/orchestrator-cli.md

Both sides modified the consult-advisor row:
- HEAD: generalize 'Opus advisor' wording and add note about
  PipelineConfig.overseer_advisor_model resolution
- origin/main: add --recent-log-bytes-cap flag documentation

Resolution: include both — kept generalized 'advisor' wording, added
the new flag to the signature, and merged both explanatory paragraphs.
@james-in-a-box

Copy link
Copy Markdown
Contributor Author

Conflict Resolution Summary

Resolved merge conflicts with main (only 1 file had a real conflict; 6 others auto-merged cleanly):

File Category Resolution
docs/reference/orchestrator-cli.md Additive Kept both changes — generalized "Opus advisor" → "advisor" wording (this PR) plus the new --recent-log-bytes-cap flag and explanatory paragraph (from #2168). Merged signature now lists both flags; explanatory text contains both the model-resolution note and the byte-cap note.

Auto-merged (no manual resolution): docs/guides/pipeline-health-monitoring.md, sandbox/agent-config/rules/overseer.md, sandbox/overseer_monitor.py, sandbox/tests/test_overseer_monitor_script.py, shared/egg_overseer/advisor.py, shared/tests/test_overseer_advisor.py.

Please review: the merged consult-advisor row in docs/reference/orchestrator-cli.md — both the model-resolution paragraph (this PR's contribution) and the byte-cap paragraph (from #2168) appear in sequence. Verify the order and wording read well together.

— Authored by egg

@james-in-a-box

This comment has been minimized.

@james-in-a-box

This comment has been minimized.

@james-in-a-box

Copy link
Copy Markdown
Contributor Author

Conflict Resolution Summary

No action needed — this branch is already up to date with main.

  • git fetch origin main → up to date
  • git merge --no-commit origin/mainAlready up to date.
  • git merge-base --is-ancestor origin/main HEAD → true

The previous merge commit (5c3fc342c) already resolved the conflict in docs/reference/orchestrator-cli.md (consult-advisor row: kept generalized "advisor" wording from this PR, added the --recent-log-bytes-cap flag from main, merged both explanatory paragraphs). PR status is MERGEABLE; the BLOCKED state reflects required reviews/checks, not conflicts.

— Authored by egg

@james-in-a-box

This comment has been minimized.

@egg-reviewer egg-reviewer Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Re-Review

The delta since c5612a06 is two commits: cdd9222 (addresses my prior non-blocking note on line 421) and 5c3fc342 (merge against main resolving docs/reference/orchestrator-cli.md).

Prior feedback resolved. Line 421 in docs/guides/pipeline-health-monitoring.md now reads "the configured advisor model (PipelineConfig.overseer_advisor_model, defaulting to the opus alias)" — matches the rephrasing this PR already does at lines 434 and 537.

Merge resolution verified. The consult-advisor row in docs/reference/orchestrator-cli.md correctly preserves both:

  • This PR's generalized "advisor" wording + the model-resolution paragraph (When [<id>] (or EGG_PIPELINE_ID) is set, the verb reads PipelineConfig.overseer_advisor_model ...).
  • #2168's [--recent-log-bytes-cap <n>] flag in the signature + the byte-cap paragraph (--recent-log-bytes-cap overrides the PipelineConfig.overseer_advisor_recent_log_bytes_cap value ...).

The two paragraphs read in the right order (model resolution → byte cap), so the row stays coherent.

Technical claims re-verified against the code:

  • orchestrator/models.py:420overseer_advisor_model: str = Field(default="opus")
  • orchestrator/routes/pipelines.py:2723 — status endpoint exposes overseer_advisor_model
  • sandbox/egg_lib/orch_cli.py:1813–1862 — pipeline-id gated; resolves via OrchClient().get_pipeline_status(...) reading status["config"]["overseer_advisor_model"]; duck-types via SimpleNamespace; three explicit fallback branches (ImportError, malformed pid, OrchestratorError) ✅
  • shared/egg_overseer/advisor.py:238config=None → "opus"
  • docs/guides/pipeline-health-monitoring.md:245overseer_advisor_model table entry preserved by the merge ✅

Non-blocking nit (carried over)

Line 537's fallback list ("falls back to opus when absent or the orchestrator is unreachable") doesn't mention the ImportError and malformed-pipeline-id branches that line 42 and line 434 do mention. Doesn't matter in practice — those are pathological paths — but worth noting for future doc passes if anyone is doing a final wording sweep.

No blocking issues. The doc-update is accurate, the prior feedback was addressed substantively (not superficially), and the merge resolution is clean.

— Authored by egg

@james-in-a-box

This comment has been minimized.

@james-in-a-box

This comment has been minimized.

Reviewer's carry-over non-blocking nit: line 537 said 'falls back to
opus when absent or the orchestrator is unreachable' — narrower than
lines 42 and 434 which both use 'lookup fails', encompassing all three
fallback branches (ImportError, malformed pipeline ID, OrchestratorError).
@james-in-a-box

Copy link
Copy Markdown
Contributor Author

Agreed. Pushed 53efb9f harmonizing line 537's fallback wording to "falls back to opus when absent or the lookup fails" — matching lines 42 and 434, which already use that broader phrasing covering all three fallback branches (ImportError, malformed pipeline-id, OrchestratorError). Line 537 previously said "the orchestrator is unreachable", which described only the OrchestratorError branch.

— Authored by egg

@james-in-a-box

This comment has been minimized.

@james-in-a-box

This comment has been minimized.

@egg-reviewer egg-reviewer Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Re-Review

The delta since 5c3fc342 is a single commit (53efb9f) touching one line of docs/guides/pipeline-health-monitoring.md.

Prior non-blocking nit resolved. Line 537 now reads "falls back to opus when absent or the lookup fails" — matching line 434 in the same file and line 42 of docs/reference/orchestrator-cli.md. The previous wording ("the orchestrator is unreachable") only described the OrchestratorError branch; the new wording correctly covers all four fallback paths in sandbox/egg_lib/orch_cli.py:1813–1862:

  1. ImportError on the OrchClient import (line 1833)
  2. Malformed pipeline-id (_SAFE_ID_PATTERN miss on line 1854)
  3. OrchestratorError from get_pipeline_status (line 1848)
  4. Status response missing config.overseer_advisor_model (lines 1842–1846 — cfg_dict is None or model is falsy)

"Lookup fails" is the correct umbrella term for all four.

Scope and risk. Doc-only, one-line change, no semantic shift in machine-readable artifacts. No CI/runtime impact.

Verified no regressions in the surrounding text. Line 245's overseer_advisor_model table entry is unchanged; line 421's "configured advisor model" rephrasing from cdd9222 is preserved; the consult-advisor row in docs/reference/orchestrator-cli.md retains both the model-resolution paragraph and #2168's --recent-log-bytes-cap paragraph in the right order.

No blocking issues.

— Authored by egg

@james-in-a-box

This comment has been minimized.

@james-in-a-box

Copy link
Copy Markdown
Contributor Author
egg is addressing review feedback...

@james-in-a-box

Copy link
Copy Markdown
Contributor Author

No actionable feedback in the latest review (53efb9f) — the reviewer confirmed the prior non-blocking nit on line 537 was resolved and explicitly noted "No blocking issues." Nothing to push.

— Authored by egg

@james-in-a-box

Copy link
Copy Markdown
Contributor Author

egg feedback addressed. View run logs

12 previous review(s) hidden.

@jwbron
jwbron merged commit db15411 into main Apr 27, 2026
24 of 25 checks passed
james-in-a-box Bot added a commit that referenced this pull request Apr 28, 2026
#2171)

* docs: update consult-advisor docs for configurable model

Document that egg-orch overseer consult-advisor now reads
PipelineConfig.overseer_advisor_model from the orchestrator status
endpoint when a pipeline ID is provided, rather than always using
the Opus default.

Triggered by: ab840a0 (overseer: wire overseer_advisor_model through
consult-advisor CLI (#2113) (#2158))

* docs: generalize advisor model reference at line 420

Address reviewer's non-blocking observation: replace 'an Opus 4.6 advisor'
with 'the configured advisor model (PipelineConfig.overseer_advisor_model,
defaulting to the opus alias)' to match the configurable model wiring this
PR is documenting elsewhere in the same file.

* Harmonize line 537 fallback wording to 'lookup fails'

Reviewer's carry-over non-blocking nit: line 537 said 'falls back to
opus when absent or the orchestrator is unreachable' — narrower than
lines 42 and 434 which both use 'lookup fails', encompassing all three
fallback branches (ImportError, malformed pipeline ID, OrchestratorError).

---------

Co-authored-by: jwbron <8340608+jwbron@users.noreply.github.com>
Co-authored-by: egg-reviewer[bot] <261018737+egg-reviewer[bot]@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant