feat(agents): allow optimization agents to configure models

[BrianNewsom]

Summary

This PR makes the Analyst, Eval Author, and Experimentalist use the models selected through nemo setup. It replaces plugin-specific endpoints, credentials, hardcoded model names, and Experimentalist's three-tier model configuration with one Platform-native pair: default_model for quality-oriented work and fast_model for latency-oriented work, with fast falling back to default.

Before this change, each optimization plugin configured inference independently. Setup-selected models and credentials in Platform Secrets were not reused, so OpenAI-compatible providers required manual plugin configuration and Anthropic could not be selected in setup and consumed through its native Messages contract.

Changes

Platform-native model selection

nemo setup now stores two workspace-qualified Model Entity references in the active CLI context:

default_model: default/gpt-5
fast_model: default/gpt-5-mini

Shared logic in nemo_platform_plugin reads the pair, resolves each entity through the existing Platform SDK, uses its actual served model name/provider route/backend format/Platform headers, constructs the corresponding Nooa/LiteLLM client, and activates those clients for the agent run.

Backend selection comes from the Model Entity rather than model-name heuristics:

• OPENAI_CHAT uses the OpenAI-compatible Chat Completions contract.
• ANTHROPIC_MESSAGES uses Anthropic's native Messages contract.

The plugins do not translate request or response bodies. Provider credentials stay in Platform Secrets and are never passed into the optimization agents.

Consistent agent roles

• default_model is the existing quality-oriented default and replaces the previous smart role.
• fast_model is used only for deliberately latency-sensitive or high-volume work, such as summarization and control steps.
• Experimentalist's former mid-tier uses are assigned to default or fast based on their quality requirements; setup does not try to infer a third provider-independent capability tier.
• Individual plugins only request the active default or fast client; Model Entity parsing, served-name lookup, routing headers, backend selection, and client lifetime are centralized.

Experimentalist's checked-in models.yaml remains separate: it controls model mutations for the agent under test, not the models running the optimizer itself. Application and benchmark credentials such as AUT_MODEL_NAME, OPENAI_API_KEY, and OPENAI_BASE_URL may therefore still be required by the evaluated workload.

Setup and compatibility

Interactive setup asks for a default and fast model. The fast prompt defaults to the chosen default, so users who want one model can press Enter twice without understanding the internal role mapping.

nemo setup --auto preserves main's behavior:

• Use NEMO_DEFAULT_MODEL when set; otherwise select the alphabetically first discovered Model Entity.
• Use NEMO_FAST_MODEL when set; otherwise reuse the effective default.

Existing contexts that only contain default_model remain compatible because fast automatically falls back to default.

For CI, containers, and other environments that cannot read the host CLI context, the pair can be overridden with Model Entity references:

export NEMO_DEFAULT_MODEL=default/gpt-5
export NEMO_FAST_MODEL=default/gpt-5-mini

The referenced entities and providers must already exist on the target Platform.

Removed optimizer-specific configuration

The optimization agents no longer consume:

• AUTHOR_API_BASE, AUTHOR_API_KEY, or AUTHOR_*_MODEL_NAME
• NEMO_EXPERIMENTALIST_API_BASE or NEMO_EXPERIMENTALIST_API_KEY
• NEMO_EXPERIMENTALIST_MODELS_SMART, NEMO_EXPERIMENTALIST_MODELS_MID, or NEMO_EXPERIMENTALIST_MODELS_FAST
• INFERENCE_API_KEY as an optimization-agent credential

Scheduled Analyst runs

nemo insights analysis enable now persists the effective pair in the server-side analysis config. The controller does not need access to the operator's local ~/.config/nmp/config.yaml. Re-run analysis enable after changing the pair; previously enabled records must also be re-enabled once to capture it.

Anthropic setup's auth_header_format fix is intentionally isolated in #1158.

Type of Change

• Code change (feature, bug fix, or refactor)
• Code change with documentation updates
• Documentation only
• Contributor tooling or automation
• CI, build, or test infrastructure

Quality Gates

• Tests added or updated for changed behavior
• Existing tests cover changed behavior — justification:
• Tests not applicable — justification:
• Documentation updated for user-visible behavior
• Documentation not applicable — justification:

Verification

• Pull request title follows the repository's Conventional Commit format
• Every commit includes an appropriate Signed-off-by: trailer
• uv run pre-commit run -a passes, or any blocked checks are identified below
• Targeted tests pass, or tests are marked not applicable above
• No secrets, API keys, or credentials are included

Targeted validation:

• uv run --frozen pytest packages/nemo_platform_ext/tests/cli/commands/test_setup.py -q — 229 passed.
• uv run --frozen pytest sdk/python/nemo-platform/tests/vendored/nemo_platform_ext/cli/commands/test_setup.py -q — 229 passed.
• Shared adapter + Analyst + Eval Author + Experimentalist scope before the latest main sync — 2,386 passed, 3 intentional live-test skips.
• Latest main conflict-resolution scope — Experimentalist: 618 passed; Eval Author: 43 passed, 3 intentional live-test skips; focused cross-plugin regression set: 25 passed; targeted ty: passed.
• Latest main sync (ff047d2a57) — preserved the new SDK/bootstrap/mise documentation and the model-pair setup text; relevant file-level pre-commit hooks passed.
• uv run pre-commit run -a after the latest main sync — Ruff, formatting, ty, config docs, Helm docs, copyright, UI, import-boundary, and merge-conflict checks passed. uv-lock-check is blocked because uv lock adds newly published wheel metadata despite no dependency-input change; that generated drift is excluded, and the committed lock resolves successfully with uv run --frozen.
• make docs-check — passed; 216 MDX files parsed cleanly and no published-to-gated links found.
• make docs-broken-links — command exited successfully but reported five existing broken links in unchanged pages (telemetry-and-privacy.mdx, using-authentication.mdx, and studio/index.mdx). None are in this pull request's changed documentation.

Live provider validation:

• Analyst completed against an internal NVIDIA inference model (default/openai-openai-gpt-5-5) and created two Insights from 19 observed sessions. Separate Analyst runs completed with native Anthropic models and with an OpenAI default + Anthropic fast pair.
• Eval Author's live repair canary completed with distinct configured models and invoked the direct OpenAI default/gpt-5-mini fast path.
• Experimentalist completed a full one-round Tau3 optimization with the internal GPT-5.5 default and direct OpenAI GPT-5 Mini fast model. Platform logs recorded 39 successful default calls and one successful fast call; the candidate won held-out validation with reward 0.5.

Summary by CodeRabbit

• New Features

  • Added separate default and fast model selection through setup, including discovery, fallback, and validation.
  • Analysis configurations now persist and use both selected models.
  • Eval Author, Experimentalist, and Analyst now use centrally configured Platform models.

• Documentation

  • Updated setup guides, examples, troubleshooting, and configuration references.
  • Replaced legacy provider-specific model and credential instructions.

• Bug Fixes

  • Improved model routing, resource cleanup, preflight checks, and scheduled-analysis model propagation.

[coderabbitai]

Note

Reviews paused

It looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the reviews.auto_review.auto_pause_after_reviewed_commits setting.

Use the following commands to manage reviews:

• @coderabbitai resume to resume automatic reviews.
• @coderabbitai review to trigger a single review.

Use the checkboxes below for quick actions:

• ▶️ Resume reviews
• 🔍 Trigger review

📝 Walkthrough

Walkthrough

The PR moves Insights, Experimentalist, and Eval Author to Platform-managed default and fast model pairs. Setup persists and resolves both models. Runtime integrations share routed clients with scoped cleanup. Insights stores model pairs for scheduled analysis. Legacy credential and tier configuration is removed.

Changes

Platform-managed model routing

Layer / File(s)	Summary
Model-pair setup and context resolution packages/nemo_platform_ext/...	nemo setup selects provider-scoped default and fast models, persists both values, supports environment overrides, and falls back from fast to default.
Shared model clients packages/nemo_platform_plugin/...	Platform model entities resolve to OpenAI- or Anthropic-compatible clients with scoped activation, deduplication, and cleanup.
Plugin runtime integration plugins/nemo-experimentalist/..., plugins/nemo-eval-author/..., plugins/nemo-insights/...	Plugin components and runners use shared default and fast clients. Legacy credential and tier handling is removed.

Insights persistence and execution

Layer / File(s)	Summary
Analysis enable contract plugins/nemo-insights/src/nemo_insights_plugin/schema.py, service.py, sdk_resources/...	Enable requests require and persist non-empty default and fast model references. Re-enabling replaces the stored pair.
Scheduled analysis execution plugins/nemo-insights/src/nemo_insights_plugin/controller.py, jobs/analyze.py, analyst/run.py	Scheduled jobs pass persisted model references to Analyst execution through Platform task SDK clients.
Preflight and validation plugins/nemo-insights/src/nemo_insights_plugin/preflight.py, plugins/nemo-experimentalist/src/nemo_experimentalist_plugin/preflight.py	Preflight checks configured models instead of plugin-owned credentials. Missing configuration returns setup guidance.

Supporting configuration and documentation

Layer / File(s)	Summary
Guides and examples README.md, SETUP.md, docs/..., plugins/*/README.md, .env.example files	Documentation describes Platform model entities, default/fast selection, overrides, provider credentials, and model-pair persistence.
CI and testbed setup .github/workflows/..., plugins/nemo-insights/testbed/...	Test jobs run nemo setup and start model-routing services instead of requiring plugin-owned inference credentials.
Legacy configuration removal docs/set-up/config-reference.mdx, script/generate_config_docs.py, plugins/nemo-experimentalist/benchmarks/...	Experimentalist configuration sections and tiered model entries are removed from configuration and generated documentation.

Possibly related PRs

• NVIDIA-NeMo/nemo-platform#679: Extends the Insights model-routing and nemo setup infrastructure.
• NVIDIA-NeMo/nemo-platform#896: Introduced Experimentalist model-configuration paths changed by this migration.
• NVIDIA-NeMo/nemo-platform#1097: Overlaps with the Analyst migration to configured Platform models.

Suggested labels: breaking

Suggested reviewers: svvarom

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name	Status	Explanation	Resolution
Docstring Coverage	⚠️ Warning	Docstring coverage is 34.22% which is insufficient. The required threshold is 80.00%.	Write docstrings for the functions missing them to satisfy the coverage threshold.

✅ Passed checks (4 passed)

Check name	Status	Explanation
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title accurately summarizes the main change: optimization agents now use configurable models through the shared setup flow.

✨ Finishing Touches 💡 1

📝 Generate docstrings 💡

• Create stacked PR
• Commit on current branch

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch plugin-model-pairs/brnewsom

---

_{Comment @coderabbitai help to get the list of available commands.}

[coderabbitai]

Actionable comments posted: 7

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (1)

plugins/nemo-insights/examples/research-agent/README.md (1)

75-80: 🎯 Functional Correctness | 🟡 Minor | ⚡ Quick win

Do not combine --auto with prompt instructions.

--auto uses automatic setup behavior. It does not support “press Enter at the fast prompt.” Remove --auto for interactive selection, or revise the text to describe automatic selection.

🤖 Prompt for AI Agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@plugins/nemo-insights/examples/research-agent/README.md` around lines 75 -
80, Update the setup instructions around the `nemo setup` command so they no
longer combine `--auto` with interactive prompt guidance. Either remove `--auto`
to preserve the documented Enter-to-reuse behavior, or revise the surrounding
text to describe automatic model selection.

🧹 Nitpick comments (2)

packages/nemo_platform_ext/src/nemo_platform_ext/config/config.py (1)

75-78: 📐 Maintainability & Code Quality | 🔵 Trivial | 💤 Low value

Model resolution logic is duplicated.

resolve() and _create_default_config() now contain the same three-step precedence chain. Extract a small helper so a future precedence change updates one place.

Also applies to: 489-492, 501-501, 551-559

🤖 Prompt for AI Agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@packages/nemo_platform_ext/src/nemo_platform_ext/config/config.py` around
lines 75 - 78, Extract the duplicated three-step model precedence chain from
resolve() and _create_default_config() into a shared helper, then reuse that
helper at all affected call sites. Preserve the existing precedence and ensure
future model-resolution changes only require updating the helper.

packages/nemo_platform_ext/src/nemo_platform_ext/cli/commands/setup.py (1)

2251-2262: 🎯 Functional Correctness | 🔵 Trivial | 💤 Low value

NEMO_FAST_MODEL is dropped when no default model resolves.

If discovery returns nothing and NEMO_DEFAULT_MODEL is unset, the else branch runs and neither model is persisted, even when the user exported NEMO_FAST_MODEL. Persist the fast model in that case, or state in the warning that the fast override is ignored without a default.

🤖 Prompt for AI Agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@packages/nemo_platform_ext/src/nemo_platform_ext/cli/commands/setup.py`
around lines 2251 - 2262, Update the model persistence flow around ModelPair and
the default_model fallback so an explicitly configured NEMO_FAST_MODEL is not
discarded when default_model is unavailable. Persist the fast override in the
no-default branch using the existing model configuration mechanism, or
explicitly warn that it is ignored; preserve the current behavior when a default
model resolves.

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In @.github/workflows/insights-testbed.yml:
- Around line 117-121: Update the NEMO_DEFAULT_MODEL fallback to the
provider-discovered ID default/openai-gpt-5-5, and remove NEMO_FAST_MODEL when
it matches the default so _run_auto_mode supplies the fallback.

In `@docs/get-started/example-agent.mdx`:
- Around line 168-169: Add the clone-mode warning immediately before the
Git-backed command containing NEMO_DEFAULT_MODEL and NEMO_FAST_MODEL, explaining
that clone mode is an integrity boundary rather than a confidentiality boundary.
Keep the existing later local-only warning unchanged.

In `@packages/nemo_platform_ext/src/nemo_platform_ext/cli/commands/setup.py`:
- Around line 390-414: Update _configure_local_connection so setup never
overwrites an existing context named by _LOCAL_CONTEXT_NAME. Before
Config.write, detect whether that context already exists and reject the
operation or use a distinct dedicated context, preserving the existing context’s
endpoint and credentials while retaining the local no-auth setup behavior.

In `@packages/nemo_platform_plugin/src/nemo_platform_plugin/nooa_model_client.py`:
- Around line 47-51: Update NooaModelClient.aclose to always attempt closing
fast when it is distinct from default, even if default.aclose fails. In the
model-resolution failure path around resolved, close every already-created
client before re-raising the original resolution error. Add failure-path tests
covering both cleanup scenarios in
packages/nemo_platform_plugin/src/nemo_platform_plugin/nooa_model_client.py:47-51
and :159-167.

In `@plugins/nemo-experimentalist/benchmarks/run.py`:
- Around line 345-348: Update the summary aggregation logic around the job field
validation to validate raw payload or stats values before calling int(). Reject
booleans explicitly, along with floats and numeric strings, and preserve the
field name in the TypeError; only convert values after they pass the strict
integer check.

In
`@plugins/nemo-experimentalist/src/nemo_experimentalist_plugin/experimentalist/run.py`:
- Around line 106-136: Preserve the selected default and fast model references
returned by resolve_model_clients in the run flow, and expose or pass them to
the benchmark caller alongside the result summary. Update the benchmark
persistence path, including benchmarks/run.py and its summary.json generation,
to record both resolved references in addition to the existing ModelSpec fields.

In `@plugins/nemo-insights/examples/research-agent/tests/test_analyst_e2e.py`:
- Around line 152-154: Update the credential check in the E2E fixture to require
TAVILY_API_KEY and accept either NVIDIA_API_KEY or NEMO_DEFAULT_INFERENCE_KEY as
the model-provider credential. Ensure the missing-key message reports the model
credential requirement without skipping when only the alternate credential is
present.

---

Outside diff comments:
In `@plugins/nemo-insights/examples/research-agent/README.md`:
- Around line 75-80: Update the setup instructions around the `nemo setup`
command so they no longer combine `--auto` with interactive prompt guidance.
Either remove `--auto` to preserve the documented Enter-to-reuse behavior, or
revise the surrounding text to describe automatic model selection.

---

Nitpick comments:
In `@packages/nemo_platform_ext/src/nemo_platform_ext/cli/commands/setup.py`:
- Around line 2251-2262: Update the model persistence flow around ModelPair and
the default_model fallback so an explicitly configured NEMO_FAST_MODEL is not
discarded when default_model is unavailable. Persist the fast override in the
no-default branch using the existing model configuration mechanism, or
explicitly warn that it is ignored; preserve the current behavior when a default
model resolves.

In `@packages/nemo_platform_ext/src/nemo_platform_ext/config/config.py`:
- Around line 75-78: Extract the duplicated three-step model precedence chain
from resolve() and _create_default_config() into a shared helper, then reuse
that helper at all affected call sites. Preserve the existing precedence and
ensure future model-resolution changes only require updating the helper.

🪄 Autofix

Fix all unresolved CodeRabbit comments on this PR:

• Push a commit to this branch (recommended)
• Create a new PR with the fixes

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: c5ee37ee-7c42-46bd-9f55-4a7d4633abbd

📥 Commits

Reviewing files that changed from the base of the PR and between 20e39e0 and c883a04.

⛔ Files ignored due to path filters (5)

• sdk/python/nemo-platform/src/nemo_platform/cli/commands/manifest_registry.py is excluded by !sdk/**
• sdk/python/nemo-platform/src/nemo_platform/cli/commands/setup.py is excluded by !sdk/**
• sdk/python/nemo-platform/src/nemo_platform/config/config.py is excluded by !sdk/**
• sdk/python/nemo-platform/src/nemo_platform/config/models.py is excluded by !sdk/**
• sdk/python/nemo-platform/tests/vendored/nemo_platform_ext/cli/commands/test_setup.py is excluded by !sdk/**

📒 Files selected for processing (88)

• .github/actions/insights-intake-stack/action.yml
• .github/workflows/insights-testbed.yml
• README.md
• SETUP.md
• docs/agents/insight-driven-optimization.mdx
• docs/get-started/example-agent.mdx
• docs/get-started/setup.mdx
• docs/set-up/config-reference.mdx
• packages/nemo_platform_ext/src/nemo_platform_ext/cli/commands/manifest_registry.py
• packages/nemo_platform_ext/src/nemo_platform_ext/cli/commands/setup.py
• packages/nemo_platform_ext/src/nemo_platform_ext/config/config.py
• packages/nemo_platform_ext/src/nemo_platform_ext/config/models.py
• packages/nemo_platform_ext/tests/cli/commands/test_setup.py
• packages/nemo_platform_plugin/src/nemo_platform_plugin/nooa_model_client.py
• packages/nemo_platform_plugin/tests/test_nooa_model_client.py
• plugins/nemo-eval-author/.env.example
• plugins/nemo-eval-author/README.md
• plugins/nemo-eval-author/src/nemo_eval_author_plugin/cli.py
• plugins/nemo-eval-author/src/nemo_eval_author_plugin/eval_author/agent.py
• plugins/nemo-eval-author/src/nemo_eval_author_plugin/eval_author/run.py
• plugins/nemo-eval-author/src/nemo_eval_author_plugin/model_config.py
• plugins/nemo-eval-author/tests/conftest.py
• plugins/nemo-eval-author/tests/test_eval_author_repair_e2e.py
• plugins/nemo-eval-author/tests/test_eval_author_run.py
• plugins/nemo-eval-author/tests/test_model_config.py
• plugins/nemo-eval-author/tests/test_plugin_boundary.py
• plugins/nemo-experimentalist/AGENTS.md
• plugins/nemo-experimentalist/README.md
• plugins/nemo-experimentalist/benchmarks/README.md
• plugins/nemo-experimentalist/benchmarks/configs/tau3-quality.yaml
• plugins/nemo-experimentalist/benchmarks/configs/tau3-smoke.yaml
• plugins/nemo-experimentalist/benchmarks/configs/terminal-bench-quality.yaml
• plugins/nemo-experimentalist/benchmarks/configs/terminal-bench-smoke.yaml
• plugins/nemo-experimentalist/benchmarks/run.py
• plugins/nemo-experimentalist/examples/tau3-nooa-agent/.env.example
• plugins/nemo-experimentalist/examples/tau3-nooa-agent/record_tau_airline_traces.py
• plugins/nemo-experimentalist/src/nemo_experimentalist_plugin/cli.py
• plugins/nemo-experimentalist/src/nemo_experimentalist_plugin/config.py
• plugins/nemo-experimentalist/src/nemo_experimentalist_plugin/experimentalist/components/analyzer.py
• plugins/nemo-experimentalist/src/nemo_experimentalist_plugin/experimentalist/components/cards.py
• plugins/nemo-experimentalist/src/nemo_experimentalist_plugin/experimentalist/components/coder.py
• plugins/nemo-experimentalist/src/nemo_experimentalist_plugin/experimentalist/components/goal_tree.py
• plugins/nemo-experimentalist/src/nemo_experimentalist_plugin/experimentalist/components/loop.py
• plugins/nemo-experimentalist/src/nemo_experimentalist_plugin/experimentalist/components/model_config.py
• plugins/nemo-experimentalist/src/nemo_experimentalist_plugin/experimentalist/components/proposer.py
• plugins/nemo-experimentalist/src/nemo_experimentalist_plugin/experimentalist/components/rationalizer.py
• plugins/nemo-experimentalist/src/nemo_experimentalist_plugin/experimentalist/components/terminator.py
• plugins/nemo-experimentalist/src/nemo_experimentalist_plugin/experimentalist/components/trace_analyzer.py
• plugins/nemo-experimentalist/src/nemo_experimentalist_plugin/experimentalist/components/trace_scorer.py
• plugins/nemo-experimentalist/src/nemo_experimentalist_plugin/experimentalist/run.py
• plugins/nemo-experimentalist/src/nemo_experimentalist_plugin/preflight.py
• plugins/nemo-experimentalist/src/nemo_experimentalist_plugin/settings.py
• plugins/nemo-experimentalist/tests/conftest.py
• plugins/nemo-experimentalist/tests/experimentalist/test_model_config.py
• plugins/nemo-experimentalist/tests/experimentalist/test_terminator.py
• plugins/nemo-experimentalist/tests/experimentalist/test_tools.py
• plugins/nemo-experimentalist/tests/test_cli_profile.py
• plugins/nemo-experimentalist/tests/test_experiment_cli.py
• plugins/nemo-experimentalist/tests/test_experimentalist_benchmark.py
• plugins/nemo-experimentalist/tests/test_experimentalist_run.py
• plugins/nemo-experimentalist/tests/test_preflight.py
• plugins/nemo-experimentalist/tests/test_resolve.py
• plugins/nemo-insights/README.md
• plugins/nemo-insights/examples/research-agent/README.md
• plugins/nemo-insights/examples/research-agent/tests/test_analyst_e2e.py
• plugins/nemo-insights/src/nemo_insights_plugin/analyst/agent.py
• plugins/nemo-insights/src/nemo_insights_plugin/analyst/model_config.py
• plugins/nemo-insights/src/nemo_insights_plugin/analyst/run.py
• plugins/nemo-insights/src/nemo_insights_plugin/cli.py
• plugins/nemo-insights/src/nemo_insights_plugin/config.py
• plugins/nemo-insights/src/nemo_insights_plugin/controller.py
• plugins/nemo-insights/src/nemo_insights_plugin/entities.py
• plugins/nemo-insights/src/nemo_insights_plugin/jobs/analyze.py
• plugins/nemo-insights/src/nemo_insights_plugin/preflight.py
• plugins/nemo-insights/src/nemo_insights_plugin/schema.py
• plugins/nemo-insights/src/nemo_insights_plugin/sdk_resources/analysis_configs.py
• plugins/nemo-insights/src/nemo_insights_plugin/service.py
• plugins/nemo-insights/testbed/README.md
• plugins/nemo-insights/testbed/cli.py
• plugins/nemo-insights/testbed/eval/stack.py
• plugins/nemo-insights/tests/test_analysis_config_contract.py
• plugins/nemo-insights/tests/test_analyst_agent.py
• plugins/nemo-insights/tests/test_analyst_run.py
• plugins/nemo-insights/tests/test_cli_profile.py
• plugins/nemo-insights/tests/test_periodic_analysis.py
• plugins/nemo-insights/tests/test_preflight.py
• plugins/nemo-insights/tests/testbed/test_cli.py
• script/generate_config_docs.py

💤 Files with no reviewable changes (16)

• docs/set-up/config-reference.mdx
• plugins/nemo-eval-author/src/nemo_eval_author_plugin/model_config.py
• plugins/nemo-insights/src/nemo_insights_plugin/analyst/model_config.py
• plugins/nemo-experimentalist/tests/test_experimentalist_benchmark.py
• plugins/nemo-experimentalist/benchmarks/configs/terminal-bench-quality.yaml
• plugins/nemo-experimentalist/benchmarks/configs/tau3-smoke.yaml
• plugins/nemo-insights/src/nemo_insights_plugin/config.py
• plugins/nemo-insights/testbed/cli.py
• plugins/nemo-experimentalist/tests/experimentalist/test_model_config.py
• plugins/nemo-insights/tests/test_analyst_agent.py
• plugins/nemo-experimentalist/src/nemo_experimentalist_plugin/experimentalist/components/model_config.py
• plugins/nemo-experimentalist/benchmarks/configs/tau3-quality.yaml
• plugins/nemo-eval-author/tests/test_model_config.py
• plugins/nemo-experimentalist/src/nemo_experimentalist_plugin/settings.py
• script/generate_config_docs.py
• plugins/nemo-experimentalist/benchmarks/configs/terminal-bench-smoke.yaml

[github-actions]

🌿 Preview your docs: https://nvidia-preview-pr-1159-plugin-model-pairs-brnewsom.docs.buildwithfern.com/nemo-platform

[github-actions]

Suite	Lines Covered	Line Rate	Branch Rate
Unit Tests	31601/40224	78.6%	63.2%
Integration Tests	18382/38176	48.1%	20.8%

[coderabbitai]

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@docs/cli/reference.mdx`:
- Around line 50-51: Move the setup-text changes from the generated
docs/cli/reference.mdx output into its source document, then run the designated
documentation generation command to regenerate the reference page and commit the
updated generated output.

🪄 Autofix

Fix all unresolved CodeRabbit comments on this PR:

• Push a commit to this branch (recommended)
• Create a new PR with the fixes

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: 8770a6e6-6dbb-4903-8374-3d69eec5cc62

📥 Commits

Reviewing files that changed from the base of the PR and between 64add97 and d0f1aed.

📒 Files selected for processing (1)

• docs/cli/reference.mdx

[gaiadilorenzo]

Looks good to me but I'm missing a lot of nemo platform context