feat(agents): allow optimization agents to configure models

.github/actions/insights-intake-stack/action.yml

@@ -1,5 +1,5 @@
 name: insights-intake-stack
-description: Self-contained Intake (ClickHouse + auth,entities,intake) on :8080 over $RUNNER_TEMP/state
+description: Self-contained Analyst stack (ClickHouse + Intake and Platform model routing) on :8080
 runs:
   using: composite
   steps:

.github/workflows/insights-testbed.yml

@@ -114,6 +114,11 @@ jobs:
       # Same gateway key; litellm/tau2 read it under the OpenAI-conventional name.
       OPENAI_API_KEY: ${{ secrets.NVIDIA_INFERENCE_KEY }}
       OPENAI_API_BASE: ${{ secrets.NVIDIA_INFERENCE_URL }}
+      # nemo setup registers this provider and persists its credential in Platform Secrets.
+      NEMO_DEFAULT_INFERENCE_KEY: ${{ secrets.NVIDIA_INFERENCE_KEY }}
+      NEMO_DEFAULT_INFERENCE_BASE_URL: ${{ secrets.NVIDIA_INFERENCE_URL }}
+      NEMO_DEFAULT_MODEL: ${{ vars.INSIGHTS_TESTBED_MODEL || 'default/openai-openai-gpt-5-5' }}
+      NEMO_FAST_MODEL: ${{ vars.INSIGHTS_TESTBED_MODEL || 'default/openai-openai-gpt-5-5' }}
     steps:
       - name: Require secrets
         run: |
@@ -129,6 +134,11 @@ jobs:
           persist-credentials: false
       - uses: ./nemo-platform/.github/actions/insights-testbed-prep
       - uses: ./nemo-platform/.github/actions/insights-intake-stack
+      - name: Configure Analyst models
+        working-directory: nemo-platform
+        run: >-
+          uv run nemo --base-url http://localhost:8080 setup --auto --workspace default
+          --no-start-services --no-install-skills --no-deploy-agent
       - name: Run subjects (tau2 -> ingest -> insights)
         working-directory: nemo-platform/plugins/nemo-insights
         env:
@@ -188,14 +198,18 @@ jobs:
       matrix:
         subject: ${{ fromJSON(needs.plan.outputs.subjects) }}
     env:
-      INFERENCE_API_KEY: ${{ secrets.NVIDIA_INFERENCE_KEY }}
+      NEMO_DEFAULT_INFERENCE_KEY: ${{ secrets.NVIDIA_INFERENCE_KEY }}
+      NEMO_DEFAULT_INFERENCE_BASE_URL: ${{ secrets.NVIDIA_INFERENCE_URL }}
+      NEMO_DEFAULT_MODEL: ${{ vars.INSIGHTS_TESTBED_MODEL || 'default/openai-openai-gpt-5-5' }}
+      NEMO_FAST_MODEL: ${{ vars.INSIGHTS_TESTBED_MODEL || 'default/openai-openai-gpt-5-5' }}
     steps:
       - name: Require secrets
         env:
           GH_TOKEN: ${{ secrets.TESTBED_STATE_GH_READ_TOKEN }}
         run: |
           [ -n "$GH_TOKEN" ] || { echo "secret TESTBED_STATE_GH_READ_TOKEN is not set"; exit 1; }
-          [ -n "$INFERENCE_API_KEY" ] || { echo "secret NVIDIA_INFERENCE_KEY is not set"; exit 1; }
+          [ -n "$NEMO_DEFAULT_INFERENCE_KEY" ] || { echo "secret NVIDIA_INFERENCE_KEY is not set"; exit 1; }
+          [ -n "$NEMO_DEFAULT_INFERENCE_BASE_URL" ] || { echo "secret NVIDIA_INFERENCE_URL is not set"; exit 1; }
       - uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6.0.3
         with: { path: nemo-platform, persist-credentials: false }
       - uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6.0.3
@@ -207,6 +221,11 @@ jobs:
       - uses: ./nemo-platform/.github/actions/insights-testbed-prep
         with: { install-tau2: "false" }
       - uses: ./nemo-platform/.github/actions/insights-intake-stack
+      - name: Configure Analyst models
+        working-directory: nemo-platform
+        run: >-
+          uv run nemo --base-url http://localhost:8080 setup --auto --workspace default
+          --no-start-services --no-install-skills --no-deploy-agent
       - name: Generate insights
         working-directory: nemo-platform/plugins/nemo-insights
         env:

README.md

@@ -39,7 +39,7 @@ source .venv/bin/activate
 nemo setup
 ```
 
-`nemo setup` starts local services, registers your LLM provider, discovers available models, installs agent skills, and deploys a sample agent (see more below).
+`nemo setup` starts local services, registers your LLM provider, discovers available models, selects default and fast agent models, installs agent skills, and deploys a sample agent (see more below).
 
 Review [Telemetry and Privacy](docs/telemetry-and-privacy.mdx) for the omnibus disclosure covering anonymous telemetry, bundled library telemetry, third-party endpoint notes, and opt-out controls.
 
@@ -91,6 +91,7 @@ If `make bootstrap` reports that Studio asset bootstrap did not complete, the AP
 ```bash
 export NVIDIA_API_KEY=nvapi...
 export NEMO_DEFAULT_MODEL=nvidia-nemotron-3-super-120b-a12b
+export NEMO_FAST_MODEL="$NEMO_DEFAULT_MODEL"
 nemo setup --auto --start-services --install-skills --deploy-agent
 ```
 

SETUP.md

@@ -1,6 +1,6 @@
 ---
 name: nemo-setup
-description: Set up a local NeMo Platform (`make bootstrap` + `nemo setup`) — services, providers, plugins, default model, and an optional demo agent. Use when the user asks to install, bootstrap, set up, run, or start a local NeMo Platform.
+description: Set up a local NeMo Platform (`make bootstrap` + `nemo setup`) — services, providers, plugins, default/fast models, and an optional demo agent. Use when the user asks to install, bootstrap, set up, run, or start a local NeMo Platform.
 version: "0.1"
 ---
 
@@ -69,14 +69,14 @@ Replace the path with whatever was chosen in Q2 (`$NMP_DATA_DIR`, `$XDG_DATA_HOM
 
 ## Bootstrap and start
 
-The README documents the streamlined path. Prefer this over the manual steps below whenever the task fits — it covers prerequisites install, service startup, provider registration, default-model selection, and demo agent deployment in one shot:
+The README documents the streamlined path. Prefer this over the manual steps below whenever the task fits — it covers prerequisites install, service startup, provider registration, default/fast model selection, and demo agent deployment in one shot:
 
 === "Interactive"
 
 ```bash
 make bootstrap           # installs Python deps, Studio assets, and plugins (including demo calculator agent)
 source .venv/bin/activate
-nemo setup               # interactive: prompts for provider, picks default model, optionally deploys calculator-agent
+nemo setup               # interactive: prompts for provider, picks default/fast models, optionally deploys calculator-agent
 ```
 
 === "Non-interactive (CI)"
@@ -97,18 +97,20 @@ nemo setup --auto --start-services --install-skills --deploy-agent
 
 If `nemo setup` is too high-level for the task (e.g. debugging startup, custom service set, custom plugin install after bootstrap), use the manual sections below.
 
-### Default model selection (under `--auto`)
+### Default and fast model selection
 
-`$NEMO_DEFAULT_MODEL` **must be a hyphenated entity ID** from `nemo models list` (e.g. `nvidia-llama-3-3-nemotron-super-49b-v1-5` or `default/nvidia-llama-3-3-nemotron-super-49b-v1-5`). The slash-with-dots form (`nvidia/llama-3.3-nemotron-super-49b-v1-5`) is the upstream catalog's `served_model_name` — it's shown for human display but the gateway rejects it as a request input.
+`$NEMO_DEFAULT_MODEL` and `$NEMO_FAST_MODEL` **must be hyphenated entity IDs** from `nemo models list` (e.g. `nvidia-llama-3-3-nemotron-super-49b-v1-5` or `default/nvidia-llama-3-3-nemotron-super-49b-v1-5`). The slash-with-dots form (`nvidia/llama-3.3-nemotron-super-49b-v1-5`) is the upstream catalog's `served_model_name` — it's shown for human display but the gateway rejects it as a request input.
 
 `nemo setup --auto` picks the default model in this order:
 
 1. `$NEMO_DEFAULT_MODEL` (if set) — used as-is. The user may have exported this from a previous session; it takes precedence over anything discovered from the registered provider.
 2. Otherwise, the first model entity returned by provider discovery.
 
-If the user is surprised by which model got picked, check `echo $NEMO_DEFAULT_MODEL` first — that's the most common cause.
+The fast model uses `$NEMO_FAST_MODEL` when set and otherwise reuses the effective default. Interactive setup prompts for both and defaults the fast choice to the selected default. Existing contexts without `fast_model` also reuse `default_model`.
 
-The first-discovered fallback is intentionally simple — providers like NVIDIA Build expose dozens of models and "first one" rarely matches the user's intent. If the user wants a specific model as the default (or wants to compare options before committing), don't rely on the `--auto` fallback. After setup finishes, the `inference` skill's "Step 2 — Discover available models" enumerates entity IDs and shows the jq filters for picking one out by vendor or family. The user can then pin their choice via `export NEMO_DEFAULT_MODEL=<workspace>/<entity-id>` or by overriding `body["model"]` per request.
+If the user is surprised by which model got picked, check `NEMO_DEFAULT_MODEL` and `NEMO_FAST_MODEL` first — that's the most common cause.
+
+The first-discovered fallback is intentionally simple — providers like NVIDIA Build expose dozens of models and "first one" rarely matches the user's intent. If the user wants specific models (or wants to compare options before committing), don't rely on the `--auto` fallback. After setup finishes, the `inference` skill's "Step 2 — Discover available models" enumerates entity IDs and shows the jq filters for picking one out by vendor or family. The user can then pin the pair via `NEMO_DEFAULT_MODEL` and `NEMO_FAST_MODEL`, or override `body["model"]` per request.
 
 If `nemo agents invoke …` fails with HTTP 422 in under a second on the first call, the cause is almost always a slash-with-dots model name reaching the gateway (e.g. via a stale `NEMO_DEFAULT_MODEL` or an agent config that hardcoded the upstream catalog form). The `inference` skill's "Common failure: HTTP 422 from chat completion" subsection has the diagnose-and-recover steps — don't conclude the platform is broken.
 
@@ -169,7 +171,7 @@ The platform is running. Don't leave the user with "you're good to go" — offer
 
 If the user's prompt doesn't already pin one down, ask: *"The platform is up. What would you like to do next — optimize an agent, deploy one, run inference, evaluate, generate data, or something else?"*
 
-If the user wants to **pick or swap the default model** (e.g. they didn't like the one `--auto` selected, or they want to compare options), don't guess — hand off to the `inference` skill. Step 2 there enumerates `served_models[].model_entity_id` and shows jq filters for picking by vendor / family. To pin the choice for subsequent runs, export `NEMO_DEFAULT_MODEL=<workspace>/<entity-id>` before the next `nemo setup --auto`. For one-off commands, pass the entity ID positionally: `nemo chat <entity-id>`.
+If the user wants to **pick or swap the default/fast model pair**, don't guess — hand off to the `inference` skill. Step 2 there enumerates `served_models[].model_entity_id` and shows jq filters for picking by vendor / family. To pin the choices for subsequent runs, export `NEMO_DEFAULT_MODEL=<workspace>/<entity-id>` and `NEMO_FAST_MODEL=<workspace>/<entity-id>` before the next `nemo setup --auto`; fast falls back to default when omitted. For one-off commands, pass the entity ID positionally: `nemo chat <entity-id>`.
 
 ### Available skills
 

docs/agents/insight-driven-optimization.mdx

@@ -49,10 +49,9 @@ want to automate portions of the agent improvement loop.
   runtime configuration) and a way to run or evaluate the agent reproducibly.
 - Traces in Intake for the agent under test. The Analyst diagnoses only what
   it can observe, so an agent with no telemetry cannot be analyzed.
-- Model access for the optimization agents themselves. The Analyst uses Claude Opus 4.8 through the
-  NVIDIA Inference Gateway. The Experimenter and Eval Author use configurable large language models
-  (LLMs) through an OpenAI-compatible endpoint. This access is separate from the model access that
-  your AUT needs at runtime.
+- A supported inference provider and default/fast model pair configured by
+  `nemo setup` for the optimization agents. This access is separate from the
+  model access that your agent under test needs at runtime.
 - Train and validation datasets in a Harbor-compatible layout, plus a task
   template and an output directory for artifacts. This is what the Eval Author
   works from and what the Experimenter uses to validate changes.
@@ -292,8 +291,8 @@ their implementations are available.
 
 ### Set Up
 
-Point the CLI at a platform and set model access for the optimization agents.
-The examples use a local platform; replace the base URL for a remote
+Point the CLI at a platform and configure model access for the optimization
+agents. The examples use a local platform; replace the base URL for a remote
 deployment.
 
 ```bash
@@ -303,17 +302,16 @@ export NMP_STUDIO_URL="$NMP_BASE_URL/studio"
 export NMP_ACCESS_TOKEN="$(nemo auth token)"
 export AGENT=your-agent-name           # This agent must have traces in Intake.
 
-# The Analyst uses Claude Opus 4.8 through NVIDIA Inference Gateway.
-export INFERENCE_API_KEY=sk-...
-
-# The Experimenter and Eval Author use an OpenAI-compatible endpoint.
-export NEMO_EXPERIMENTALIST_API_BASE=https://inference-api.nvidia.com/v1
-export NEMO_EXPERIMENTALIST_API_KEY=sk-...
-export NEMO_EXPERIMENTALIST_MODELS_SMART=openai/openai/openai/gpt-5.6-sol
-export NEMO_EXPERIMENTALIST_MODELS_MID=openai/openai/openai/gpt-5.6-terra
-export NEMO_EXPERIMENTALIST_MODELS_FAST=openai/openai/openai/gpt-5.6-luna
+nemo setup
 ```
 
+During setup, register a supported provider and select the default and fast
+Platform Model Entities. The default model handles quality-critical work and
+the fast model handles latency-sensitive work; press Enter at the fast-model
+prompt to reuse the default. The Analyst, Eval Author, and Experimentalist all
+use this pair. Provider credentials are stored in Platform Secrets rather than
+plugin-specific environment variables.
+
 Set `NMP_STUDIO_URL` explicitly if Studio uses a different origin or path.
 For an authenticated remote deployment, log in against that base URL before
 requesting the token. For local development with authentication disabled, omit
@@ -457,8 +455,8 @@ come from the profile:
 nemo agents experimentalist run
 ```
 
-The Experimenter's own models are configured through environment variables
-(refer to [Models](#models)). Train and validation datasets are required, and they must be local
+The Experimenter's own models come from the default/fast pair selected by
+`nemo setup` (refer to [Models](#models)). Train and validation datasets are required, and they must be local
 paths for the Harbor evaluator. The loop runs a baseline evaluation,
 performs root cause analysis, implements changes, then validates and picks a
 winner.
@@ -499,6 +497,10 @@ per agent that is *due* and has enough new telemetry. Runs are incremental:
 each successful run records a cursor (`last_successful_run_at`), and subsequent
 runs only consider traces newer than that cursor.
 
+`enable` captures the effective default/fast pair in the server-side analysis
+config. Re-run it after changing the pair with `nemo setup`; enabled records
+created before model-pair persistence must also be re-enabled.
+
 ### Configure the Analysis Schedule
 
 The global schedule lives in the Insights plugin configuration or environment. The `timezone`
@@ -547,7 +549,8 @@ Run the Analyst for one analysis of an agent's traces.
 
 Manage per-agent opt-in for periodic analysis. `enable` and `disable` require
 `--agent`; `status` takes an optional `--agent` (omit it to list all configs in
-the workspace). All three accept `--workspace` and `--base-url`.
+the workspace). `enable` stores the active default/fast model pair for future
+scheduled jobs. All three accept `--workspace` and `--base-url`.
 
 ### `nemo agents analyst doctor`
 
@@ -577,7 +580,7 @@ Run the local Experimenter loop.
 
 ### `nemo agents experimentalist doctor`
 
-Diagnose the Experimenter setup: profile, credentials, Insight resolution,
+Diagnose the Experimenter setup: profile, configured models, Insight resolution,
 datasets, and the experiment plan.
 
 ### `nemo agents eval-author`
@@ -588,21 +591,26 @@ exit with a nonzero status until their implementations are available.
 
 ### Models
 
-The optimization agents use the following model configuration:
+The optimization agents use the active Platform CLI context:
 
 | Variable | Used by | Default and notes |
 |----------|---------|-------------------|
-| `INFERENCE_API_KEY` | Analyst | Required. API key for Claude Opus 4.8 through the NVIDIA Inference Gateway. The Analyst model and endpoint are fixed. |
-| `NEMO_EXPERIMENTALIST_API_BASE` | Experimenter, Eval Author | OpenAI-compatible model API base URL. `run` and `doctor` fall back to the NVIDIA Inference Gateway when it is unset, so it is only required against another endpoint. Also settable as `api_base` under the `experimentalist:` config section; the environment wins. |
-| `NEMO_EXPERIMENTALIST_API_KEY` | Experimenter, Eval Author | Required. Model API key. On the gateway, `INFERENCE_API_KEY` fills this. |
-| `NEMO_EXPERIMENTALIST_MODELS_SMART` | Experimenter, Eval Author | Required. High-capability model for analysis, proposing, coding, and curation. No default: a model name is only meaningful against a specific endpoint. |
-| `NEMO_EXPERIMENTALIST_MODELS_MID` | Experimenter, Eval Author | Required. Mid-tier model used for trajectory scoring and architecture documentation. No default. |
-| `NEMO_EXPERIMENTALIST_MODELS_FAST` | Experimenter, Eval Author | Required. Low-latency model for lightweight steps such as termination checks and summarization. No default. |
-| `AUTHOR_API_BASE` | Eval Author | Optional override for the OpenAI-compatible model API base URL. Falls back to `NEMO_EXPERIMENTALIST_API_BASE`. |
-| `AUTHOR_API_KEY` | Eval Author | Optional override for the model API key. Falls back to `NEMO_EXPERIMENTALIST_API_KEY`, or to `INFERENCE_API_KEY` for the NVIDIA Inference Gateway. |
-| `AUTHOR_SMART_MODEL_NAME` | Eval Author | Optional high-capability model override. |
-| `AUTHOR_MID_MODEL_NAME` | Eval Author | Optional mid-tier model override for Experimentalist helpers used by Eval Author. |
-| `AUTHOR_FAST_MODEL_NAME` | Eval Author | Optional low-latency model override. |
+| stored `default_model` | Analyst, Experimenter, Eval Author | Required workspace-qualified Model Entity selected by `nemo setup`. Used for quality-critical analysis, authoring, proposing, and coding. |
+| stored `fast_model` | Analyst, Experimenter, Eval Author | Selected by `nemo setup`; defaults to the default model. Used for high-volume scoring, summarization, and control steps. |
+| `NEMO_DEFAULT_MODEL` | Analyst, Experimenter, Eval Author | Optional non-interactive override for `default_model`, in `workspace/model-name` form. |
+| `NEMO_FAST_MODEL` | Analyst, Experimenter, Eval Author | Optional non-interactive override for `fast_model`, in `workspace/model-name` form. Falls back to the effective default model. |
+
+Each Model Entity owns its backend format and provider route. OpenAI-compatible
+and Anthropic entities are adapted to Nooa's unified completion client through the
+Platform route; the agents never inspect provider model-name conventions or
+read provider credentials directly.
+
+For OpenAI Chat Completions entities, the shared client disables reasoning while
+function tools are active and disables LiteLLM's automatic Responses bridge.
+This keeps frontier models on the Model Entity's declared Chat Completions
+contract without a model-name allowlist; LiteLLM drops the reasoning parameter
+for models that do not support it. Anthropic entities use their native Messages
+route.
 
 ## Troubleshooting
 
@@ -616,7 +624,8 @@ flowing and that you are filtering by the right `agent` name.
 Check that the agent is enabled
 (`nemo insights analysis status`), that `insights.analyst.enabled` is true,
 that the scheduled window has passed in the configured `timezone`, and that at
-least 10 new sessions have landed since the last run.
+least 10 new sessions have landed since the last run. If status shows no model
+selection, re-run `nemo insights analysis enable` after `nemo setup`.
 
 ### Insights Are Not in the Platform