feat(optimization): Agents-owned Fabric Optuna HPO with Hermes + MCP (AALGO-277)

docs/agents/index.mdx

@@ -2,6 +2,7 @@
 title: "About Agents"
 description: ""
 ---
+
 <a id="agents"></a>
 
 An agent on NeMo Platform calls tools, accesses models through shared Platform
@@ -43,29 +44,30 @@ undeploying the candidate.
 
 Agents are managed end-to-end through the `nemo agents` command group:
 
-| Stage | Command | What it does |
-|-------|---------|--------------|
-| Register | `nemo agents create --name <name> --agent-config <path>` | Store the agent configuration as an `agent` entity in a workspace. |
-| Deploy | `nemo agents deploy --agent <name>` | Start a running service from the stored config. |
-| Wait | `nemo agents deployments wait --agent <name>` | Block until the deployment is `running` or `failed`. |
-| Invoke | `nemo agents invoke --agent <name> --input "..."` or `nemo agents invoke --agent-config <path> --input "..."` | Send a single request through the Agents gateway or run a local config directly. |
-| Tear down | `nemo agents undeploy --agent <name>` then `nemo agents delete <name>` | Stop the running service and remove the agent entity. |
+| Stage     | Command                                                                                                       | What it does                                                                     |
+| --------- | ------------------------------------------------------------------------------------------------------------- | -------------------------------------------------------------------------------- |
+| Register  | `nemo agents create --name <name> --agent-config <path>`                                                      | Store the agent configuration as an `agent` entity in a workspace.               |
+| Deploy    | `nemo agents deploy --agent <name>`                                                                           | Start a running service from the stored config.                                  |
+| Wait      | `nemo agents deployments wait --agent <name>`                                                                 | Block until the deployment is `running` or `failed`.                             |
+| Invoke    | `nemo agents invoke --agent <name> --input "..."` or `nemo agents invoke --agent-config <path> --input "..."` | Send a single request through the Agents gateway or run a local config directly. |
+| Optimize  | `nemo agents optimize run --optimize-config <path>`                                                           | Run Fabric-backed numeric HPO (Optuna). See [Optimize Agents](/documentation/agents/optimize-agents). |
+| Tear down | `nemo agents undeploy --agent <name>` then `nemo agents delete <name>`                                        | Stop the running service and remove the agent entity.                            |
 
 To run an `agent.yaml` directly without registering it on the platform, pass `--agent-config <path>` to `nemo agents invoke` or `nemo agents run`.
 
 Legacy NAT-only commands:
 
-| Stage | Command | What it does |
-|-------|---------|--------------|
-| Evaluate | `nemo agents evaluate run --eval-config <path> --agent <name>` | Run a NAT evaluation against the deployed agent. |
-| Optimize | `nemo agents optimize run --optimize-config <path> --agent <name>` | Run NAT parameter or prompt tuning trials against the agent's stored config. |
+| Stage    | Command                                                            | What it does                                                                 |
+| -------- | ------------------------------------------------------------------ | ---------------------------------------------------------------------------- |
+| Evaluate | `nemo agents evaluate run --eval-config <path> --agent <name>`     | Run a NAT evaluation against the deployed agent.                             |
 
 ## Agent Definition
 
 ### Platform-managed agents
 
 #### About NeMo Fabric
-NeMo Fabric gives users one configurable, observable way to run applications across multiple agent harnesses. 
+
+NeMo Fabric gives users one configurable, observable way to run applications across multiple agent harnesses.
 It standardizes configuration, lifecycle management, and results without requiring a separate integration for every harness.
 To learn more about Fabric, see
 [NeMo Fabric](https://docs.nvidia.com/nemo/fabric/about-nemo-fabric/overview/).
@@ -74,7 +76,6 @@ NeMo Platform uses Fabric as the runtime wrapper around your agent so the
 platform can deploy it and route its model traffic
 through shared infrastructure.
 
-
 An agent's behavior is described by the platform-managed `agent.yaml`:
 
 | Section | Purpose | Example |
@@ -165,14 +166,13 @@ platform can deploy it, evaluate it, optimize it, and route its model traffic
 through shared infrastructure. For the toolkit itself, see the
 [NeMo Agent Toolkit documentation](https://docs.nvidia.com/nemo/agent-toolkit/latest/).
 
-
 An agent's behavior is described by a NAT workflow YAML with three top-level sections:
 
-| Section | Purpose | Example |
-|---------|---------|---------|
-| `functions` | Tools the agent can call | `wiki_search`, `current_datetime`, custom MCP tools |
-| `llms` | Model bindings the workflow can reference | OpenAI-compatible endpoints, NIM endpoints |
-| `workflow` | The agent type and its wiring | `react_agent`, `tool_calling_agent`, custom NAT workflows |
+| Section     | Purpose                                   | Example                                                   |
+| ----------- | ----------------------------------------- | --------------------------------------------------------- |
+| `functions` | Tools the agent can call                  | `wiki_search`, `current_datetime`, custom MCP tools       |
+| `llms`      | Model bindings the workflow can reference | OpenAI-compatible endpoints, NIM endpoints                |
+| `workflow`  | The agent type and its wiring             | `react_agent`, `tool_calling_agent`, custom NAT workflows |
 
 ReAct is a common agent pattern where the model alternates between a
 reasoning step and a tool call until it has enough information to answer. It
@@ -212,8 +212,9 @@ config targets a deployed agent:
   durable container on Docker or Kubernetes, and invoke it through the Agents gateway.
 - [Observe Agents](/documentation/agents/observe-agents): ingest and query agent telemetry with NeMo
   Intake, then review traces, feedback, and evaluator results.
-- [Optimize Agents](/documentation/agents/optimize-agents): analyze deployed agents for model routing,
-  skill, prompt, and new-model opportunities. (Applies to NAT workflows)
+- [Optimize Agents](/documentation/agents/optimize-agents): Fabric-backed numeric HPO
+  (`nemo agents optimize run`), plus model-routing / skill / prompt suggestions for
+  deployed agents.
 - [Secure Agents](/documentation/agents/secure-agents): check guardrail coverage and scan recent
   telemetry for sensitive data.
 - [Plugins and Skills](/documentation/agents/plugins-and-skills): understand how agent, middleware, and

docs/agents/optimization.mdx

@@ -2,6 +2,7 @@
 title: "Optimize Agents"
 description: ""
 ---
+
 <a id="agents-optimization"></a>
 
 Use the Agent Optimizer to analyze a deployed agent and act on improvement
@@ -16,12 +17,12 @@ evaluation result before promotion.
 
 ## What the Optimizer Checks
 
-| Suggestion type | Signal | Result |
-|-----------------|--------|--------|
-| Model optimization | An agent uses a single frontier model where a smaller model or route split may preserve quality at lower cost | Suggests a model swap or Switchyard random-routing virtual model |
-| Skill optimization | The agent uses skills and has an evaluation suite | Suggests running `nemo agents optimize-skills` to improve skill files and keep changes that pass evaluation |
-| Prompt optimization | The agent has an optimization config and baseline dataset | Suggests `nemo agents optimize run` for NAT prompt or parameter tuning |
-| New model scan | Difference between the current model list and the previous optimizer snapshot | Suggests evaluating or auditing newly available models |
+| Suggestion type     | Signal                                                                                                        | Result                                                                                                      |
+| ------------------- | ------------------------------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------- |
+| Model optimization  | An agent uses a single frontier model where a smaller model or route split may preserve quality at lower cost | Suggests a model swap or Switchyard random-routing virtual model                                            |
+| Skill optimization  | The agent uses skills and has an evaluation suite                                                             | Suggests running `nemo agents optimize-skills` to improve skill files and keep changes that pass evaluation |
+| Prompt optimization | The agent has an optimization config and baseline dataset                                                     | Suggests `nemo agents optimize run` for Fabric-backed tuning                                                |
+| New model scan      | Difference between the current model list and the previous optimizer snapshot                                 | Suggests evaluating or auditing newly available models                                                      |
 
 Optimizer state is stored in the `nemo-agent-optimizer` fileset:
 
@@ -38,7 +39,8 @@ Before running the optimizer, make sure you have:
 1. Local services running (`nemo services run`).
 1. The agents plugin installed. For local development from this repository:
    ```bash
-   uv pip install -e packages/nemo_platform_plugin -e plugins/nemo-agents
+   uv sync --package nemo-agents-plugin
+   source .venv/bin/activate   # puts `nemo` on PATH
    ```
 1. A workspace with at least one model provider and discovered model entities.
 1. At least one deployed platform-managed agent.
@@ -268,20 +270,30 @@ nemo files list nemo-agent-telemetry
 
 ## Run Prompt and Parameter Tuning
 
-The `nemo agents optimize run` command runs the NAT optimizer path for
-parameter or prompt tuning. Use it when you already have a NAT optimization
-YAML and want to run `nat optimize` through the Agents plugin.
+The `nemo agents optimize run` command runs Fabric-backed numeric
+optimization through `agents.optimize` (implementation in
+`nemo-optimization`). Input must be a Fabric-native agent package
+(`schema_version: fabric.agent/v1alpha1`). The golden-path harness is
+Hermes (`nvidia.fabric.hermes`); see
+`plugins/nemo-optimization/examples/hermes-optimize/` (install steps live in
+that README).
+
+After `uv sync --package nemo-agents-plugin` (and activating `.venv`), invoke
+`nemo` directly.
 
-For the ReAct example:
+`--optimize-config` must be an **absolute** path. Run from the
+`nemo-platform` repo root so dataset / `base_dir` paths resolve.
+
+### Chat-only Hermes (no MCP)
 
 <Tabs>
 
 <Tab title="CLI">
 
 ```bash
 nemo agents optimize run \
-  --optimize-config plugins/nemo-agents/examples/react-agent/react-optimize.yml \
-  --agent react-agent
+  --optimize-config "$(pwd)/plugins/nemo-optimization/examples/hermes-optimize/optimize-chatonly.yaml" \
+  --workspace default
 ```
 
 </Tab>
@@ -301,7 +313,7 @@ nemo skills show agents-optimize
 
 What it does under the hood:
 
-- Confirms the agent has a NAT optimization YAML.
+- Confirms the agent has a Fabric-native optimization YAML.
 - Runs `nemo agents optimize run` (or `submit` for platform jobs).
 - Compares results against the evaluation baseline and surfaces deltas
   for review.
@@ -313,23 +325,24 @@ What it does under the hood:
 import os
 from pathlib import Path
 
-from nemo_agents_plugin.jobs.optimize_agent import OptimizeAgentJob
+from nemo_optimization.jobs.optimize import OptimizeJob
 from nemo_platform import NeMoPlatform
 from nemo_platform_plugin.scheduler import NemoJobScheduler
 
 WORKSPACE = "default"
-optimize_config = Path("plugins/nemo-agents/examples/react-agent/react-optimize.yml")
+optimize_config = Path(
+    "plugins/nemo-optimization/examples/hermes-optimize/optimize-chatonly.yaml"
+).resolve()
 
 client = NeMoPlatform(
     base_url=os.environ.get("NMP_BASE_URL", "http://localhost:8080"),
     workspace=WORKSPACE,
 )
 
 result = NemoJobScheduler().run_local(
-    OptimizeAgentJob,
+    OptimizeJob,
     {
         "optimize_config": str(optimize_config),
-        "agent": "react-agent",
         "workspace": WORKSPACE,
     },
     workspace=WORKSPACE,
@@ -341,11 +354,87 @@ print(result)
 </Tab>
 
 </Tabs>
+
+### MCP Hermes (phishing analyzer)
+
+Point `PHISHING_AGENT_SRC` / `PHISHING_MCP_BIN` at an
+`email-phishing-analyzer-harnesses` checkout (its own `.venv` after
+`uv sync`). Do **not** pip-install that agent into the platform venv.
+Full setup steps are in `plugins/nemo-optimization/examples/hermes-optimize/README.md`.
+
+<Tabs>
+
+<Tab title="CLI">
+
+```bash
+export PHISHING_AGENT_ROOT="${PHISHING_AGENT_ROOT:-$HOME/work/email-phishing-analyzer-harnesses}"
+export PHISHING_AGENT_SRC="$PHISHING_AGENT_ROOT/src"
+export PHISHING_MCP_BIN="$PHISHING_AGENT_ROOT/.venv/bin/email-phishing-analyzer-mcp"
+
+# These environment variables are templated into optimize-mcp.yaml.
+
+nemo agents optimize run \
+  --optimize-config "$(pwd)/plugins/nemo-optimization/examples/hermes-optimize/optimize-mcp.yaml" \
+  --workspace default
+```
+
+</Tab>
+<Tab title="Python SDK">
+
+```python
+import os
+from pathlib import Path
+
+from nemo_optimization.jobs.optimize import OptimizeJob
+from nemo_platform import NeMoPlatform
+from nemo_platform_plugin.scheduler import NemoJobScheduler
+
+WORKSPACE = "default"
+agent_root = Path(
+    os.environ.get(
+        "PHISHING_AGENT_ROOT",
+        Path.home() / "work/email-phishing-analyzer-harnesses",
+    )
+)
+os.environ.setdefault("PHISHING_AGENT_SRC", str(agent_root / "src"))
+os.environ.setdefault(
+    "PHISHING_MCP_BIN",
+    str(agent_root / ".venv/bin/email-phishing-analyzer-mcp"),
+)
+
+optimize_config = Path(
+    "plugins/nemo-optimization/examples/hermes-optimize/optimize-mcp.yaml"
+).resolve()
+
+client = NeMoPlatform(
+    base_url=os.environ.get("NMP_BASE_URL", "http://localhost:8080"),
+    workspace=WORKSPACE,
+)
+
+result = NemoJobScheduler().run_local(
+    OptimizeJob,
+    {
+        "optimize_config": str(optimize_config),
+        "workspace": WORKSPACE,
+    },
+    workspace=WORKSPACE,
+    sdk=client,
+)
+print(result)
+```
+
+</Tab>
+
+</Tabs>
+
 When `--agent` is a platform-managed agent name, the job fetches the stored
-agent config, merges it with the optimization config, injects the Inference
-Gateway URL, and runs trials locally. When `--agent` is a raw HTTP endpoint,
-the endpoint is treated as an opaque remote service, so local parameter sweeps
-do not change the remote agent behavior.
+Fabric agent config, overlays the optimization settings, runs Inference Gateway
+model preflight, and dispatches to the Tune backend. `--agent` must be a
+workspace agent name (`hermes-optimize-chatonly` or
+`default/hermes-optimize-chatonly`). Endpoint URLs and other URI forms
+(`http://...`, `https://...`, `file://...`) are rejected -- raw HTTP endpoint
+optimize mode was removed. Use a platform-managed agent reference or an inline
+Fabric agent package in `--optimize-config`.
 
 ## Troubleshooting
 

docs/cli/reference.mdx

@@ -110,7 +110,7 @@ Manage authentication for NeMo Platform.
 **Usage:**
 
 ```shell
-nemo auth [OPTIONS] COMMAND [ARGS]...
+nemo auth [OPTIONS] [COMMAND] [ARGS]...
 ```
 
 **Help:**
@@ -297,7 +297,7 @@ Run platform services locally.
 **Usage:**
 
 ```shell
-nemo services [OPTIONS] COMMAND [ARGS]...
+nemo services [OPTIONS] [COMMAND] [ARGS]...
 ```
 
 **Help:**
@@ -607,7 +607,7 @@ nemo skills install --agent claude --skill inference
 **Usage:**
 
 ```shell
-nemo skills [OPTIONS] COMMAND [ARGS]...
+nemo skills [OPTIONS] [COMMAND] [ARGS]...
 ```
 
 **Help:**
@@ -954,7 +954,7 @@ nemo agent commands
 **Usage:**
 
 ```shell
-nemo agent [OPTIONS] COMMAND [ARGS]...
+nemo agent [OPTIONS] [COMMAND] [ARGS]...
 ```
 
 **Help:**
@@ -1027,7 +1027,7 @@ nemo plugins list
 **Usage:**
 
 ```shell
-nemo plugins [OPTIONS] COMMAND [ARGS]...
+nemo plugins [OPTIONS] [COMMAND] [ARGS]...
 ```
 
 **Help:**