feat(skills): add nemo-analyst workflow

AGENTS.md

@@ -36,7 +36,7 @@ User-facing skills in `packages/nemo_platform_ext/src/nemo_platform_ext/skills/`
 - `nemo-status`: read-only health dashboard.
 - `nemo-teardown`: guided shutdown with confirmation.
 
-Plugin-owned skills under `plugins/*/src/*/skills/` handle their own routing for customization, guardrails, evaluations, optimization, data designer, anonymizer, auditor, and Experimentalist source/harness improvement.
+Plugin-owned skills under `plugins/*/src/*/skills/` handle their own routing for customization, guardrails, evaluations, optimization, data designer, anonymizer, auditor, Experimentalist source/harness improvement, and Analyst telemetry analysis.
 
 ### Working in a sandboxed environment
 

CLAUDE.md

@@ -34,7 +34,7 @@ User-facing skills in `packages/nemo_platform_ext/src/nemo_platform_ext/skills/`
 - `nemo-status`: read-only health dashboard. Run this before assuming the platform is up.
 - `nemo-teardown`: guided shutdown with confirmation.
 
-Plugin-owned skills live under `plugins/*/src/*/skills/` and handle their own routing for customization, guardrails, evaluations, optimization, data designer, anonymizer, auditor, and Experimentalist source/harness improvement.
+Plugin-owned skills live under `plugins/*/src/*/skills/` and handle their own routing for customization, guardrails, evaluations, optimization, data designer, anonymizer, auditor, Experimentalist source/harness improvement, and Analyst telemetry analysis.
 
 ### Working in a sandboxed coding-agent environment
 

packages/nemo_platform_ext/src/nemo_platform_ext/skills/nemo-skill-selection/SKILL.md

@@ -62,6 +62,7 @@ Match the user's intent to one downstream skill. Pick exactly one.
 | "status", "what is running", "platform health", "is the platform up", "what's deployed", "show me what's running" | `nemo-status` | Read-only dashboard: platform, agents, providers, models |
 | "shut down", "stop NeMo", "tear down", "clean up" | `nemo-teardown` | Stop the cluster (keep data, delete platform data, or full cleanup) |
 | "fine-tune", "customize the model", "train on my data", "SFT", "LoRA" | `nemo-customizer` | Model customization via installed customization contributor plugins (`nemo-customizer-plugin`). Requires plugin skills to be installed (`nemo skills install` / enabled-plugins). |
+| "why does my agent keep failing", "analyze my agent's traces", "find recurring failure patterns", "generate insights for my agent" | `nemo-analyst` (plugin-owned, in `plugins/nemo-insights`) | Reads an agent's existing telemetry and files each recurring failure pattern as an Insight citing the traces that evidence it. Requires the Insights plugin; produces the Insight `nemo-experimentalist` acts on. |
 | "improve the agent's own code", "fix my agent harness", "candidate code change", "optimize from an Insight", "improve on train and validation datasets" | `nemo-experimentalist` (plugin-owned, in `plugins/nemo-experimentalist`) | Source/harness optimization: generate and validate candidate code changes against Harbor-compatible evaluation data. Requires the Experimentalist plugin; use after `agents analyst` has created an Insight, or with explicit datasets. |
 | "optimize my agent", "make it cheaper", "reduce latency", "smaller model", "switchyard", "routing split", "compare against a newer model" | `agents-optimize` (plugin-owned, in `plugins/nemo-agents`) | Cost / latency / quality optimization for a **deployed** agent. Routing splits, skill tuning, prompt tuning, new-model scans. |
 | "secure my agent", "harden my agent", "check for PII", "leaked secrets", "guardrail coverage" | `agents-secure` (plugin-owned, in `plugins/nemo-agents`) | Safety and security audit for a **deployed** agent. Guardrails, PII, secrets scan. |
@@ -133,6 +134,7 @@ Plugin-owned skills:
   agents-secure     safety and security audit for a deployed agent
   nemo-evaluator    evaluation metrics, LLM-judge, benchmark jobs
   nemo-customizer   fine-tuning of models
+  nemo-analyst      analyze agent telemetry and file recurring problems as Insights
   nemo-experimentalist  source/harness optimization from Insights or evaluation datasets
   guardrails        content-safety middleware via virtual models
   auditor           red-team vulnerability scanning (garak)

plugins/nemo-insights/pyproject.toml

@@ -35,6 +35,9 @@ insights-analysis = "nemo_insights_plugin.controller:InsightsAnalysisController"
 [project.entry-points."nemo.sdk"]
 insights = "nemo_insights_plugin.sdk:insights_sdk_resources"
 
+[project.entry-points."nemo.skills"]
+insights = "nemo_insights_plugin.skills:skills_dir"
+
 [build-system]
 requires = ["hatchling"]
 build-backend = "hatchling.build"

plugins/nemo-insights/src/nemo_insights_plugin/skills.py

@@ -0,0 +1,17 @@
+# SPDX-FileCopyrightText: Copyright (c) 2025-2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
+# SPDX-License-Identifier: Apache-2.0
+
+"""Skills directory exposure — registered under ``nemo.skills``.
+
+Returns the path to the ``skills/`` directory inside this package so the
+platform can discover and load skill markdown files shipped with the plugin.
+"""
+
+from __future__ import annotations
+
+from pathlib import Path
+
+
+def skills_dir() -> Path:
+    """Return the directory containing this plugin's skills."""
+    return Path(__file__).parent / "skills"

plugins/nemo-insights/src/nemo_insights_plugin/skills/nemo-analyst/SKILL.md

@@ -0,0 +1,163 @@
+---
+name: nemo-analyst
+description: >-
+  Analyze an agent's production traces to find recurring failure patterns and
+  record each as an Insight. Surveys spans, evaluator scores, and user feedback
+  across many sessions, clusters similar failures, then files every finding as a
+  titled Insight carrying the trace IDs that evidence the problem. Answers why
+  an agent keeps failing, where it gets things wrong, and the recurring
+  problems hiding in production traces. Produces the Insight that the
+  Experimentalist later acts on.
+triggers:
+  - nemo-analyst
+  - analyze my agent's traces
+  - why my agent keeps failing
+  - generate insights for my agent
+  - find recurring failure patterns
+  - run the analyst
+  - my agent keeps getting things wrong
+not-for:
+  - nemo-experimentalist (use to act on an Insight and change the agent; this skill produces the Insight it consumes)
+  - nemo-intake (use to instrument an agent, ingest telemetry, or query raw spans; this skill interprets telemetry that already landed)
+  - nemo-experiments-upload (use to upload traces and evaluation results into Intake; this skill reads them back out)
+  - nemo-explore (use to design an agent that does not exist yet; this skill needs a running agent with traces)
+  - nemo-evaluator (use to author evaluations and metrics; this skill analyzes production behavior)
+compatibility: >-
+  nemo-platform >= 0.1.0; requires the Insights plugin, a reachable platform
+  with Intake telemetry for the target agent, and a model the platform can call
+  on the Analyst's behalf. No Docker or datasets needed.
+maturity: beta
+license: Apache-2.0
+user-invocable: true
+allowed-tools: [Bash, Read]
+---
+
+# NeMo Analyst
+
+Analyze an agent's behavior from its own telemetry and record what recurs as
+Insights.
+
+## What it produces
+
+An Insight is a persistent, named description of one recurring problem, and it
+is the unit of work the rest of the optimization loop runs on. Each carries:
+
+- `title` — a sentence naming the failure, such as "Retrieval drops relevant
+  context near the token limit"
+- `description` — the failure mode, the tool or model call it affects, and the
+  conditions that trigger it
+- `trace_refs` — the Intake trace IDs cited as evidence, so a developer can
+  audit the reasoning and build regression tests
+
+The Analyst targets at least three representative traces per Insight and appends
+evidence to an existing Insight rather than filing a near-duplicate. It judges
+behavior rather than status or scores, so it finds failures in sessions that
+reported success and passed their evaluations. Two well-evidenced Insights are
+worth more than ten vague ones, so a run that files nothing is a valid outcome.
+
+## Before running
+
+The Analyst reads telemetry; it cannot create it. Confirm all three:
+
+- The target agent already has traces in Intake. No traces means no Insights.
+- The platform is reachable at `NMP_BASE_URL`.
+- The Analyst has a model to run on. It is an LLM agent itself, and how that is
+  configured is changing, so let pre-flight tell you whether it is satisfied —
+  it names what is missing and how to set it. Don't reach for the
+  Experimentalist's configuration; that is a different contract.
+
+An `AGENT-SPEC.md` is optional but makes the Analyst materially better. It
+carries the intent behind the agent — what it is for, its constraints, what
+counts as success — none of which is recoverable from code or traces, so
+without it the Analyst can only judge an agent against itself.
+
+## Pre-flight
+
+```bash
+nemo agents analyst doctor
+```
+
+Only two results block a run: no usable model configured, and an
+`optimizer.yaml` that is missing or unparseable — the second only if you intend
+to run without `--agent`. Doctor takes no `--agent` flag, so it always checks
+for a profile and always reports a red line when there is none; when you pass
+`--agent`, that line is noise. Platform reachability and the workspace probe
+only ever warn.
+
+## Run it
+
+```bash
+nemo agents analyst run --agent <agent-name> --workspace <workspace>
+```
+
+Add `--agent-spec AGENT-SPEC.md` to tell it what the agent is supposed to do,
+and `--verbose` to stream its tool calls and reasoning to stderr. Expect several
+minutes; it surveys many sessions before drilling into any of them.
+
+From an agent directory, an `optimizer.yaml` profile supplies `agent`,
+`workspace`, and `agent_spec`, so the flags above become optional:
+
+```bash
+nemo agents analyst run
+```
+
+The profile is discovered by walking up from the current directory. Only those
+three fields are read from it; other keys belong to the Experimentalist and are
+ignored.
+
+## Where Insights are stored
+
+Insights always go to the platform. `--insights-file-output` additionally
+mirrors what the platform stored, platform IDs included, merging into that file
+on each run; a mirror that cannot be written warns rather than failing the run.
+
+```bash
+nemo agents analyst run --agent <agent-name> --insights-file-output .nemo-optimizer/insights.yaml
+```
+
+That path is what the Experimentalist reads by default, so it is the
+conventional choice when handing off locally.
+
+## Verify
+
+Do not report success on an exit code. The run prints a line per operation —
+`- created: <title> [<insight-id>] (<n> trace refs)`, `- updated: <insight-id>
+(<n> trace refs)`, or `- no insights created or updated`, which is a successful
+run too. Read back by id whatever it says it wrote, and check each carries a
+clear title, an actionable description, and non-empty `trace_refs`. Listing by
+`?agent=` also returns earlier runs, so it attests the store, not this run:
+
+```bash
+curl --fail-with-body \
+  "$NMP_BASE_URL/apis/insights/v2/workspaces/<workspace>/insights/<insight-id>"
+```
+
+On an authenticated platform pass the token through curl's config, not argv
+where any process on the host can read it:
+
+```bash
+printf 'header = "Authorization: Bearer %s"' "$(nemo auth token)" | curl -K - <url>
+```
+
+Stored Insights also appear in Studio's optimizer view for the workspace.
+
+## When it finds nothing
+
+Besides a real "nothing worth filing", three things produce an empty result.
+Scoping: the Analyst reads only what `--agent` and `--workspace` together
+select, and `agent_name` is carried on agent-level spans, not on their model and
+tool children. Volume: too few traces looks the same as a healthy agent. And
+telemetry that captures only the shape of a run, spans without the inputs and
+outputs, leaves nothing to judge however many spans there are.
+
+## Hand off
+
+Once an Insight exists, the Experimentalist acts on it:
+
+```bash
+nemo agents experimentalist run
+```
+
+For the full data model, the Analyst's tool set, periodic analysis via
+`nemo insights analysis enable`, and the rest of the loop, see
+[Insight-Driven Optimization](https://github.com/NVIDIA-NeMo/nemo-platform/blob/main/docs/agents/insight-driven-optimization.mdx).

plugins/nemo-insights/src/nemo_insights_plugin/skills/nemo-analyst/tests.json

@@ -0,0 +1,65 @@
+{
+  "skill": "nemo-analyst",
+  "tests": [
+    {
+      "type": "explicit",
+      "prompt": "Use nemo-analyst to find out what my agent keeps getting wrong.",
+      "expected_skill": "nemo-analyst"
+    },
+    {
+      "type": "explicit",
+      "prompt": "Run the nemo-analyst skill against my agent's telemetry.",
+      "expected_skill": "nemo-analyst"
+    },
+    {
+      "type": "explicit",
+      "prompt": "Invoke nemo-analyst on the support agent's traces and record the insights it finds.",
+      "expected_skill": "nemo-analyst"
+    },
+    {
+      "type": "implicit",
+      "prompt": "Analyze my agent's traces and tell me what it keeps getting wrong.",
+      "expected_skill": "nemo-analyst"
+    },
+    {
+      "type": "implicit",
+      "prompt": "Why is my agent failing? Look at the traces and file insights.",
+      "expected_skill": "nemo-analyst"
+    },
+    {
+      "type": "implicit",
+      "prompt": "Find recurring problems in my agent from production traces.",
+      "expected_skill": "nemo-analyst"
+    },
+    {
+      "type": "implicit",
+      "prompt": "I want insights about my agent's failures from its spans and user feedback.",
+      "expected_skill": "nemo-analyst"
+    },
+    {
+      "type": "implicit",
+      "prompt": "Run the analyst to find failure patterns in recent telemetry.",
+      "expected_skill": "nemo-analyst"
+    },
+    {
+      "type": "negative-control",
+      "prompt": "Optimize my deployed agent's model routing to cut cost and latency.",
+      "expected_skill_not": "nemo-analyst"
+    },
+    {
+      "type": "negative-control",
+      "prompt": "Send my agent's OTLP traces into Intake and check ClickHouse is ready.",
+      "expected_skill_not": "nemo-analyst"
+    },
+    {
+      "type": "negative-control",
+      "prompt": "Design a new customer-support agent and write its specification.",
+      "expected_skill_not": "nemo-analyst"
+    },
+    {
+      "type": "negative-control",
+      "prompt": "Set up a Postgres database with seed data for local development.",
+      "expected_skill_not": "nemo-analyst"
+    }
+  ]
+}