feat(studio): revive nemo-agent & wire coding agent UI to use it

agents/nemo-agent-local-spec/AGENT-SPEC.md

@@ -0,0 +1,113 @@
+---
+name: nemo-agent-local
+created_timestamp: 2026-07-28T20:09:49Z
+author: Danielle Ali and Codex
+---
+
+# Agent Spec: nemo-agent-local
+
+> This file is the durable contract for the local NeMo Platform assistant.
+> Keep it aligned with the implementation under `agents/nemo-agent-local/`.
+
+## Role
+
+Help NeMo Platform developers inspect and operate their current workspace through Studio using the NeMo Platform SDK.
+
+## Purpose
+
+This agent provides a conversational backend for NeMo Studio so developers can confirm that a packaged NVIDIA NeMo Agent Toolkit (NAT) agent can be built, deployed, and used to interact with NeMo Platform. It should make routine discovery and operational tasks faster without requiring users to translate their intent into CLI commands or raw API requests.
+
+The mission is grounded in the current deployment proof of concept and its implementation: answer simple read-only questions immediately, carry out explicit multi-step platform tasks through supported SDK operations, verify consequential results, and request missing context rather than guessing or entering an unbounded reasoning loop.
+
+## Scope
+
+- Audience: NeMo Platform developers and internal operators using local Studio or a development environment
+- Categories: platform resource discovery; workspace-scoped resource management; agent and job status checks; evaluation and data operations; deployment troubleshooting
+- In scope: list and inspect supported platform resources; create, update, or delete resources when explicitly requested; check jobs and deployments; perform multi-step SDK workflows and report verified results; ask for missing workspace, resource, or operation details
+- Out of scope: invoking the NeMo CLI or arbitrary subprocesses; bypassing NeMo Platform APIs; silently choosing an ambiguous workspace or destructive target; claiming success without a successful SDK response or verification
+
+## Tools
+
+| Tool or source | Purpose | Credentials/scopes | Side effects | Freshness / expected failures |
+|---|---|---|---|---|
+| NeMo Platform Python SDK (`nemo_api`) | Access supported platform resources and actions through dot-separated SDK resource paths | Uses the deployment's platform base URL and active workspace; upstream secrets remain managed by NeMo Platform | Read and write operations depend on the requested SDK action | Workspace-scoped calls fail when no workspace is supplied; unavailable plugin resources or invalid SDK paths must be reported without repeated retries |
+| Platform status helper (`check_status`) | Check evaluation, customization, audit, and Data Designer jobs | Same platform access as the SDK client | Read-only | A service may expose different status subresources; report when no supported status method exists |
+| Deterministic list fast paths | Return names for common read-only list requests without an LLM round trip | Same platform access as the SDK client | Read-only | Must use the active workspace and return a direct clarification or error if required context is absent; must not fall through to the full agent loop after a recognized request fails |
+| Packaged agent skills | Supply task-specific playbooks when spec-compliant skills are included in the image | No separate credentials | Depends on the selected playbook and SDK action | The agent must log which skills are loaded; an empty or malformed skills directory means no playbooks are available |
+
+## Model
+
+- Mode: cloud
+- Family: NVIDIA Nemotron 3 Super 120B A12B
+
+## Framework
+
+- Resolution: langgraph-nat
+- Notes: LangGraph Deep Agent wrapped as a custom NAT workflow
+
+## Harness
+
+- Description: A LangGraph Deep Agent with SDK tools, optional packaged skills, deterministic read-only fast paths, and a custom NAT wrapper compatible with Studio and evaluation request shapes
+- Agent loop: Deep Agents orchestrates model and tool turns; recognized list requests bypass the model; complex runs must have explicit iteration, retry, and wall-clock limits
+- Tool dispatch: The model calls typed LangChain tools that resolve NeMo Platform SDK resources and return serialized results or concise errors
+- Context management: Studio conversation messages are normalized into LangChain messages; the model receives a concise system prompt and packaged skills when available
+- State management: LangGraph state holds the conversation and scratch data; packaged skill files and temporary files use routed filesystem backends
+- Guardrails: API-only operation; no CLI or arbitrary subprocess route; ambiguous workspace or destructive target requires clarification
+- Observability: Structured application logs for fast-path selection, tool failures, model requests, retries, health probes, and final workflow errors; agent-specific telemetry exporters are disabled
+- Verification: Consequential multi-step requests should read back final state when the SDK supports it; unit tests and the NAT evaluation configuration validate core behavior
+- Runtime: Custom NAT workflow served in a Docker deployment and consumed by local NeMo Studio
+- Notes: A recognized fast-path failure must terminate with a useful clarification or error instead of falling through to the model-driven graph
+
+## Behavior
+
+- Be concise and action-oriented. Distinguish completed work from instructions or proposed work.
+- Use the active request workspace automatically when it is available.
+- When a required workspace, resource name, target, or other consequential parameter is missing or ambiguous, ask one focused clarification question and stop that run.
+- Do not interpret missing context as permission to choose a destructive target.
+- For recognized deterministic requests, return the SDK result or a direct actionable failure; never use the full agent as an error-recovery fallback.
+- Use only NeMo Platform SDK tools. Never invoke the CLI, shell, or subprocesses.
+- Attempt reasonable equivalent SDK operations when a method name differs, but bound retries and do not loop over equivalent failures.
+- Report upstream model, SDK, and service failures honestly. Never claim that a mutation or deployment succeeded without verification.
+- Avoid exposing API keys or secret values in prompts, logs, or responses.
+
+## Success Criteria
+
+- Common read-only list requests, including workspaces, models, providers, filesets, datasets, benchmarks, and metrics, complete through one deterministic SDK operation and normally return within a few seconds.
+- Missing workspace or target context produces a clear clarification question rather than an exception, guessed value, or model-driven recovery loop.
+- Explicit multi-step operations use the correct workspace, execute only requested side effects, verify the final state when possible, and return a concise summary.
+- Complex requests have bounded model calls, retries, and execution time. A failed upstream decode must surface promptly instead of keeping Studio busy through repeated ten-minute retries.
+- Studio receives incremental, nonduplicated streaming output and reaches a terminal success or error state.
+- The agent never routes through the CLI, leaks managed credentials, or silently reaches a different NeMo Platform environment.
+
+## Evaluation Setup
+
+Unit coverage lives in `agents/nemo-agent-local/tests/test_nemo_agent.py`. Run it with:
+
+```bash
+uv run --frozen pytest agents/nemo-agent-local/tests/test_nemo_agent.py -v
+```
+
+The NAT evaluation workflow is `agents/nemo-agent-local/src/nemo_agent/nemo-eval.yml`, with inputs in `nemo-eval-data.json`. It currently evaluates workspace creation, workspace listing, and model/provider listing with a tunable RAG judge weighted for coverage (0.5), correctness (0.3), and relevance (0.2). No formal aggregate pass threshold is defined.
+
+Manual Studio validation is documented in `agents/nemo-agent-local/tests/smoke_test.md`. Current coverage gaps include fileset listing, missing-workspace clarification, fast-path failure containment, destructive-action ambiguity, iteration limits, retry limits, cancellation, and end-to-end latency thresholds.
+
+## Change Scope
+
+- System prompt: yes
+- Tools: yes
+- Middleware: yes
+- Inference params: yes
+- Model swap (within mode): yes
+- Skills: yes
+- Fine-tuning: no
+- Notes: Preserve API-only operation, managed secrets, and disabled agent telemetry; require human approval before broadening destructive capabilities or changing deployment mode
+
+## Signals
+
+Prioritize fast-path hit rate, per-request model-call count, tool-error repetition, total latency, upstream decode timeouts, retry count, clarification quality, and verified task completion. Treat repeated identical SDK errors or dozens of model calls for a simple request as runaway behavior. Routine container health probes are operational noise and should not be interpreted as user traffic. Until a dedicated agent telemetry pipeline is added, use container and platform logs for diagnosis.
+
+## Open Questions
+
+- Should `default` always be assumed when Studio does not propagate a workspace, or should the agent ask whenever more than one workspace exists?
+- Which mutation categories require an explicit confirmation step even when the target is unambiguous?
+- What maximum model-call count and wall-clock limit should apply to the complex path?

agents/nemo-agent-local/.dockerignore

@@ -0,0 +1,21 @@
+# Managed by `nemo agents package` — safe to delete if you take ownership.
+.env
+.env.*
+*.pem
+*.key
+credentials.json
+.git/
+.gitignore
+__pycache__/
+*.pyc
+.pytest_cache/
+.mypy_cache/
+.ruff_cache/
+*.egg-info/
+dist/
+build/
+.venv/
+node_modules/
+tests/
+src/nemo_agent/nemo-eval.yml
+src/nemo_agent/nemo-eval-data.json

agents/nemo-agent-local/Dockerfile

@@ -0,0 +1,72 @@
+# Managed by `nemo agents package` — safe to delete if you take ownership.
+ARG BASE_IMAGE_URL=nvcr.io/nvidia/base/ubuntu
+ARG BASE_IMAGE_TAG=noble-20260217
+ARG PYTHON_VERSION=3.13
+ARG NAT_VERSION=1.8.0
+FROM ${BASE_IMAGE_URL}:${BASE_IMAGE_TAG}
+ARG PYTHON_VERSION
+ARG NAT_VERSION
+
+COPY --from=ghcr.io/astral-sh/uv:0.8.15 /uv /uvx /bin/
+
+ENV PYTHONDONTWRITEBYTECODE=1
+
+# Keep the uv-managed Python in a world-readable location and use copy
+# link-mode so the venv is self-contained (no cross-directory symlinks),
+# letting the non-root runtime user exec it without needing /root access.
+ENV UV_PYTHON_INSTALL_DIR=/opt/uv/python \
+    UV_LINK_MODE=copy
+
+RUN apt-get update && \
+    apt-get install -y --no-install-recommends g++ gcc ca-certificates curl && \
+    update-ca-certificates && \
+    rm -rf /var/lib/apt/lists/*
+
+ENV REQUESTS_CA_BUNDLE=/etc/ssl/certs/ca-certificates.crt
+ENV SSL_CERT_FILE=/etc/ssl/certs/ca-certificates.crt
+
+WORKDIR /workspace
+
+COPY ./ /workspace
+
+# Project mode.  ``pyproject.toml`` is the single source of truth: it must
+# declare ``nvidia-nat[...]`` (for the ``nat`` CLI), a concrete ``version``
+# (or a container-resolvable dynamic version), and every runtime dep.
+# ``uv sync`` is deliberately not used because it honors
+# ``[tool.uv.sources]`` path overrides pointing at sibling workspace
+# packages that typically do not exist inside the build context.
+RUN --mount=type=cache,id=uv_cache,target=/root/.cache/uv,sharing=locked \
+    uv venv --python ${PYTHON_VERSION} /workspace/.venv && \
+    . /workspace/.venv/bin/activate && \
+    uv pip install . && \
+    chmod -R a+rX /opt/uv /workspace/.venv
+
+LABEL org.opencontainers.image.title="builtin-nemo-agent" \
+      org.opencontainers.image.version="0.1.0" \
+      org.opencontainers.image.authors="NVIDIA" \
+      org.opencontainers.image.created="2026-07-22T20:51:42.179931+00:00" \
+      org.opencontainers.image.description="Built-in NeMo Platform agent — a LangChain Deep Agent that uses the NeMo SDK to perform platform tasks." \
+      org.opencontainers.image.revision="" \
+      org.opencontainers.image.source="" \
+      com.nemo.agent.id="790bd61b98d0" \
+      com.nemo.agent.framework="nemo_agent_toolkit" \
+      com.nemo.agent.nat-version="1.8.0" \
+      com.nemo.agent.contract-version="0.0.0"
+
+ENV NAT_CONFIG_FILE=/workspace/src/nemo_agent/nemo-agent.yml
+
+ENV PATH="/workspace/.venv/bin:$PATH"
+
+# Some modern base images (notably Ubuntu 24.04 "noble" and the NVIDIA base
+# images derived from it) ship with a default unprivileged user at
+# uid=1000/gid=1000.  Reclaim 1000 for ``agent`` *by id, not by name* so this
+# layer is portable across older base images (where 1000 is free; the guarded
+# delete is a no-op) and across future base images that might rename the
+# default user.
+RUN if getent passwd 1000 >/dev/null; then userdel -rf "$(getent passwd 1000 | cut -d: -f1)" 2>/dev/null || true; fi && \
+    if getent group  1000 >/dev/null; then groupdel -f "$(getent group  1000 | cut -d: -f1)" 2>/dev/null || true; fi && \
+    groupadd -g 1000 agent && useradd -u 1000 -g agent -m agent && \
+    chown -R agent:agent /workspace
+USER agent
+
+ENTRYPOINT ["sh", "-c", "exec nat serve --config_file=$NAT_CONFIG_FILE --host 0.0.0.0"]

agents/nemo-agent-local/pyproject.toml

@@ -0,0 +1,33 @@
+# SPDX-FileCopyrightText: Copyright (c) 2026, NVIDIA CORPORATION & AFFILIATES. All rights reserved.
+# SPDX-License-Identifier: Apache-2.0
+
+[build-system]
+requires = ["hatchling"]
+build-backend = "hatchling.build"
+
+[project]
+name = "builtin-nemo-agent"
+version = "0.1.0"
+description = "Built-in NeMo Platform agent — a LangChain Deep Agent that uses the NeMo SDK to perform platform tasks."
+requires-python = ">=3.11,<3.14"
+dependencies = [
+    "nvidia-nat-core>=1.8.0,<1.9",
+    "nvidia-nat-langchain>=1.8.0,<1.9",
+    "nemo-platform",
+    "deepagents",
+    "langchain-openai",
+    "langchain-core",
+    "httpx",
+]
+
+[project.entry-points."nat.components"]
+nemo_agents_nemo_agent = "nemo_agent.register"
+# Custom NAT workflow type that wraps the nemo-agent graph with permissive
+# input/output schemas. See src/nemo_agent/wrapper.py for the full rationale —
+# in short, NAT 1.6.0's stock `langgraph_wrapper` doesn't accept the
+# `{"input_message": ...}` shape that `nvidia-nat-eval` posts and crashes on
+# deep-agent state deltas that don't carry `messages`.
+nemo_agents_nemo_agent_wrapper = "nemo_agent.wrapper"
+
+[tool.hatch.build.targets.wheel]
+packages = ["src/nemo_agent"]

agents/nemo-agent-local/src/nemo_agent/nemo-agent.remote.yml

@@ -0,0 +1,22 @@
+# Remote NeMo Platform deployment configuration.
+#
+# Keep the model workspace-qualified: the agent is deployed in a personal
+# workspace while inference is served by the shared, platform-managed route.
+# The platform injects a pod-reachable Inference Gateway base URL at deploy
+# time; no upstream URL or credential belongs in this file.
+
+llms:
+  agent:
+    _type: openai
+    api_key: not-used
+    model_name: default/nvidia-nemotron-3-super-120b-a12b
+    do_auto_retry: false
+    num_retries: 1
+    retry_on_status_codes: [599]
+    retry_on_errors: ["__never_retry__"]
+    max_retries: 0
+    request_timeout: 120
+
+workflow:
+  _type: nemo_agent_wrapper
+  description: NeMo Platform assistant using LangChain Deep Agents

agents/nemo-agent-local/src/nemo_agent/nemo-agent.yml

@@ -0,0 +1,81 @@
+# NeMo Platform Agent — NAT workflow configuration
+#
+# A LangChain Deep Agent that uses the NeMo SDK to perform platform tasks.
+# Deliberately configured as the "expensive baseline" for the optimization
+# pipeline to compress.
+#
+# Deploy:
+#   uv pip install -e plugins/nemo-agents
+#   uv pip install -e plugins/nemo-agents/builtin_agents/nemo-agent
+#   nemo agents create --name nemo-agent \
+#     --agent-config plugins/nemo-agents/builtin_agents/nemo-agent/src/nemo_agent/nemo-agent.yml
+#   nemo agents deploy --agent nemo-agent
+#
+# Invoke:
+#   nemo agents invoke --agent nemo-agent --input "Create a workspace called demo"
+
+llms:
+  agent:
+    _type: openai
+    # All AUT model traffic routes through the platform Inference Gateway
+    # (IGW). Both `nemo agents deploy` (production) and the agentic-use
+    # benchmark runner (tests/agentic-use/nat_runner.py via
+    # _prepare_aut_config_for_runtime → inject_gateway_url) inject the IGW
+    # base_url at runtime. Do NOT set `base_url` here — `inject_gateway_url`
+    # uses setdefault, so an explicit value would silently bypass IGW.
+    # `api_key: not-used` is a sentinel; IGW retrieves the real upstream
+    # credential from the platform secrets service.
+    api_key: not-used
+    # Model entity name. IGW resolves this to the upstream provider that
+    # owns it (each entity is owned by exactly one provider, so no provider
+    # field is needed in this yml). Use entity (dash) form — IGW handles
+    # the mapping to the served slash form internally.
+    #
+    # The dash-form name encodes the upstream catalog. nvidia-build serves
+    # NIM models as `nvidia/<model>` (single namespace), so its entities
+    # look like `nvidia-<model>`. nvidia-inference-api proxies LiteLLM,
+    # which serves the same NIM catalog as `nvidia/nvidia/<model>` and the
+    # bedrock catalog as `aws/anthropic/<model>`, so its entities look like
+    # `nvidia-nvidia-<model>` / `aws-anthropic-<model>`. The double prefix
+    # is therefore a reliable signal that an entity routes through
+    # nvidia-inference-api.
+    #
+    # Examples (each empirically validated end-to-end):
+    #   model_name: nvidia-nvidia-nemotron-3-super-v3   # via nvidia-inference-api
+    #   model_name: aws-anthropic-claude-opus-4-5       # via nvidia-inference-api
+    #   model_name: nvidia-llama-3-3-nemotron-super-49b-v1-5  # via nvidia-build
+    #
+    # For the full list of currently-discovered entities, hit
+    # ${NMP_BASE_URL}/v1/models on the running platform, or run the
+    # optional "list available models" cell in the playground notebook.
+    model_name: nvidia-nemotron-3-super-120b-a12b
+    # One total attempt: a retryable upstream decode timeout must not restart
+    # another long-running generation and keep Studio busy indefinitely.
+    do_auto_retry: false
+    num_retries: 1
+    # NAT 1.8 currently patches LangChain methods even when do_auto_retry is
+    # false. Some layers replace empty lists with their defaults, so use
+    # impossible sentinels to prevent nested wrappers from matching 429.
+    retry_on_status_codes: [599]
+    retry_on_errors: ["__never_retry__"]
+    # Disable the OpenAI client's independent retry loop. Without this, one
+    # model call can issue eleven requests before NAT sees the final error.
+    max_retries: 0
+    # Bound the HTTP model call below the Inference Gateway's 600-second
+    # decode wall-clock timeout so the agent can return a terminal error.
+    request_timeout: 120
+
+    # model_name: vm-opus-nemotron-random
+    # AWS-hosted Claude Opus 4.5 rejects Switchyard's Anthropic translation
+    # fallback of 128k output tokens. Keep this below the 64k AWS route limit
+    # while leaving room for prompt/tool tokens on longer benchmark tasks.
+    # Only necessary for Switchyard virtual models.
+    # max_tokens: 49152
+
+workflow:
+  # Custom workflow type registered by `nemo-agents-nemo-agent`. Wraps the same
+  # deep-agent graph as `langgraph_wrapper` would, but with input/output schemas
+  # that survive both the eval client's `{"input_message": ...}` payload and the
+  # deep-agent's wider state shape. See src/nemo_agent/wrapper.py.
+  _type: nemo_agent_wrapper
+  description: NeMo Platform assistant using LangChain Deep Agents

agents/nemo-agent-local/src/nemo_agent/nemo-eval-data.json

@@ -0,0 +1,14 @@
+[
+  {
+    "input_message": "Show the details for the default workspace without making any changes.",
+    "expected_output": "details for the default workspace"
+  },
+  {
+    "input_message": "List all workspaces on the platform",
+    "expected_output": "list of workspaces"
+  },
+  {
+    "input_message": "List the available models and inference providers using the platform API.",
+    "expected_output": "list of available models and inference providers"
+  }
+]