feat(nemo-agents): validate fabric relay telemetry

plugins/nemo-agents/examples/nemo-agent-config/README.md

@@ -1,33 +1,67 @@
-# Fabric-Backed Agent Config
+# NeMo Agent Config
 
-This Platform-owned config invokes Codex or Hermes through NeMo Fabric. Run the
-commands below from the repository root.
+This directory contains example Platform-owned `agent.yaml` configs for
+Fabric-backed NeMo Agents. Run the commands below from the repository root.
 
 Fabric dependencies are currently optional so the default workspace does not
-force other Fabric consumers onto the `0.1.0a20260724` SDK API before they migrate.
-Install them explicitly before local Fabric smoke tests:
+force other Fabric consumers onto the current Fabric SDK/runtime API before they
+migrate.
 
 ```bash
 uv pip install -e "plugins/nemo-agents[fabric]"
 ```
 
+For Codex/Claude Relay telemetry, install the Relay CLI separately:
+
+```bash
+cargo install nemo-relay-cli --root .venv
+.venv/bin/nemo-relay --version
+```
+
 Top-level `skills`, `mcp`, and `tools` are Platform-owned shared fields that
 translate into `FabricConfig`. Prompt settings are harness-specific for now and
 should be configured under `harnesses.<name>.settings`.
 
-## Codex
+## Local invoke
 
-Authenticate Codex, leave `default_harness: codex` in `agent.yaml`, and run:
+`agent.yaml` is the telemetry-neutral example. Authenticate Codex, then run:
 
 ```bash
-npm install -g @openai/codex
 codex login
 
 nemo agents invoke \
   --agent-config plugins/nemo-agents/examples/nemo-agent-config/agent.yaml \
   --input "Reply with exactly: platform fabric works"
 ```
 
+## Relay Local Files
+
+`agent-relay.yaml` enables Relay telemetry without Intake. It writes local ATIF
+and ATOF artifacts under the deployment artifacts directory.
+
+```bash
+nemo agents create \
+  --name fabric-relay-local-test \
+  --agent-config plugins/nemo-agents/examples/nemo-agent-config/agent-relay.yaml
+
+nemo agents deploy \
+  --agent fabric-relay-local-test \
+  --name fabric-relay-local-test-deployment \
+  --mode subprocess
+
+nemo agents invoke \
+  --agent-deployment fabric-relay-local-test-deployment \
+  --input "Reply with exactly: relay local works"
+```
+
+Confirm Relay emitted both ATIF and ATOF files:
+
+```bash
+find ~/.local/share/nemo/agents/system/default/fabric-relay-local-test-deployment-fabric/artifacts \
+  \( -name "*atif*" -o -name "*atof*" \) \
+  -exec ls -lh {} \;
+```
+
 ## Hermes
 
 Hermes Agent has dependencies that conflict with the Platform environment, so
@@ -51,3 +85,45 @@ nemo agents invoke \
   --agent-config plugins/nemo-agents/examples/nemo-agent-config/agent.yaml \
   --input "Reply with exactly: platform hermes works"
 ```
+
+## Relay to Intake (Local)
+
+`agent-relay-intake.yaml` enables Relay ATIF export to a locally running
+Platform Intake API. Start ClickHouse for Intake, start Platform, then create
+and deploy the agent:
+
+```bash
+services/intake/scripts/spans/run_clickhouse.sh
+
+nemo agents create \
+  --name fabric-relay-intake \
+  --agent-config plugins/nemo-agents/examples/nemo-agent-config/agent-relay-intake.yaml
+
+nemo agents deploy \
+  --agent fabric-relay-intake \
+  --name fabric-relay-intake-deployment \
+  --mode subprocess
+
+nemo agents invoke \
+  --agent-deployment fabric-relay-intake-deployment \
+  --input "Reply with exactly: platform fabric works"
+```
+
+Confirm Intake received ATIF-derived spans:
+
+```bash
+curl -g -s "http://127.0.0.1:8080/apis/intake/v2/workspaces/default/spans?page_size=20" \
+  | jq '.data[] | {session_id, name, source, started_at}'
+```
+
+To inspect traces in Studio, run the web app with Intake enabled:
+
+```bash
+cd web
+VITEST=true \
+VITE_FF_INTAKE_ENABLED=true \
+VITE_PLATFORM_BASE_URL=http://localhost:8080 \
+pnpm --filter nemo-studio-ui start --host 127.0.0.1
+```
+
+Then open `http://localhost:5173/studio/workspaces/default/intake/traces`.

plugins/nemo-agents/examples/nemo-agent-config/agent-relay-intake.yaml

@@ -0,0 +1,61 @@
+config_format: nemo-agents-spec-v1
+name: test-agent
+description: Test agent config
+
+default_harness: codex
+
+harnesses:
+  hermes:
+    kind: hermes
+    model:
+      provider: nvidia
+      model: nvidia/nemotron-3-nano-30b-a3b
+      api_key_env: NVIDIA_API_KEY
+      temperature: 0.0
+    settings:
+      python_env: HERMES_ADAPTER_PYTHON
+      base_url: https://integrate.api.nvidia.com/v1
+      max_iterations: 1
+      max_tokens: 512
+      reasoning_config:
+        effort: none
+      enabled_toolsets: []
+      system_prompt: You are a concise test assistant.
+  codex:
+    kind: codex
+    settings:
+      sandbox: workspace-write
+      system_prompt: You are a concise test assistant.
+      config_overrides:
+        model_reasoning_effort: high
+
+models:
+  default:
+    provider: openai
+    model: openai/gpt-5.4
+
+skills:
+  paths: []
+
+mcp:
+  servers: {}
+
+tools:
+  blocked: []
+
+environment:
+  workspace: ./workspace
+  artifacts: ./artifacts
+
+telemetry:
+  enabled: true
+  provider: relay
+  output_dir: ./artifacts/relay
+  project: test-agent
+  atif:
+    enabled: true
+    filename_template: trajectory-{session_id}.atif.json
+    storage:
+      - type: http
+        endpoint: http://127.0.0.1:8080/apis/intake/v2/workspaces/default/ingest/atif
+        timeout_millis: 3000

plugins/nemo-agents/examples/nemo-agent-config/agent-relay.yaml

@@ -0,0 +1,61 @@
+config_format: nemo-agents-spec-v1
+name: test-agent
+description: Test agent config
+
+default_harness: codex
+
+harnesses:
+  hermes:
+    kind: hermes
+    model:
+      provider: nvidia
+      model: nvidia/nemotron-3-nano-30b-a3b
+      api_key_env: NVIDIA_API_KEY
+      temperature: 0.0
+    settings:
+      python_env: HERMES_ADAPTER_PYTHON
+      base_url: https://integrate.api.nvidia.com/v1
+      max_iterations: 1
+      max_tokens: 512
+      reasoning_config:
+        effort: none
+      enabled_toolsets: []
+      system_prompt: You are a concise test assistant.
+  codex:
+    kind: codex
+    settings:
+      sandbox: workspace-write
+      system_prompt: You are a concise test assistant.
+      config_overrides:
+        model_reasoning_effort: high
+
+models:
+  default:
+    provider: openai
+    model: openai/gpt-5.4
+
+skills:
+  paths: []
+
+mcp:
+  servers: {}
+
+tools:
+  blocked: []
+
+environment:
+  workspace: ./workspace
+  artifacts: ./artifacts
+
+telemetry:
+  enabled: true
+  provider: relay
+  output_dir: ./artifacts/relay
+  project: test-agent
+  atif:
+    enabled: true
+    filename_template: trajectory-{session_id}.atif.json
+  atof:
+    enabled: true
+    filename: events.atof.jsonl
+    mode: overwrite

plugins/nemo-agents/pyproject.toml

@@ -63,10 +63,9 @@ fabric = [
     # TODO(AIRCORE-932): Move Fabric into the default plugin dependencies once evaluator has migrated
     # to the 0.1.0a20260724+ config-first SDK API and Fabric packaging is stable across Platform environments.
     # TODO(AIRCORE-897): Move this to a stable Fabric version before release once available.
-    # TODO(AIRCORE-897): Add the `relay` extra once nemo-evaluator-sdk's nemo-relay pin allows >=0.5.
-    "nemo-fabric[runtime]>=0.1.0a20260724,<0.2.0",
-    "nemo-fabric-adapters-codex>=0.1.0a20260724,<0.2.0",
-    "nemo-fabric-adapters-hermes>=0.1.0a20260724,<0.2.0; python_version < '3.14'",
+    "nemo-fabric[runtime]>=0.1.0rc2,<0.2.0",
+    "nemo-fabric-adapters-codex>=0.1.0rc2,<0.2.0",
+    "nemo-fabric-adapters-hermes>=0.1.0rc2,<0.2.0; python_version < '3.14'",
 ]
 container = [
     "jinja2>=3.1",

plugins/nemo-agents/src/nemo_agents_plugin/fabric/translator.py

@@ -134,7 +134,7 @@ def _apply_telemetry(fabric_config: Any, config: AgentConfig, model: ModelConfig
 
 def _relay_observability_config(config: AgentConfig, model: ModelConfig) -> dict[str, Any]:
     telemetry = config.telemetry
-    observability: dict[str, Any] = {"version": 1}
+    observability: dict[str, Any] = {"version": 2}
 
     if telemetry.atif is not None:
         atif = dict(telemetry.atif)
@@ -145,9 +145,34 @@ def _relay_observability_config(config: AgentConfig, model: ModelConfig) -> dict
         observability["atif"] = atif
 
     if telemetry.atof is not None:
-        atof = dict(telemetry.atof)
-        if telemetry.output_dir is not None:
-            atof.setdefault("output_directory", telemetry.output_dir)
-        observability["atof"] = atof
+        observability["atof"] = _relay_atof_config(telemetry.atof, output_dir=telemetry.output_dir)
 
     return observability
+
+
+def _relay_atof_config(atof_config: dict[str, Any], output_dir: str | None) -> dict[str, Any]:
+    atof = dict(atof_config)
+    sinks = list(atof.pop("sinks", []) or [])
+
+    file_sink = {"type": "file"}
+    if output_dir is not None:
+        file_sink["output_directory"] = output_dir
+    for key in ("output_directory", "filename", "mode"):
+        if key in atof:
+            file_sink[key] = atof.pop(key)
+    if len(file_sink) > 1:
+        sinks.insert(0, file_sink)
+
+    for endpoint in atof.pop("endpoints", []) or []:
+        stream_sink = dict(endpoint)
+        endpoint_url = stream_sink.pop("endpoint", None)
+        if endpoint_url is not None and "url" not in stream_sink:
+            stream_sink["url"] = endpoint_url
+        stream_sink["type"] = "stream"
+        sinks.append(stream_sink)
+
+    translated = {"enabled": atof.pop("enabled", False)}
+    if sinks:
+        translated["sinks"] = sinks
+    translated.update(atof)
+    return translated

plugins/nemo-agents/tests/unit/test_fabric_translator.py

@@ -200,7 +200,7 @@ def test_relay_telemetry_uses_latest_fabric_shape(self) -> None:
         assert fabric_config.relay.project == "example-agent"
         assert fabric_config.relay.output_dir == "./artifacts/relay"
         assert fabric_config.relay.observability.model_dump(exclude_none=True) == {
-            "version": 1,
+            "version": 2,
             "atif": {
                 "enabled": True,
                 "filename_template": "trajectory-{session_id}.atif.json",
@@ -210,8 +210,38 @@ def test_relay_telemetry_uses_latest_fabric_shape(self) -> None:
             },
             "atof": {
                 "enabled": True,
-                "filename": "events.atof.jsonl",
-                "mode": "overwrite",
-                "output_directory": "./artifacts/relay",
+                "sinks": [
+                    {
+                        "type": "file",
+                        "output_directory": "./artifacts/relay",
+                        "filename": "events.atof.jsonl",
+                        "mode": "overwrite",
+                    }
+                ],
             },
         }
+
+    def test_relay_atof_endpoint_sinks_translate_to_stream_sinks(self) -> None:
+        payload = copy.deepcopy(_example_yaml_config())
+        payload["telemetry"]["enabled"] = True
+        payload["telemetry"]["atof"] = {
+            "enabled": True,
+            "endpoints": [
+                {
+                    "type": "file",
+                    "endpoint": "http://localhost:4318/v1/events",
+                    "timeout_millis": 3000,
+                }
+            ],
+        }
+        config = AgentConfig.model_validate(payload)
+
+        fabric_config = translate_agent_config(config)
+
+        assert fabric_config.relay.observability.model_dump(exclude_none=True)["atof"]["sinks"] == [
+            {
+                "type": "stream",
+                "url": "http://localhost:4318/v1/events",
+                "timeout_millis": 3000,
+            }
+        ]