diff --git a/docs/about-nemo-relay/release-notes/highlights.mdx b/docs/about-nemo-relay/release-notes/highlights.mdx deleted file mode 100644 index d92cefd4d..000000000 --- a/docs/about-nemo-relay/release-notes/highlights.mdx +++ /dev/null @@ -1,148 +0,0 @@ ---- -title: "Highlights" -description: "Explore the notable capabilities in NVIDIA NeMo Relay 0.6." -position: 1 ---- -{/* SPDX-FileCopyrightText: Copyright (c) 2026, NVIDIA CORPORATION & AFFILIATES. All rights reserved. -SPDX-License-Identifier: Apache-2.0 */} - -This page focuses on the changes that are most likely to affect how you run, -instrument, or extend NeMo Relay. - -## NVIDIA NeMo Relay 0.6 - -NVIDIA NeMo Relay 0.6 focuses on reliable evidence. It starts Relay before a -coding agent needs it, keeps sensitive data out of emitted events, preserves -the right context in each export, and gives embedding hosts direct control of -dynamic plugins. - -### Coding-Agent Gateway and Skills - -- Coding agents now connect through `nemo-relay mcp`, which starts or adopts - the authenticated gateway before it reads MCP protocol frames. Codex, Claude - Code, and Hermes clients can share the gateway, keep it alive while they are - connected, and coordinate a single recovery attempt before idle shutdown. -- Installed integrations now use agent-owned hooks, generated MCP entries, - generation fencing, transactional rollback, and user-scoped configuration. - Generated configuration carries approved environment variable names, not - their secret values. -- Relay now emits tool-parented `skill.load` marks when an agent uses a - first-class skill tool or requests a complete read of a `SKILL.md` file. The - behavior covers coding agents, direct tools, LangChain, LangGraph, and Deep - Agents. -- The public skill catalog now starts from the task at hand: install Relay, - reach a first useful result, instrument an application, configure a plugin, - migrate, or debug. Its guidance favors installers pinned to immutable release - tags, synthetic non-sensitive exporter checks, runtime-injected secrets, and - tightly scoped migration writes. - -### Event Security and Runtime Semantics - -- You can now sanitize mark, scope-start, and scope-end events globally, within - a scope, or from a plugin context. Rust, Python, and Node.js expose the same - lifecycle, along with experimental Go and C bindings, native plugins, and - `grpc-v1` workers. -- The built-in PII redaction plugin now cleans mark data, category profiles, - metadata, and generic scope inputs and outputs before subscribers or - exporters receive them. -- The PII redaction plugin now supports ordered, composable profiles. A single - component can apply multiple privacy policies to marks, LLM and tool - inputs/outputs, and scope metadata. Relay runs profiles by priority and uses - array order to break ties. -- The opt-in `trajectory_context` preset removes request and response content - from buffered and streaming events for OpenAI Chat Completions, OpenAI - Responses, and Anthropic Messages. It preserves conversation topology, - tool-call relationships, agent hierarchy, routing, usage, cost, and - optimization telemetry. You can preserve custom mark payloads or redact them - recursively. -- Relay now tracks LLM event-history freshness per agent. A new agent and the - first LLM start after a `compaction` mark retain the complete sanitized - history. Later starts keep the system instructions, latest user turn, and - following messages. The provider request itself does not change. -- A central provider-codec factory now resolves canonical provider names and - the built-in request, response, and streaming codecs. The coding-agent - gateway uses lossless request codecs for Anthropic Messages, OpenAI Chat - Completions, and OpenAI Responses generation routes, so middleware can edit - portable or provider-native request annotations without rebuilding untouched - wire data. - -### Observability and Optimization - -- Observability configuration version 2 gives each ATOF file or stream sink - its own settings. One component can send every event to several destinations, - including named HTTP sinks whose credentials come from environment variables. -- OpenTelemetry and OpenInference now project top-level event data into typed - OTLP attributes instead of raw JSON payload attributes. Optional - `attribute_mappings` keep backend-specific aliases available during - migration. -- Trace exporters can show selected marks as visible, zero-duration tool spans. - OpenInference also exposes the projected tool name, output, and metadata in - its standard fields. -- A single `session.start` mark now carries the IDs that correlate startup - sessions across ATOF, ATIF, OpenTelemetry, and OpenInference. -- Relay now accepts plugin-neutral LLM optimization evidence and combines it - into bounded summaries of baseline and effective models, token effects, - estimated cost, evidence quality, and pricing provenance. -- ATIF now attributes each LLM step to the effective normalized response model. - ATOF remains the canonical mark stream, while ATIF remains a step-oriented - trajectory projection. - -### Dynamic Plugin Hosts - -- Embedding hosts can now own a `PluginHostActivation` for explicit native and - worker plugins. Static configuration initializes first, dynamic components - are added in the same transaction, and teardown clears callbacks and flushes - subscribers before releasing resources that it can unload safely. -- Python and Node.js can now activate explicitly specified dynamic plugins and - clean them up asynchronously. Experimental C and Go entry points expose the - same owned lifecycle for source-first evaluation. - -### CLI Configuration and Operations - -- `nemo-relay plugins edit` now understands recursive lists, string maps, and - tagged unions. You can edit nested Adaptive, NeMo Guardrails, PII, - Observability, and other built-in configuration without editing raw JSON. -- Operational commands now initialize process-wide logging before they run. - Logs always go to stderr and can also flow to bounded asynchronous file sinks - in human-readable or JSONL form. `nemo-relay config` and - `nemo-relay plugins edit` skip initialization so invalid logging - configuration cannot block its own repair. - -### Switchyard Decision Routing (Experimental) - -- The published `nemo-relay-switchyard` crate and optional CLI feature connect - Relay to a separately running Switchyard Decision API. Relay validates the - selected backend, translates supported provider protocols in process, and - dispatches to targets that Relay owns. -- You can run in enforce or observe-only mode, use bounded retry-aware routing, - use trusted same-protocol fallbacks, and route buffered or streaming calls. - Relay records the resulting model-routing optimization evidence. -- Startup health checks validate the Switchyard service, and named ATOF - stream-sink checks protect history-backed profiles. When every configured - target uses the inbound protocol, Relay preserves supported provider - extensions. Mixed-protocol configurations still enforce portability checks. - -### Compatibility and Migration - -- Reinstall persistent coding-agent integrations so they use the MCP-owned - gateway lifecycle. -- Move ATOF plugin configuration to Observability configuration version 2, and - use typed sinks for manually constructed exporters. Move OTLP queries to the - typed attribute paths at the same time. -- Check ATIF and event consumers for assumptions about marks, requested model - names, or complete request histories. -- Update Rust exhaustive matches and direct `AnnotatedLlmRequest`, message, - content, and tool literals for the new public variants and fields. Rebuild - Rust native plugins and `grpc-v1` workers that consume request annotations - against 0.6, and narrow their manifest requirement to - `compat.relay = ">=0.6,<1.0"`. -- Migrate Rust code that directly constructs `LlmJsonStream`, and update Go - call sites to handle the error returned by `LlmStream.Close`. Call an - explicit close method when a consumer stops reading an LLM stream early. -- Point any automation that opens public skills by directory name at the new - task-oriented catalog. - -For the exact migration actions, documentation, and originating PRs, refer to -[Known Issues and Compatibility Notes](/about-nemo-relay/release-notes/known-issues#compatibility-and-migration-notes). -The complete PR-by-PR history and release artifacts are available on -[GitHub Releases](https://github.com/NVIDIA/NeMo-Relay/releases). diff --git a/docs/about-nemo-relay/release-notes/index.mdx b/docs/about-nemo-relay/release-notes/index.mdx index 28bac2c95..84ff8e406 100644 --- a/docs/about-nemo-relay/release-notes/index.mdx +++ b/docs/about-nemo-relay/release-notes/index.mdx @@ -1,116 +1,143 @@ --- -title: "Release Notes" -description: "Read NeMo Relay release notes and find the official release history." +title: "Release Notes for NVIDIA NeMo Relay" +sidebar-title: "Release Notes" +description: "Review highlights, compatibility updates, fixed known issues, and current known issues for NVIDIA NeMo Relay 0.6." +template-library-version: "1.0.0" position: 6 --- {/* SPDX-FileCopyrightText: Copyright (c) 2026, NVIDIA CORPORATION & AFFILIATES. All rights reserved. SPDX-License-Identifier: Apache-2.0 */} -This page gives you the short version of the current release. For the complete -PR-by-PR history, release artifacts, and notes for a specific tag, use -[GitHub Releases](https://github.com/NVIDIA/NeMo-Relay/releases). - -## Current Release - -NVIDIA NeMo Relay 0.6 makes local coding-agent sessions easier to start, -observe, and extend. It adds an MCP-managed shared gateway, broader event -sanitization, richer observability and optimization evidence, direct control -of dynamic plugin lifecycles for embedding hosts, configurable operational -logging, and an experimental Switchyard Decision API integration. - -If you're upgrading from 0.5, plan for these changes: - -- Reinstall your Codex, Claude Code, and Hermes integrations with - `nemo-relay install --force` or `nemo-relay install all --force`. - Installations generated by 0.5 do not use the 0.6 gateway lifecycle. -- Move Observability plugin ATOF configuration to - `components.config.version = 2` and the tagged `atof.sinks` list. If you - query OTLP data, replace removed raw `*_json` attributes with the new typed - paths. -- Review event consumers that assume sanitizer failures are fail-open, every - LLM event contains the full request history, ATIF contains marks, or - `step.model_name` always matches the requested model. -- Update Rust code that exhaustively matches public enums or directly - constructs `AnnotatedLlmRequest`, message/content/tool variants, mark types, - ATOF sink types, or editor metadata types. Rebuild Rust native plugins and - `grpc-v1` workers that consume request annotations against 0.6, and declare - `compat.relay = ">=0.6,<1.0"` for those artifacts. The provided builders and - constructors are the safest migration path. -- Update managed LLM stream consumers that construct `LlmJsonStream` directly - or ignore Go `LlmStream.Close` results. Use the stream constructors and - explicit close methods described in the compatibility notes when exiting a - stream early. -- Replace direct references to old public skill directories with the - task-oriented `nemo-relay-install`, `nemo-relay-get-started`, - `nemo-relay-instrument-*`, and `nemo-relay-plugin-*` entry points. - -Before you upgrade, work through the complete -[Compatibility and Migration Notes](/about-nemo-relay/release-notes/known-issues#compatibility-and-migration-notes) -for the exact actions, documentation, and implementation PRs. Go and the raw C -FFI remain experimental and source-first. Node.js 24 or newer remains the -minimum supported Node.js version. - -## Release Scope - -Here's what changed across the main product surfaces: - -- **Coding agents:** Relay starts or adopts one authenticated gateway before an - installed client emits hooks or routed provider traffic. Codex, Claude Code, - and Hermes can share that gateway, coordinate a single recovery attempt, and - let it shut down after it becomes idle. -- **Events and security:** Mark, scope-start, and scope-end sanitizers now work - at global, scope-local, and plugin levels. The PII plugin sanitizes these - surfaces and supports ordered, composable profiles plus an opt-in - `trajectory_context` preset. Later LLM events can omit stale history, and - Relay records skill-load attempts with `skill.load` marks. -- **Observability and optimization:** ATOF can send every event to multiple - file and stream sinks. OTLP exporters use typed attributes, trace exporters - can show marks as tool spans, session identifiers correlate across formats, - ATIF reports the effective response model, and plugins can contribute - optimization evidence. -- **Provider request codecs:** The coding-agent gateway now enables lossless - request annotations for Anthropic Messages, OpenAI Chat Completions, and - OpenAI Responses generation routes. Interceptors edit `annotated_request`, - while no-op round trips retain provider-native fields, explicit nulls, and - wire representations. -- **Plugins and configuration:** Rust, Python, and Node.js embedding hosts can - own native and worker dynamic plugin lifecycles. Experimental C and Go entry - points expose the same lifecycle, and the CLI can edit nested lists, maps, and - tagged unions in built-in plugin configuration. -- **Operational logging:** Commands can write configurable human-readable or - JSONL logs to stderr and asynchronous file sinks without writing to stdout. -- **Decision routing:** The opt-in experimental Switchyard plugin validates and - routes buffered or streaming requests across OpenAI Chat Completions, OpenAI - Responses, and Anthropic Messages. -- **Managed LLM streams:** Explicit early close now deterministically stops the - producer, finalizes the partial response, and emits the interrupted end - event across Rust, Python, Node.js, Go, and C. Rust and Go callers have - source-compatibility changes; see the migration notes before upgrading. -- **Agent skills:** The public catalog now follows task-oriented workflows for - installation, getting started, instrumentation, plugin configuration, - migration, and debugging. The guidance calls for installers pinned to - immutable release tags, runtime-injected secrets, non-sensitive exporter - checks, and tightly scoped migration writes. - -## Feature Documentation - -Use these guides to explore the main 0.6 capabilities: +{/* @tpl:naming +Use title casing: Start with "Release Notes for" followed by the product name. Fern resolves the page title from frontmatter; do not add an H1 heading. +*/} +{/* @tpl:formatting +This is the Release Notes template. Document one release version per page and use the documentation version selector or previous-release links for earlier versions. Include patch releases on the same page in reverse chronological order. +*/} + +{/* @tpl:formatting +- Release notes summarize what changed in a version. Link to guides and reference pages instead of duplicating full procedures. +- Use tables for support matrices, compatibility lists, and fixed or known issue summaries when they improve scanability. +- Use callouts for upgrade blockers, breaking changes, and security advisories. +*/} + +This page contains the release notes for [NVIDIA NeMo +Relay](/about-nemo-relay/overview). + +## Release 0.6 + +NVIDIA NeMo Relay 0.6 strengthens local coding-agent observability, event +sanitization, observability exports, and dynamic plugin lifecycle management. + +### Highlights + +- Coding agents now connect through `nemo-relay mcp`, which starts or adopts a + shared authenticated gateway before hooks or routed provider traffic arrive. + Codex, Claude Code, and Hermes clients can share the gateway and coordinate + recovery before idle shutdown. +- Relay adds global, scope-local, and plugin-installed sanitizers for mark, + scope-start, and scope-end events. The PII redaction plugin now supports + ordered, composable profiles and the opt-in `trajectory_context` preset. +- ATOF configuration version 2 supports multiple independently configured file + and stream sinks. OpenTelemetry and OpenInference now use typed attributes, + and trace exporters can project selected marks as tool spans. +- Rust, Python, and Node.js embedding hosts can own native and worker dynamic + plugin lifecycles. Experimental Go and C entry points expose the same + source-first lifecycle. +- The coding-agent gateway supports lossless request annotations for Anthropic + Messages, OpenAI Chat Completions, and OpenAI Responses generation routes. +- The CLI adds recursive plugin configuration editing and configurable + human-readable or JSONL operational logging. +- The opt-in Switchyard integration can validate and route buffered or + streaming provider requests through a separately managed Decision API. + +### Support Matrix and Compatibility Updates + +The [Support Matrix](/reference/support-matrix) is the canonical reference for +supported platforms and architectures, worker runtimes, coding agents, and +integrations. It also records current limitations, including platform-specific +worker requirements. + +This release requires migration work for persistent coding-agent installations, +ATOF configuration, typed observability attributes, annotated-request plugin +consumers, and managed LLM streams. Refer to the [Migration +Guides](/reference/migration-guides) for upgrade actions from 0.5 to 0.6. + +### Fixed Known Issues in 0.6 + +- Coding-agent gateway generation routes now decode request annotations for + Anthropic Messages, OpenAI Chat Completions, and OpenAI Responses while + preserving unchanged nested fields, explicit nulls, and provider + representations. +- Installed coding-agent sessions now acquire the shared gateway through MCP + before hooks or routed provider traffic, preventing cold-start loss and + coordinating concurrent startup and recovery. +- Transparent Codex routing now recognizes `at-...` access tokens and sends + them to the ChatGPT Codex backend without rewriting unrelated bearer tokens + or provider API keys. +- Relay now asks Codex to prefer the readable legacy multi-agent path during + managed runs and restores the user's prior setting during uninstall. +- ATIF now reports the normalized provider response model when a routed or + translated call runs on a model that differs from the request. +- The NeMo Flow migration skill now skips credential-bearing dotenv files and + symbolic links, requires exact project-root confirmation, and refuses write + mode for filesystem roots and home directories. + +## Known Issues in 0.6 + +- Go and the raw C FFI remain experimental and source-first. Generated API + pages focus on Rust, Python, and Node.js. +- Local coding-agent observability depends on host hooks and provider traffic + reaching the local gateway. Relay cannot fully capture remote or cloud + execution that bypasses the local host. +- Persistent Codex, Claude Code, and Hermes integrations use user-scoped + configuration and a shared loopback gateway. Use `nemo-relay run` when a + launch must retain project-specific configuration. +- On Windows, a restrictive host Job Object can limit gateway reuse or prevent + persistent bootstrap. Codex can also make a cold-start `/models` request + before required MCP servers start; Relay retries the request. +- Codex 0.143 does not expose `SessionEnd`, and Codex multi-agent v2 encrypts + delegated-task payloads that Relay cannot decrypt or reliably link. +- The Node.js binding and package workflows require Node.js 24 or later. +- Switchyard is experimental, excluded from default CLI builds, and requires a + separately managed compatible service and a named authenticated ATOF HTTP + stream sink. +- OpenClaw has public hook-backed telemetry. Its security and optimization + coverage is partial because it does not own a managed execution path. +- The NeMo Guardrails remote backend inherits its configured service's + availability, latency, and policy behavior. The local backend requires Python + 3.11 or later and `nemoguardrails==0.22.0`. +- The PII redaction plugin currently supports its deterministic local backend; + local-model backend configuration is reserved for future work. +- Pricing and optimization estimates depend on model names, token data, pricing + sources, and freshness evidence. Missing or inconsistent evidence produces + partial or absent cost fields rather than zero values. +- ATOF stream sinks and remote ATIF storage require reachable, correctly + configured destinations. A failed stream sink does not stop file output or + other active sinks. +- ATIF omits point-in-time marks. Use ATOF for the canonical mark stream; + OpenTelemetry and OpenInference tool-style mark projection is only a + visualization aid. +- Native dynamic plugins run in the Relay process and are not sandboxed. A + `grpc-v1` worker runs in a separate process, but that process is not a + security sandbox. +- Treat Python `LLMRequest` objects as immutable. Request middleware that + changes content must return a new request object. +- Native subscriber callbacks arrive asynchronously. Flush subscribers before + depending on their side effects, captured events, files, or exporter output. +- Operational logging configuration and sink lifecycle are available, but broad + operational log coverage across commands is not yet available. + +## Previous Releases + +For previous release notes, release artifacts, and the complete PR-by-PR +history, refer to [GitHub Releases](https://github.com/NVIDIA/NeMo-Relay/releases). + +## Related Topics + +- [Support Matrix](/reference/support-matrix) +- [Migration Guides](/reference/migration-guides) - [Installation](/getting-started/installation) -- [NeMo Relay CLI](/nemo-relay-cli/about) -- [CLI Basic Usage](/nemo-relay-cli/basic-usage) -- [Coding Agent Installation and Diagnostics](/nemo-relay-cli/plugin-installation) -- [Build Plugins](/build-plugins/about) -- [Plugin Configuration Files](/configure-plugins/plugin-configuration-files) -- [Events](/about-nemo-relay/concepts/events) -- [Event Sanitizers](/reference/event-sanitizers) -- [LLM Request Intercept Outcomes](/reference/llm-request-intercept-outcomes) -- [Tool Execution Intercept Outcomes](/reference/tool-execution-intercept-outcomes) -- [Instrument an LLM Call](/instrument-applications/instrument-llm-call) -- [Provider Response Codecs and Model Pricing](/integrate-into-frameworks/provider-response-codecs) -- [Codecs](/about-nemo-relay/concepts/codecs) -- [Observability Configuration](/configure-plugins/observability/configuration) -- [ATOF Export](/configure-plugins/observability/atof) -- [OpenTelemetry Export](/configure-plugins/observability/opentelemetry) -- [OpenInference Export](/configure-plugins/observability/openinference) -- [Switchyard (Experimental)](/configure-plugins/switchyard/about) +- [NVIDIA NeMo Relay CLI](/nemo-relay-cli/about) +- [Known Issues and Troubleshooting](/resources/support-and-faqs) diff --git a/docs/about-nemo-relay/release-notes/known-issues.mdx b/docs/about-nemo-relay/release-notes/known-issues.mdx deleted file mode 100644 index ad638d561..000000000 --- a/docs/about-nemo-relay/release-notes/known-issues.mdx +++ /dev/null @@ -1,422 +0,0 @@ ---- -title: "Known Issues" -description: "Review current limitations, migration steps, and fixed issues for NVIDIA NeMo Relay." -position: 2 ---- -{/* SPDX-FileCopyrightText: Copyright (c) 2026, NVIDIA CORPORATION & AFFILIATES. All rights reserved. -SPDX-License-Identifier: Apache-2.0 */} - -Use this page to check whether a current limitation affects your deployment and -to plan an upgrade from an earlier release. It also keeps the complete list of -fixes recorded in prior release-note pages. - -## NVIDIA NeMo Relay 0.6 - -The following limitations and migration steps apply to NVIDIA NeMo Relay 0.6. - -### Current Limitations and Support Notes - -- Go and the raw C FFI remain experimental and source-first. Generated API - pages focus on Rust, Python, and Node.js, and the new C and Go dynamic-plugin - activation entry points are not ready for production use. -- The NeMo Relay CLI remains experimental. Local coding-agent observability - depends on host hooks and provider traffic reaching the local gateway. Relay - cannot fully capture remote or cloud execution that bypasses the machine. -- Persistent Codex, Claude Code, and Hermes integrations share a loopback - gateway and user-scoped Relay configuration. Use `nemo-relay run` when one - launch must keep its project configuration isolated. -- On Windows, a restrictive host Job Object can prevent a coding-agent plugin's - Relay gateway from breaking away. When the host permits nested assignment, - the gateway remains scoped to the host job, so its 300-second idle reuse - window can end early. If the host rejects nested assignment, persistent - bootstrap stops and reports the conflict. -- Codex can request `/models` during provider discovery before it launches the - required plugin MCP servers. Codex retries this cold-start request if it - fails. The captured turn and `/responses` traffic still wait for verified - Relay gateway readiness. -- The Codex 0.143 plugin hook schema does not expose `SessionEnd`. Relay updates - the cumulative ATIF snapshot after each delivered `Stop`, so the final `Stop` - captures the full session. -- Codex multi-agent v2 encrypts inter-agent message payloads. Relay can observe - parent and child activity but cannot decrypt delegated task text or reliably - link their traces. Managed sessions request the readable legacy path, but - higher-precedence configuration or the selected model can still choose v2. - Check the captured payload before relying on readable delegation text. -- Node.js binding and package workflows require Node.js 24 or newer. -- The Switchyard integration is experimental, is not part of default CLI - builds, and needs a separately managed compatible service. ATOF-backed - profiles also need one named, authenticated HTTP stream sink. A local file - sink does not populate the Switchyard accumulator. -- OpenClaw uses public hook-backed telemetry, so its security and optimization - coverage is partial. Security stops at pre-tool conditional guardrails, and - optimization stops at adaptive telemetry unless the integration owns a - managed execution path. -- The NeMo Guardrails remote backend inherits the availability, latency, and - policy behavior of its configured service. The local backend requires Python - 3.11 or newer and `nemoguardrails==0.22.0`. -- The PII redaction plugin currently supports its deterministic local backend. - Local-model backend configuration is reserved for future work. -- Pricing and optimization estimates are only as complete as the model name, - token data, pricing source, and freshness evidence that Relay receives. When - evidence is missing or inconsistent, cost fields remain partial or absent - rather than defaulting to zero. -- ATOF stream sinks and remote ATIF storage need reachable, correctly - configured destinations. If one ATOF stream sink fails, file output and the - other active sinks continue. -- ATIF models steps and leaves out point-in-time marks. Use ATOF when you need - the canonical mark stream. Tool-style mark projection in OpenTelemetry and - OpenInference is an opt-in visualization aid, not a replacement event - format. -- Native dynamic plugins run in the Relay process and are not sandboxed. A - `grpc-v1` worker runs in a separate process, but that process is not a - security sandbox. -- Treat Python `LLMRequest` objects as immutable. Request middleware that - changes content should return a new request object. -- Native subscriber callbacks arrive asynchronously. Flush subscribers before - you depend on callback side effects, captured events, files, or exporter - output. Deregistration stops future emissions, but callbacks that were - already queued can still run. -- Version 0.6 adds operational logging configuration and sink lifecycle, but - doesn't yet provide broad operational log coverage across commands. - -### Compatibility and Migration Notes - -#### Migrate Composable PII Redaction Configuration - - -If your Rust code constructs `PiiRedactionConfig` with a struct literal, -initialize the new `profiles` field or use `..Default::default()`. Serialization -now omits the default-valued legacy fields (`mode`, `input`, `output`, `mark`, -`tool_input`, `tool_output`, and `priority`). Update any code that depends on -the previous exact serialized shape. - -Existing TOML and JSON files still work. You can continue to use the legacy -single-policy form, but don't combine its fields with `profiles`. Relay runs -profiles in ascending priority and uses array order to break ties. Profiles can -apply to marks, LLM input/output, tool input/output, and scope metadata. -Existing PII redaction configurations now also sanitize LLM and tool scope -metadata, so downstream telemetry can change even when its schema does not. - -For profile examples and migration guidance, refer to the -[PII Redaction Configuration](/configure-plugins/pii-redaction/configuration) -guide. PR -[#512](https://github.com/NVIDIA/NeMo-Relay/pull/512) introduced the composable -configuration API. - - -#### Configure Structure-Preserving Trajectory Redaction - - -If your Rust code constructs `BuiltinBackendConfig` with a struct literal, -initialize the new `preset` and `custom_mark_payload_policy` fields or use -`..Default::default()`. Serialization now omits the default `action = "remove"` -value. Update any code that depends on the previous exact serialized shape. - -The `trajectory_context` preset is opt-in. If you leave `preset` unset, existing -runtime behavior stays the same. With the preset enabled, Relay removes request -and response content from buffered and streaming events for OpenAI Chat -Completions, OpenAI Responses, and Anthropic Messages. It preserves conversation -topology, tool-call relationships, agent hierarchy, routing, usage, cost, and -optimization fields. Relay preserves opaque custom marks by default; set -`custom_mark_payload_policy = "redact_all_leaves"` to redact their scalar -values recursively. If retained metadata or custom marks can contain email -addresses, apply an email-redaction profile later in the profile list. - -The policy affects observability data only. It doesn't change the provider -request or the client-visible provider response. For the full configuration -options, refer to the -[PII Redaction Configuration](/configure-plugins/pii-redaction/configuration) -guide. PR -[#513](https://github.com/NVIDIA/NeMo-Relay/pull/513) added trajectory-context -redaction on top of [PR #512](https://github.com/NVIDIA/NeMo-Relay/pull/512). - - -#### Update Plugins and Workers That Consume Request Annotations - - -`AnnotatedLlmRequest` now includes shared instructions, a tagged -`api_specific` surface, and additional message, content, tool, and tool-choice -variants. This is a Rust source-compatibility break for exhaustive matches and -direct struct literals. - -Rebuild Rust native plugins and Rust `grpc-v1` workers that accept annotated -LLM requests against NeMo Relay 0.6. Update Python workers to the 0.6 worker -SDK. Set their manifest requirement to -`compat.relay = ">=0.6,<1.0"` instead of the broader `>=0.5,<1.0` guidance. A -0.5 consumer cannot decode the added fields and enum variants. Relay 0.6 labels -request annotations as `nemo.relay.AnnotatedLlmRequest@2` and request-intercept -results as `nemo.relay.LlmRequestInterceptOutcome@2`. The host rejects a -dynamic plugin that registers an LLM request intercept while its -`compat.relay` range still admits Relay 0.5. - -Python workers continue to receive annotations as `dict[str, Any]`, but the -worker SDK validates the versioned envelope. Node.js, Go, and raw C FFI -request-intercept callbacks continue to carry the annotation as JSON, so they -do not require new binding DTOs. Consumers in those languages should still -avoid closed-world matching on role or component discriminator strings. - -The [Codecs](/about-nemo-relay/concepts/codecs) guide documents the expanded -annotation schema and mutation contract. PR -[#509](https://github.com/NVIDIA/NeMo-Relay/pull/509) introduced the request -codec integration and Rust source break. - - -#### Update Managed LLM Stream Consumers - - -Managed LLM streams now have one explicit close contract across bindings. When -you stop consuming a stream before natural exhaustion, close it so Relay can -stop the producer, finalize the partial response, and emit the interrupted end -event. Natural exhaustion is unchanged. - -Rust `LlmJsonStream` is no longer the former boxed-stream type alias. Replace -direct construction and alias-dependent type usage with `LlmJsonStream::new` or -`LlmJsonStream::from_closeable`; call `stream.close().await` on early exit. Go -`LlmStream.Close` now returns an `error`, so update call sites to handle its -result. Python keeps `await stream.aclose()`, Node.js provides -`await stream.close()`, and C callers can call `nemo_relay_stream_close` before -`nemo_relay_stream_free`. - -The [Rust Library Reference](/reference/api/rust-library-reference) documents -the current Rust API. Go and raw C FFI remain experimental and source-first. -PR [#465](https://github.com/NVIDIA/NeMo-Relay/pull/465) introduced the unified -close behavior. - - -#### Reinstall Coding-Agent Integrations - - -The installed integration format changed. Existing Codex, Claude Code, and -Hermes integrations do not automatically pick up the MCP-owned gateway, -agent-owned hooks, generation fencing, or user-scoped configuration. - -Run `nemo-relay install --force` for each installed integration, or run -`nemo-relay install all --force`. Otherwise, integration state generated by -0.5 can fail the 0.6 readiness and doctor checks. - -Follow the installation and doctor workflow in -[Plugin Installation](/nemo-relay-cli/plugin-installation). PR -[#395](https://github.com/NVIDIA/NeMo-Relay/pull/395) introduced the new -lifecycle. - - -#### Migrate ATOF Configuration and Manual Exporters - - -ATOF configuration that uses the 0.5 schema no longer validates. Set -`components.config.version = 2`, then replace the legacy `output_directory`, -`filename`, `mode`, and `endpoints` fields with a tagged `atof.sinks` list. -Relay rejects the old fields instead of translating them. - -If you construct an ATOF exporter directly, pass one typed file or stream sink. -A stream-backed exporter no longer exposes a file path. - -The [ATOF Export](/configure-plugins/observability/atof) guide shows the version -2 configuration and current binding examples. PR -[#416](https://github.com/NVIDIA/NeMo-Relay/pull/416) introduced the multi-sink -API and configuration. - - -#### Update OTLP Attribute Queries - - -OpenTelemetry and OpenInference no longer emit raw `*_json` event-payload -attributes. Move trace queries, dashboards, and processors to the typed -top-level paths. If a backend still needs an older key, configure an -`attribute_mappings` alias. Nested values remain JSON strings. - -Use the [OpenTelemetry Export](/configure-plugins/observability/opentelemetry) -and [OpenInference Export](/configure-plugins/observability/openinference) -guides to find the current attribute paths. PR -[#414](https://github.com/NVIDIA/NeMo-Relay/pull/414) changed the attribute -schema. - - -#### Move Mark Consumers to ATOF or Trace Projection - - -ATIF no longer turns Relay marks into synthetic `system` steps or populates -`AtifStepExtra.event_payload` for them. Use ATOF when you need the canonical -mark stream. If a trace viewer needs marks as visible tree nodes, set -`mark_projection = "tool"` for OpenTelemetry or OpenInference and review the -default `llm.chunk` exclusion. - -The [ATIF Export](/configure-plugins/observability/atif) and -[ATOF Export](/configure-plugins/observability/atof) guides explain the format -boundary and canonical mark stream. Use the -[OpenTelemetry](/configure-plugins/observability/opentelemetry) or -[OpenInference](/configure-plugins/observability/openinference) guide to -configure trace projection. PR -[#382](https://github.com/NVIDIA/NeMo-Relay/pull/382) changed the ATIF behavior -and added trace projection. - - -#### Recheck ATIF Model Attribution - - -ATIF now uses the effective model from the normalized provider response for -`step.model_name`. That value can differ from the requested or profile model -for routed and cross-protocol calls. - -Audit any consumer that groups, compares, or prices trajectories by -`step.model_name`. When the response does not identify a model, Relay still -uses the previous fallback. - -The [ATIF Export](/configure-plugins/observability/atif) guide explains the -trajectory semantics. PR -[#399](https://github.com/NVIDIA/NeMo-Relay/pull/399) changed the model -attribution. - - -#### Treat LLM Event Histories as Bounded Observations - - -Do not treat every LLM start event as a complete replay log. After the first LLM -start records an agent's complete history, later event inputs and annotations -can omit earlier turns. A new agent and the first start after a `compaction` -mark keep the full sanitized history; later starts keep the system instructions, -latest user turn, and messages that follow it. - -This projection changes observability events only. Relay sends the original -request to the provider. - -The [Events](/about-nemo-relay/concepts/events) guide explains the freshness and -scope-ownership rules. PR -[#398](https://github.com/NVIDIA/NeMo-Relay/pull/398) introduced the -bounded-history behavior. - - -#### Handle Sanitizer Failures Deliberately - - -A runtime error from a sanitizer now drops the affected event. Subscribers and -exporters can therefore have a gap instead of receiving unsanitized data. - -If your policy requires fail-open behavior, catch the error inside the -sanitizer and return the fields that are safe to emit. - -The [Event Sanitizers](/reference/event-sanitizers) guide explains registration, -lifetime, and field replacement. PR -[#371](https://github.com/NVIDIA/NeMo-Relay/pull/371) made sanitizer failures -fail-closed. - - -#### Update Rust Editor Metadata Consumers - - -If your Rust code consumes editor metadata, handle `EditorFieldKind::List` and -`EditorFieldKind::TaggedUnion` in exhaustive matches. Direct `EditorFieldSpec` -literals must also initialize the required `list_item` and `tagged_union` -fields. - -Schema consumers should now expect typed collections such as `learners`, -`config_ids`, and `target_paths` to report `EditorFieldKind::List` instead of -`EditorFieldKind::Json`. Serialized plugin configuration remains compatible. - -The [Rust Library Reference](/reference/api/rust-library-reference) lists the -current metadata types, and -[Plugin Configuration Files](/configure-plugins/plugin-configuration-files#gateway-editing-files) -explains the interactive editor workflow. PR -[#410](https://github.com/NVIDIA/NeMo-Relay/pull/410) changed the Rust metadata -API. - - -#### Update Rust Runtime Matches and Struct Construction - - -Add `FlowError::Upstream` to downstream exhaustive matches. Replace direct mark -and ATOF struct literals with `EmitMarkEventParams::builder()` and the current -typed sink constructors when possible. If you keep direct literals, initialize -the new mark `data_schema`, `category`, and `category_profile` fields and the -ATOF stream `header_env` field. The version 2 ATOF migration above supersedes -legacy `AtofEndpointSectionConfig` construction. - -The [Rust Library Reference](/reference/api/rust-library-reference) lists the -current public types, and -[Adding Scopes and Marks](/instrument-applications/adding-scopes-and-marks) -shows builder-based mark emission. PR -[#369](https://github.com/NVIDIA/NeMo-Relay/pull/369) introduced the Rust -source-compatibility change. - - -#### Update Public Skill Paths - - -Update any automation or agent instructions that open 0.5 skill directories by -name. Use `nemo-relay-install` and `nemo-relay-get-started` for setup; -`nemo-relay-instrument-context-isolation` and -`nemo-relay-instrument-typed-wrappers` for application instrumentation; -`nemo-relay-plugin-build` for plugin creation; and the consolidated -`nemo-relay-plugin-observability` and `nemo-relay-plugin-adaptive-tuning` -skills for exporter and adaptive workflows. - -The [NeMo Relay User Skills](https://github.com/NVIDIA/NeMo-Relay/blob/main/skills/README.md) -catalog lists the current paths and task routing. PR -[#407](https://github.com/NVIDIA/NeMo-Relay/pull/407) reorganized the catalog. - - -### Fixed in NVIDIA NeMo Relay 0.6 - -- Coding-agent gateway generation routes now decode request annotations for - Anthropic Messages, OpenAI Chat Completions, and OpenAI Responses. Request - interceptors can edit portable and provider-native fields while unchanged - nested fields, explicit nulls, and provider representations survive encoding. -- Installed coding-agent sessions now acquire the shared gateway through MCP - before hooks or routed provider traffic. This prevents cold-start loss and - coordinates concurrent startup and recovery. -- Transparent Codex routing now recognizes `at-...` access tokens and sends - them to the ChatGPT Codex backend without rewriting unrelated bearer tokens - or provider API keys. -- Relay now asks Codex to prefer the readable legacy multi-agent path during - managed runs, then restores the user's previous setting during uninstall. -- ATIF now reports the normalized provider response model when a routed or - translated call runs on a different model from the request. -- The NeMo Flow migration skill now skips credential-bearing dotenv files and - symbolic links, asks for exact project-root confirmation, and refuses write - mode for filesystem roots and home directories. - -### Fixed in Prior Releases - -#### NVIDIA NeMo Relay 0.5 - -- Gateway shutdown flushes queued events before it exits. -- Dynamic worker invocations support cancellation, and the CLI manages Python - dynamic-plugin environments. -- ATOF file export creates a missing output directory before opening JSONL - output. -- CLI status output reports daemon binding, exporter status, and remote ATIF - storage destinations more reliably. -- Codex host-plugin discovery uses the supported text output from the Codex - plugin command. - -#### NVIDIA NeMo Relay 0.4 - -- ATIF shutdown no longer deadlocks queued subscribers. -- Sanitized LLM requests are resolved from annotations for observability output. -- Structured ATIF tool results and Hermes tool-result observations are - preserved more reliably. -- Hermes routed-provider spans, wrapped ATIF fidelity, subagent lineage, and - error-path export consistency are covered and corrected. -- OpenClaw observability output is more consistent for nested subagents, model - timing diagnostics, hook-backed provenance, placeholder replay, and - hook-only fallback exports. -- LangChain serialization handles wrapped integration payloads more reliably. -- Plugin context-manager teardown avoids the previous deadlock path. -- Node.js `withScope` callbacks receive a real `ScopeHandle`. -- Deep Agents model responses are annotated for downstream observability. - -#### NVIDIA NeMo Relay 0.3 - -- Managed LLM start events are emitted before execution intercepts. -- Coding-agent trace scopes are aligned with NeMo Relay agent scope semantics. -- ATIF tool observations are correlated with their matching tool calls. -- OpenClaw tool call replay visibility is preserved. -- LangChain serialization handles wrapped integration payloads more reliably. - -#### Earlier Releases - -- Enabled TLS support for OTLP HTTP export. -- Preserved Go scope stacks across OS threads. - -For complete PR-by-PR history and release artifacts, refer to -[GitHub Releases](https://github.com/NVIDIA/NeMo-Relay/releases). diff --git a/docs/contribute/runtime-contract-docs.mdx b/docs/contribute/runtime-contract-docs.mdx index 748751adf..8c0040c43 100644 --- a/docs/contribute/runtime-contract-docs.mdx +++ b/docs/contribute/runtime-contract-docs.mdx @@ -118,7 +118,10 @@ Before landing a change that touches shared runtime wording, check: OpenInference remain projections. - Examples consume shared terminology instead of defining new support or contract language locally. -- Support matrices stay aligned across README, CLI docs, supported integrations, - and install or quick-start routing pages. +- The [Support Matrix](/reference/support-matrix) is the canonical source for + platform, worker-runtime, coding-agent, and integration support. Task-scoped + tables in the README, CLI docs, supported integrations, and install or + quick-start routing pages must remain aligned with it and link to it when + readers need the complete view. - Generated API reference pages are not used as the only explanation for runtime ownership, ordering, cleanup, or failure behavior. diff --git a/docs/getting-started/installation.mdx b/docs/getting-started/installation.mdx index 4f11aa039..1d7c5eac3 100644 --- a/docs/getting-started/installation.mdx +++ b/docs/getting-started/installation.mdx @@ -94,6 +94,9 @@ The installer reports the detected operating system and architecture when no matching asset exists. Intel macOS is not supported because no matching release asset is published. +For support across the core library, plugins, worker runtimes, and CLI assets, +refer to the [Support Matrix](/reference/support-matrix). + ### Installer Reference diff --git a/docs/nemo-relay-cli/about.mdx b/docs/nemo-relay-cli/about.mdx index 5481c5330..9ef04b2cc 100644 --- a/docs/nemo-relay-cli/about.mdx +++ b/docs/nemo-relay-cli/about.mdx @@ -59,6 +59,9 @@ controls. | Codex | Yes | Yes | Partial | Persistent install verifies all 10 hooks in the supported schema. Each `Stop` finalizes a turn snapshot because the plugin schema does not expose `SessionEnd`. | | Hermes Agent | Yes | Yes | Partial | User config installs the shared native MCP gateway lifecycle plus exact trusted hooks. Optimization depends on Hermes shell and API-request hook coverage. | +For minimum agent versions, platform support, and current limitations, refer to +the [Support Matrix](/reference/support-matrix). + ## Guides Use these guide links to move from CLI setup into agent-specific instructions. diff --git a/docs/reference/migration-guides.mdx b/docs/reference/migration-guides.mdx new file mode 100644 index 000000000..4431926e1 --- /dev/null +++ b/docs/reference/migration-guides.mdx @@ -0,0 +1,109 @@ +--- +title: "Migration Guides" +description: "Upgrade NeMo Relay integrations, plugins, exporters, and public API consumers." +position: 6 +--- +{/* SPDX-FileCopyrightText: Copyright (c) 2026, NVIDIA CORPORATION & AFFILIATES. All rights reserved. +SPDX-License-Identifier: Apache-2.0 */} + +Use this page to plan an upgrade from NeMo Relay 0.5 to 0.6. It groups the +actions from the release notes by the surface you operate. If you skip one or +more releases, review the migration guides and release notes for every +intervening release in sequence. + +## Upgrade to NeMo Relay 0.6 + +### Coding-Agent Integrations + +Reinstall every persistent Codex, Claude Code, and Hermes Agent integration: + +```bash +nemo-relay install --force +``` + +Or refresh all detected supported hosts: + +```bash +nemo-relay install all --force +``` + +Version 0.5 installations do not use the 0.6 MCP-owned gateway lifecycle, +agent-owned hooks, generation fencing, or user-scoped configuration. Confirm +that the host satisfies the current minimum version before reinstalling: +Claude Code 2.1.121, Codex CLI 0.143.0, or Hermes Agent 0.18.2. Then run +`nemo-relay doctor --plugin ` to verify the refreshed installation. + +### Dynamic Plugins and Workers + +Rebuild Rust native plugins and Rust `grpc-v1` workers that consume annotated +LLM requests against NeMo Relay 0.6. Update Python workers to the 0.6 worker +SDK and declare: + +```toml +[compat] +relay = ">=0.6,<1.0" +``` + +The annotation and LLM request-intercept outcome envelopes gained fields and +variants. A plugin that registers an LLM request intercept cannot claim a +compatibility range that admits Relay 0.5. Node.js, Go, and raw C FFI callbacks +continue to receive JSON, but their consumers must not exhaustively match role +or component discriminator strings. Go and raw C FFI remain experimental and +source-first. + +### Streaming Consumers + +Close a managed LLM stream when you stop consuming it early so Relay can stop +the producer and emit the interrupted end event. Replace direct Rust +`LlmJsonStream` construction with its constructors, handle the result from Go +`LlmStream.Close`, use Python `await stream.aclose()`, Node.js +`await stream.close()`, or call `nemo_relay_stream_close` before freeing a C +stream. + +### Exporters and Observability Queries + +Update ATOF configuration to version 2 and replace the legacy output fields +with the tagged `atof.sinks` list. Direct ATOF exporter construction now takes +one typed file or stream sink. + +Update OpenTelemetry and OpenInference queries, dashboards, and processors to +the typed attribute paths. The former raw `*_json` payload attributes are no +longer emitted. Use `attribute_mappings` only when an older key must continue +to be available. + +ATIF no longer models marks as synthetic system steps. Use ATOF for canonical +mark data or enable the OpenTelemetry or OpenInference `mark_projection = +"tool"` visualization when a trace viewer needs visible mark nodes. Review +any consumers that assume `step.model_name` always identifies the requested +model; it now uses the effective response model where available. + +### Middleware, Sanitizers, and Runtime APIs + +Update PII redaction configuration to the composable `profiles` form when you +need ordered policies. Existing single-policy configuration continues to work, +but it cannot be combined with `profiles`. The optional +`trajectory_context` preset changes observability payloads only; it does not +change provider requests or client-visible responses. + +Sanitizer failures now drop the affected event. If your policy requires +fail-open behavior, handle the failure inside the sanitizer and return only the +safe fields. + +For Rust consumers, update exhaustive matches and direct struct literals for +the expanded public enums and types. Prefer the current builders and typed sink +constructors over direct literals where available. + +### Skills and Automation + +Replace retired 0.5 skill directories with the task-oriented public entry +points, including `nemo-relay-install`, `nemo-relay-get-started`, the +`nemo-relay-instrument-*` skills, and `nemo-relay-plugin-*` skills. The +[NeMo Relay User Skills](https://github.com/NVIDIA/NeMo-Relay/blob/main/skills/README.md) +catalog lists the current paths. + +## Related Release Information + +For release highlights, compatibility updates, and current known issues, refer +to the [Release Notes](/about-nemo-relay/release-notes). Use [GitHub +Releases](https://github.com/NVIDIA/NeMo-Relay/releases) for the complete +release history and notes for a specific tag. diff --git a/docs/reference/support-matrix.mdx b/docs/reference/support-matrix.mdx new file mode 100644 index 000000000..d834a37fc --- /dev/null +++ b/docs/reference/support-matrix.mdx @@ -0,0 +1,87 @@ +--- +title: "Support Matrix" +description: "Supported NeMo Relay platforms, CLI agent hosts, and framework integrations." +position: 5 +--- +{/* SPDX-FileCopyrightText: Copyright (c) 2026, NVIDIA CORPORATION & AFFILIATES. All rights reserved. +SPDX-License-Identifier: Apache-2.0 */} + +This matrix describes the support scope for the current NVIDIA NeMo Relay +release. It distinguishes published, prebuilt release artifacts from components +that you build and package for the host yourself. CI coverage indicates where +the repository runs validation; it does not by itself create a separate support +commitment. For release-specific changes and known limitations, refer to +[Release Notes](/about-nemo-relay/release-notes). + +## Platforms and Architectures + +| Surface | Linux x86_64 | Linux ARM64 | macOS x86_64 | macOS ARM64 | Windows x86_64 | Windows ARM64 | Limitations | +| --- | --- | --- | --- | --- | --- | --- | --- | +| Core Library | Required CI coverage | Required CI coverage | Not in the current CI matrix | Required CI coverage | Non-blocking CI coverage | Non-blocking CI coverage | Rust is the source of truth. Python and Node.js packages require their supported runtimes; Go and raw C FFI remain experimental and source-first. | +| CLI Application | Published `x86_64-unknown-linux-musl` asset | Published `aarch64-unknown-linux-musl` asset | No published release asset | Published `aarch64-apple-darwin` asset | Published `x86_64-pc-windows-msvc` asset | Published `aarch64-pc-windows-msvc` asset | Intel macOS has no prebuilt CLI asset. Use a source install on unsupported platforms. | +| Native Plugins | Build for the host ABI | Build for the host ABI | Build and validate locally | Build for the host ABI | Build for the host ABI; CI is non-blocking | Build for the host ABI; CI is non-blocking | Native plugins run in-process and are not sandboxed. Rebuild for the host ABI and compatible Relay version. | +| gRPC Worker Plugins | Depends on the selected worker runtime | Depends on the selected worker runtime | Build and validate locally | Depends on the selected worker runtime | Depends on the selected worker runtime | Depends on the selected worker runtime | Workers run as local child processes over `grpc-v1`, not as a security sandbox. Python workers are unsupported on Windows ARM64. Refer to the worker runtime matrix for runtime-specific requirements. | + +### Linux Distribution Notes + +The published Linux CLI binaries use musl targets and are not tied to a +distribution-specific package manager. Linux CI uses Ubuntu runners. NeMo Relay +does not publish distribution-specific DEB or RPM packages or certify individual +Ubuntu, Debian, RHEL-compatible, or Fedora releases. On an unsupported CPU, +unsupported operating system, or a system whose local requirements prevent the +release asset from running, install from source with Cargo. + +## Worker Runtime Compatibility + +gRPC worker plugins run as local processes. Package and test each worker for +its Relay host platform. This matrix does not make a compatibility claim for +emulated or cross-architecture worker execution. + +| Worker runtime | Linux x86_64 | Linux ARM64 | macOS x86_64 | macOS ARM64 | Windows x86_64 | Windows ARM64 | Requirements and limitations | +| --- | --- | --- | --- | --- | --- | --- | --- | +| Python | Worker E2E CI coverage | Not separately worker E2E validated | Build and validate locally | Worker E2E CI coverage | Worker E2E CI coverage; CI is non-blocking | Unsupported: no usable `grpcio` wheel | Use `runtime = "python"`, a `module:function` entrypoint, and `source.manifest_root`. `nemo-relay plugins add` creates and retains the managed Python environment. Package dependencies for the target host. | +| Rust | Host-compatible executable; core CI coverage | Host-compatible executable; core CI coverage | Build and validate locally | Host-compatible executable; core CI coverage | Host-compatible executable; core CI is non-blocking | Host-compatible executable; core CI is non-blocking | Use `runtime = "rust"` and provide an executable that implements `grpc-v1`. | +| Command | Host-compatible executable; core CI coverage | Host-compatible executable; core CI coverage | Build and validate locally | Host-compatible executable; core CI coverage | Host-compatible executable; core CI is non-blocking | Host-compatible executable; core CI is non-blocking | Use `runtime = "command"` for another local executable that implements `grpc-v1`. Its runtime, dependencies, and compatibility are the operator's responsibility. | + +The process boundary isolates crashes and dependencies, but it is not a security +sandbox. Refer to [gRPC Worker Plugin +Concepts](/build-plugins/dynamic-plugins/grpc-worker/about) for manifests, +lifecycle, and trust requirements. + +## Coding Agents Supported by the CLI + +The CLI supports local coding-agent sessions where agent hooks and provider +traffic can reach the local Relay gateway. It rejects prerelease, malformed, +and older CLI versions during installation, diagnostics, and transparent runs. + +| Agent | Minimum version | Supported CLI capabilities | Current limitations | +| --- | --- | --- | --- | +| Claude Code | 2.1.121 | Persistent install, transparent run, lifecycle hooks, local gateway routing, and pre-tool security | Claude desktop, web, and application sessions are unsupported unless they expose the same local hook and gateway controls. Optimization requires gateway-routed LLM traffic and available hooks. | +| Codex CLI | 0.143.0 | Persistent install, transparent run, 10 supported plugin hooks, local gateway routing, and pre-tool security | Cloud or remote tasks that bypass the local machine have partial or no LLM capture. The plugin hook schema has no `SessionEnd`; Relay finalizes the cumulative session snapshot at `Stop`. Encrypted Codex multi-agent v2 payloads cannot be decrypted or reliably linked. | +| Hermes Agent | 0.18.2 | Persistent MCP integration, transparent run, lifecycle hooks, local gateway routing, and pre-tool security | Optimization depends on Hermes shell and API-request hook coverage. Hook-observed and gateway-observed LLM spans can both be present when both paths are enabled. | + +For installation, diagnostics, and host-specific behavior, refer to [Coding Agent +Installation](/nemo-relay-cli/plugin-installation), [Claude +Code](/nemo-relay-cli/claude-code), [Codex](/nemo-relay-cli/codex), and [Hermes +Agent](/nemo-relay-cli/hermes). + +## Framework and Agent Harness Integrations + +| Integration | Observability | Security | Optimization | Notes | +| --- | --- | --- | --- | --- | +| LangChain | Yes | Yes | Yes | Wrapped tool and LLM calling. | +| LangGraph | Yes | Yes | Yes | Wrapped tool and LLM calling. | +| Deep Agents | Yes | Yes | Yes | Wrapped tool and LLM calling. | +| OpenClaw | Yes | Partial | No | Public hook-backed telemetry with pre-tool guardrails. Public hooks do not expose managed execution rewrites. | + +Install the maintained Python integrations with the `langchain`, `langgraph`, +and `deepagents` extras. Install the OpenClaw integration as the +`nemo-relay-openclaw` plugin. Refer to [Supported +Integrations](/supported-integrations/about) for setup and guide links. + +## Related Information + +- [Installation](/getting-started/installation) lists published CLI assets and source-install options. +- [Build Plugins](/build-plugins/about) explains native and worker plugin boundaries. +- [Migration Guides](/reference/migration-guides) gives the current upgrade actions. +- [Release Notes](/about-nemo-relay/release-notes) records release-specific compatibility notes and known issues. diff --git a/docs/resources/glossary.mdx b/docs/resources/glossary.mdx index 3c5a17d44..9dfb2fed1 100644 --- a/docs/resources/glossary.mdx +++ b/docs/resources/glossary.mdx @@ -1,315 +1,552 @@ --- title: "Glossary" -description: "" +sidebar-title: "Glossary" +description: "Definitions for NVIDIA NeMo Relay runtime, plugin, integration, adaptive, and observability terms." +template-library-version: "1.0.0" position: 2 --- + {/* SPDX-FileCopyrightText: Copyright (c) 2026, NVIDIA CORPORATION & AFFILIATES. All rights reserved. SPDX-License-Identifier: Apache-2.0 */} +{/* @tpl:formatting +This is the Glossary template. Use it for a lookup-oriented glossary page that defines domain terms, acronyms, and product language readers will encounter elsewhere in the documentation; use the .md variant for Sphinx/MyST projects, the .rst variant for reStructuredText, or the -gfm.md variant for GitHub Flavored Markdown. +*/} + +{/* @tpl:example +To get started, refer to [NVIDIA NeMo Relay Glossary](https://docs.nvidia.com/nemo/relay/resources/glossary). +*/} + +{/* @tpl:formatting +- Use alphabetical H2 sections (`## A`, `## B`, …) so readers can jump by letter. +- Use definition lists under each letter: one term line, then `:` and the definition (dictionary-style). +- Use short first sentences for most definitions; add code, tables, or lists only when they add lookup value. +- Use related links when readers should continue to a deeper concept, guide, or reference page. +*/} NeMo Relay uses specialized runtime, integration, plugin, adaptive, and observability terms across bindings. This glossary defines the shared terms so the rest of the documentation can use them consistently. -**Activation Report** +--- -An **activation report** records which plugin components validated, initialized, or registered behavior successfully. Use it to distinguish configuration problems from runtime behavior problems. +{/* @tpl:formatting +- Add **alphabetical H2 headings** (`## A` through `## Z`, plus numerals/symbols if needed). +- Under each heading, use a **definition list**: write the **term** on its own line, then a colon and the definition text. +- Indent continuation lines (examples, **Related**, *See also*) so they stay part of the same entry. +- Add a `{ref}` target line above an entry when other pages must link to that definition. +- Add code blocks or tables only when they make the definition meaningfully clearer. +*/} -**Adaptive Cache Governor (ACG)** +## A -The **adaptive cache governor (ACG)** is the adaptive subsystem that analyzes LLM prompt structure, tracks stable prompt blocks, and plans provider-specific prompt-cache breakpoints. +**Activation Report** +: An activation report records which plugin components validated, initialized, + or registered behavior successfully. Use it to distinguish configuration + problems from runtime behavior problems. -**Adaptive Component** +**Adaptive Cache Governor (ACG)** +: The adaptive cache governor (ACG) analyzes LLM prompt structure, tracks + stable prompt blocks, and plans provider-specific prompt-cache breakpoints. -The **adaptive component** is the built-in plugin component with kind ``adaptive``. It can register telemetry subscribers, adaptive hint intercepts, tool-parallelism behavior, cache-governor behavior, and adaptive state backends. +**Adaptive Component** +: The adaptive component is the built-in plugin component with kind + `adaptive`. It can register telemetry subscribers, adaptive hint intercepts, + tool-parallelism behavior, cache-governor behavior, and adaptive state + backends. **Adaptive Hint** - -An **adaptive hint** is metadata injected into an outgoing model request by an adaptive request intercept. Downstream code or provider adapters can use the hint to adjust behavior when explicitly configured to do so. - -**Adaptive Tuning** - -**Adaptive tuning** is the NeMo Relay runtime capability that observes instrumented work and enables controlled behavior changes through the plugin system. +: An adaptive hint is metadata injected into an outgoing model request by an + adaptive request intercept. Downstream code or provider adapters can use the + hint to adjust behavior when explicitly configured to do so. **Adaptive State Backend** - -An **adaptive state backend** stores observations and learned state for adaptive behavior. In-memory state is process-local; Redis-backed state can be shared across workers or survive process restarts. +: An adaptive state backend stores observations and learned state for adaptive + behavior. In-memory state is process-local; Redis-backed state can be shared + across workers or survive process restarts. **Adaptive Telemetry** +: Adaptive telemetry is the subscriber path that observes lifecycle events for + adaptive learners without changing execution by itself. -**Adaptive telemetry** is the subscriber path that observes lifecycle events for adaptive learners without changing execution by itself. +**Adaptive Tuning** +: Adaptive tuning is the NeMo Relay runtime capability that observes + instrumented work and enables controlled behavior changes through the plugin + system. **Agent Trajectory Interchange Format (ATIF)** - -**Agent Trajectory Interchange Format (ATIF)** is an external trajectory format used for offline analysis, replay, or evaluation. The NeMo Relay ATIF exporter collects lifecycle events and exports ATIF v1.7 trajectory data. +: Agent Trajectory Interchange Format (ATIF) is an external trajectory format + used for offline analysis, replay, or evaluation. The NeMo Relay ATIF + exporter collects lifecycle events and exports ATIF v1.7 trajectory data. **Agent Trajectory Observability Format (ATOF)** - -**Agent Trajectory Observability Format (ATOF)** is the canonical event format NeMo Relay emits for scope lifecycle events and mark events. Subscribers and exporters consume ATOF events before translating compatible scope data into ATIF trajectories and trace data into OpenTelemetry or OpenInference. +: Agent Trajectory Observability Format (ATOF) is the canonical event format + NeMo Relay emits for scope lifecycle events and mark events. Subscribers and + exporters consume ATOF events before translating compatible scope data into + ATIF trajectories and trace data into OpenTelemetry or OpenInference. **Annotated Request And Response Data** +: Annotated request and response data is normalized provider information + created by LLM codecs. It lets request intercepts, subscribers, and exporters + reason about provider payloads through a shared model while preserving the raw + provider request or response shape. -**Annotated request and response data** is normalized provider information created by LLM codecs. It lets request intercepts, subscribers, and exporters reason about provider payloads through a shared model while preserving the raw provider request or response shape. +## B **Binding** - -A **binding** is a language-specific public API surface for the NeMo Relay runtime, such as Python, Node.js, Go, Rust, or C FFI. +: A binding is a language-specific public API surface for the NeMo Relay + runtime, such as Python, Node.js, Go, Rust, or C FFI. **Break Chain** +: `break_chain` is the request-intercept setting that stops later request + intercepts after the current intercept returns. Use it only when the current + request transform should be final. -``break_chain`` is the request-intercept setting that stops later request intercepts after the current intercept returns. Use it only when the current request transform should be final. +## C **Callback** - -A **callback** is the application, framework, tool, or provider function that does the real work. Managed execution passes this callback through NeMo Relay so middleware and lifecycle events surround the invocation. +: A callback is the application, framework, tool, or provider function that + does the real work. Managed execution passes this callback through NeMo Relay + so middleware and lifecycle events surround the invocation. **Category Profile** - -A **category profile** is the event field that stores category-specific semantic details. NeMo Relay uses it for values such as LLM ``model_name``, tool ``tool_call_id``, and custom ``subtype``. +: A category profile is the event field that stores category-specific semantic + details. NeMo Relay uses it for values such as LLM `model_name`, tool + `tool_call_id`, and custom `subtype`. **Codec** - -A **codec** is a deterministic translator at a NeMo Relay boundary. Codecs let framework or provider-native values remain convenient for application code while NeMo Relay observes JSON-compatible or normalized data. +: A codec is a deterministic translator at a NeMo Relay boundary. Codecs let + framework or provider-native values remain convenient for application code + while NeMo Relay observes JSON-compatible or normalized data. **Collection Window** - -A **collection window** is the period during which an in-process exporter, such as the ATIF exporter, buffers events before export or clear. Bounded collection windows prevent unrelated runs from mixing in one artifact. +: A collection window is the period during which an in-process exporter, such + as the ATIF exporter, buffers events before export or clear. Bounded + collection windows prevent unrelated runs from mixing in one artifact. **Collector** - -A **collector** is the callback used by streaming LLM helpers to observe each streamed chunk and accumulate state for the final response. +: A collector is the callback used by streaming LLM helpers to observe each + streamed chunk and accumulate state for the final response. **Conditional Execution** +: A conditional-execution guardrail decides whether the call is allowed to run + at all. -A **conditional-execution guardrail** decides whether the call is allowed to run at all. +## D -**Event** +**Dynamic Plugin** +: A dynamic plugin is a discoverable plugin package that the operator registers + from a `relay-plugin.toml` manifest. Dynamic plugins are either native shared + libraries or local gRPC workers. Refer to [Discoverable + Plugins](/build-plugins/dynamic-plugins/about). + +## E -An **event** is the runtime record of something that happened. NeMo Relay emits events for scope start and end, tool start and end, LLM start and end, and named mark points. Events are the shared data model consumed by subscribers and exporters. +**Event** +: An event is the runtime record of something that happened. NeMo Relay emits + events for scope start and end, tool start and end, LLM start and end, and + named mark points. Events are the shared data model consumed by subscribers + and exporters. **Event Envelope** +: The event envelope is the shared set of fields carried by every ATOF event, + including identifiers, timestamps, names, data, data schema, metadata, and + parent linkage. -The **event envelope** is the shared set of fields carried by every ATOF event, including identifiers, timestamps, names, data, data schema, metadata, and parent linkage. +**Event Sanitizer** +: An event sanitizer rewrites observability fields on mark, scope-start, or + scope-end events. It does not change the real callback arguments or result. + Refer to [Event Sanitizers](/reference/event-sanitizers). **Execution Intercept** +: An execution intercept wraps or replaces the real callback. Use it when + behavior belongs around the invocation boundary itself, such as retries, + timing, routing, wrapper logic, or framework integration. -An **execution intercept** wraps or replaces the real callback. Use this when behavior belongs around the invocation boundary itself, such as retries, timing, routing, wrapper logic, or framework integration. +**Experimental Binding** +: An experimental binding exposes runtime behavior for source-first users but + is not the primary documentation path. Go and the raw C FFI surface are + experimental unless a page says otherwise. **Explicit Lifecycle API** - -An **explicit lifecycle API** is a manual start, end, or mark helper used when a framework owns the real invocation internally. It preserves observability but does not let execution intercepts wrap the real callback automatically. +: An explicit lifecycle API is a manual start, end, or mark helper used when a + framework owns the real invocation internally. It preserves observability but + does not let execution intercepts wrap the real callback automatically. **Exporter** +: An exporter is a subscriber-oriented component that translates NeMo Relay + events into an external artifact or backend format, such as an ATIF trajectory + or OTLP trace spans. -An **exporter** is a subscriber-oriented component that translates NeMo Relay events into an external artifact or backend format, such as an ATIF trajectory or OTLP trace spans. - -**Experimental Binding** - -An **experimental binding** exposes runtime behavior for source-first users but -is not the primary documentation path. Go and the raw C FFI -surface are experimental unless a page says otherwise. +## F **FFI** - -**FFI** means foreign function interface. NeMo Relay's C FFI layer exposes core runtime behavior to non-Rust languages and is used by the Go binding. +: FFI means foreign function interface. NeMo Relay's C FFI layer exposes core + runtime behavior to non-Rust languages and is used by the Go binding. **Finalizer** +: A finalizer is the callback used by streaming LLM helpers when the stream ends + through natural exhaustion or explicit close. It turns collected stream state + into the response payload that sanitize-response guardrails, subscribers, and + exporters can observe. -A **finalizer** is the callback used by streaming LLM helpers when the stream ends through natural exhaustion or explicit close. It turns collected stream state into the response payload that sanitize-response guardrails, subscribers, and exporters can observe. +## G **Global And Scope-Local Registration** +: NeMo Relay supports two main ownership levels for middleware and subscribers. + + - **Global registrations** stay active for the whole process until removed. + - **Scope-local registrations** are owned by one active scope and are cleaned + up automatically when that scope closes. -NeMo Relay supports two main ownership levels for middleware and subscribers. - **Global registrations** stay active for the whole process until removed. - **Scope-local registrations** are owned by one active scope and are cleaned up automatically when that scope closes. This split lets process-wide defaults coexist with request-local policy or instrumentation. + This split lets process-wide defaults coexist with request-local policy or + instrumentation. + +**gRPC Worker Plugin** +: A gRPC worker plugin is a dynamic plugin that runs as a local child process + and installs proxy callbacks through the stable `grpc-v1` protocol. It can + use a Python, Rust, or command runtime. The process boundary isolates crashes + and dependencies, but it is not a security sandbox. Refer to [gRPC Worker + Plugin Concepts](/build-plugins/dynamic-plugins/grpc-worker/about). **Guardrail** +: A guardrail is middleware that either blocks execution or rewrites the data + recorded on emitted events. Sanitize guardrails are observability-oriented. + They do not rewrite the real arguments passed to the callback or the real + value returned to the caller. -A **guardrail** is middleware that either blocks execution or rewrites the data recorded on emitted events. Sanitize guardrails are observability-oriented. They do not rewrite the real arguments passed to the callback or the real value returned to the caller. +## I **Integration Boundary** - -An **integration boundary** is the stable point in an application, framework, or provider adapter where NeMo Relay can wrap, observe, or transform a tool or LLM invocation. +: An integration boundary is the stable point in an application, framework, or + provider adapter where NeMo Relay can wrap, observe, or transform a tool or + LLM invocation. **Intercept** +: An intercept is middleware that changes the real request path or wraps the + real callback. -An **intercept** is middleware that changes the real request path or wraps the real callback. +## J **JSON-Compatible Payload** +: A JSON-compatible payload is data that can be represented in NeMo Relay's + JSON model. Event data, middleware payloads, and codec output should be + JSON-compatible. -A **JSON-compatible payload** is data that can be represented in NeMo Relay's JSON model. Event data, middleware payloads, and codec output should be JSON-compatible. +## L **Learner** +: A learner is an adaptive component that consumes observed event data and + derives reusable guidance, such as tool parallelism plans or cache stability + signals. -A **learner** is an adaptive component that consumes observed event data and derives reusable guidance, such as tool parallelism plans or cache stability signals. +**Lifecycle Hook** +: A lifecycle hook is a coding-agent callback that reports a session or tool + milestone. Relay-generated hooks forward the canonical payload to the local + Relay gateway. **Lifecycle Pair** - -A **lifecycle pair** is the matching start and end event for one scope, tool call, or LLM call. Subscribers and exporters use lifecycle pairs to compute durations, reconstruct boundaries, and preserve nesting. +: A lifecycle pair is the matching start and end event for one scope, tool + call, or LLM call. Subscribers and exporters use lifecycle pairs to compute + durations, reconstruct boundaries, and preserve nesting. **LLM Call** - -An **LLM call** is an instrumented model-provider invocation. Managed LLM calls emit start and end events, run LLM middleware, and can carry a normalized ``model_name`` for observability and trajectory export. +: An LLM call is an instrumented model-provider invocation. Managed LLM calls + emit start and end events, run LLM middleware, and can carry a normalized + `model_name` for observability and trajectory export. **LLM Stream** +: An LLM stream is a streaming model response managed across multiple chunks + rather than a single response object. NeMo Relay captures the originating + scope stack, runs stream execution intercepts, collects chunks, and finalizes + the stream into a response-side event payload. Consumers that stop early + explicitly close the stream so the producer can clean up and finalize the + partial response. -An **LLM stream** is a streaming model response managed across multiple chunks rather than a single response object. NeMo Relay captures the originating scope stack, runs stream execution intercepts, collects chunks, and finalizes the stream into a response-side event payload. Consumers that stop early explicitly close the stream so the producer can clean up and the partial response can be finalized. +## M **Managed Execution And Manual Lifecycle** +: NeMo Relay supports two main ways to model tool and LLM work. -NeMo Relay supports two main ways to model tool and LLM work. - **Managed execution** means NeMo Relay owns the middleware pipeline and emitted lifecycle around the invocation. - **Manual lifecycle** means some other framework or runtime owns the real call boundary, and NeMo Relay only records the start and end points explicitly. Managed execution is the default choice for application code. Manual lifecycle exists mainly for framework integrations that cannot delegate the real invocation to NeMo Relay. + - **Managed execution** means NeMo Relay owns the middleware pipeline and + emitted lifecycle around the invocation. + - **Manual lifecycle** means another framework or runtime owns the real call + boundary, and NeMo Relay records the start and end points explicitly. -**Managed Execution Wrapper** + Managed execution is the default choice for application code. Manual + lifecycle exists mainly for framework integrations that cannot delegate the + real invocation to NeMo Relay. -A **managed execution wrapper** is the integration pattern where a tool or LLM provider callback is routed through NeMo Relay's managed execute helper. This is the preferred pattern when NeMo Relay can own middleware ordering, lifecycle pairing, and event emission around the real callback. +**Managed Execution Wrapper** +: A managed execution wrapper is the integration pattern where a tool or LLM + provider callback is routed through NeMo Relay's managed execute helper. This + is the preferred pattern when NeMo Relay can own middleware ordering, + lifecycle pairing, and event emission around the real callback. **Mark Event** - -A **mark event** is a point-in-time event for a named runtime checkpoint that is not a full start/end lifecycle pair. Use marks for retries, checkpoints, interrupts, state transitions, or framework milestones that do not represent a complete nested invocation. +: A mark event is a point-in-time event for a named runtime checkpoint that is + not a full start/end lifecycle pair. Use marks for retries, checkpoints, + interrupts, state transitions, or framework milestones that do not represent + a complete nested invocation. **Middleware** +: Middleware is the runtime behavior that runs around tool or LLM work. -**Middleware** is the runtime behavior that runs around tool or LLM work. Middleware can inspect, reject, transform, wrap, or sanitize execution at well-defined lifecycle points. NeMo Relay has two major middleware families: - **Intercepts** affect the real execution path. - **Guardrails** block work or rewrite the observability payload. + - **Intercepts** affect the real execution path. + - **Guardrails** block work or rewrite the observability payload. **Middleware Registry** - -A **middleware registry** stores named middleware entries for one runtime surface, such as tool request intercepts or LLM sanitize-response guardrails. The runtime combines global registry entries with visible scope-local entries before managed execution. +: A middleware registry stores named middleware entries for one runtime surface, + such as tool request intercepts or LLM sanitize-response guardrails. The + runtime combines global registry entries with visible scope-local entries + before managed execution. + +**Model Context Protocol (MCP)** +: Model Context Protocol (MCP) is the protocol that NeMo Relay uses for the + host-neutral lifecycle client in persistent coding-agent integrations. The + `nemo-relay mcp` server manages gateway lifetime and exposes no tools. + +**Model Pricing** +: Model pricing is the built-in `pricing` plugin component that loads catalog + sources and annotates managed LLM responses with cost estimates when model and + token data are available. The component resolves configured sources through a + pricing resolver chain. + +## N + +**Native Dynamic Plugin** +: A native dynamic plugin is a Rust shared library loaded into the Relay + process through the native plugin ABI. It must be rebuilt for the host ABI + and compatible Relay version, and it is not sandboxed. Refer to [Native + Dynamic Plugins](/build-plugins/dynamic-plugins/native-dynamic/about). + +**NeMo Guardrails** +: NeMo Guardrails is the built-in `nemo_guardrails` plugin component for + first-party guardrail policy around managed tool and LLM execution. It + supports remote Guardrails-service and Python-backed local backends. **Next Function** - -The **next function** is the continuation passed to an execution intercept. The intercept calls ``next`` to run the next intercept or the original callback. An intercept that does not call ``next`` intentionally replaces or short-circuits the rest of the invocation. +: The next function is the continuation passed to an execution intercept. The + intercept calls `next` to run the next intercept or the original callback. An + intercept that does not call `next` intentionally replaces or short-circuits + the rest of the invocation. **Non-Serializable Data** +: Non-serializable data is framework or SDK state that cannot be represented as + JSON, such as clients, streams, callbacks, file handles, or class instances. + Keep those objects outside NeMo Relay payloads and pass only stable + identifiers or projections through events and middleware. -**Non-serializable data** is framework or SDK state that cannot be represented as JSON, such as clients, streams, callbacks, file handles, or class instances. Keep those objects outside NeMo Relay payloads and pass only stable identifiers or projections through events and middleware. +## O **OpenInference** - -**OpenInference** is an AI-observability semantic convention layered on trace spans. NeMo Relay's OpenInference subscriber maps lifecycle payloads to OpenInference-oriented attributes such as model inputs, outputs, and token usage. +: OpenInference is an AI-observability semantic convention layered on trace + spans. NeMo Relay's OpenInference subscriber maps lifecycle payloads to + OpenInference-oriented attributes such as model inputs, outputs, and token + usage. **OpenTelemetry** - -**OpenTelemetry** is a vendor-neutral observability ecosystem. NeMo Relay can export lifecycle events as OpenTelemetry-compatible trace spans. +: OpenTelemetry is a vendor-neutral observability ecosystem. NeMo Relay can + export lifecycle events as OpenTelemetry-compatible trace spans. **OpenTelemetry Protocol (OTLP)** +: OpenTelemetry Protocol (OTLP) is the transport protocol used by the + OpenTelemetry and OpenInference subscribers to send trace data to a collector + or backend. -**OpenTelemetry Protocol (OTLP)** is the transport protocol used by the OpenTelemetry and OpenInference subscribers to send trace data to a collector or backend. +## P -**Plugin** +**Persistent Installation** +: A persistent installation is a Relay-owned coding-agent integration created + with `nemo-relay install `. It registers an MCP lifecycle client and + trusted hooks in user configuration so the agent can use a shared local + gateway without a wrapper command. The installation uses an immutable + generation fence to reject stale managed entries. -A **plugin** is a reusable runtime component that installs middleware, subscribers, or related behavior from configuration rather than through hand-written registration at every call site. Plugins let you package reusable policy bundles, observability components, and adaptive behavior. +**PII Redaction** +: PII redaction is the built-in `pii_redaction` plugin component that sanitizes + sensitive fields in emitted observability payloads. It does not change the + real callback arguments or returned result. -**Plugin Component** +**Plugin** +: A plugin is a reusable runtime component that installs middleware, + subscribers, or related behavior from configuration rather than through + hand-written registration at every call site. Plugins let you package reusable + policy bundles, observability components, and adaptive behavior. -A **plugin component** is one configured unit inside plugin configuration. Each component has a kind and component-specific settings, such as the built-in ``adaptive`` component. +**Plugin Component** +: A plugin component is one configured unit inside plugin configuration. Each + component has a kind and component-specific settings, such as the built-in + `adaptive` component. **Plugin Configuration** - -**Plugin configuration** is the versioned document or object that describes which plugin components should be validated, initialized, and activated. +: Plugin configuration is the versioned document or object that describes which + plugin components should be validated, initialized, and activated. **Plugin Context** +: A plugin context is the activation-time object that plugin code uses to + register middleware, subscribers, or other runtime behavior. -A **plugin context** is the activation-time object that plugin code uses to register middleware, subscribers, or other runtime behavior. - -**Priority** - -**Priority** is the ordering value attached to middleware registrations. NeMo Relay runs visible middleware in priority order after merging global and scope-local registrations. +**Plugin Manifest** +: A plugin manifest is the `relay-plugin.toml` package contract for a dynamic + plugin. It declares the plugin type, compatibility, capabilities, source, + integrity evidence, and loading details. **Primary Binding** +: A primary binding is one of the documented binding surfaces used for + first-line examples and generated API references. The primary bindings are + Rust, Python, and Node.js. -A **primary binding** is one of the documented binding surfaces used for -first-line examples and generated API references. The primary bindings are Rust, -Python, and Node.js. +**Priority** +: Priority is the ordering value attached to middleware registrations. NeMo + Relay runs visible middleware in priority order after merging global and + scope-local registrations. **Projection** - -A **projection** translates canonical NeMo Relay events into a downstream format -such as ATIF, OpenTelemetry, or OpenInference. A projection should preserve the -meaning of the runtime event stream without redefining ownership, middleware -ordering, or provider-specific codec policy. +: A projection translates canonical NeMo Relay events into a downstream format + such as ATIF, OpenTelemetry, or OpenInference. A projection should preserve + the meaning of the runtime event stream without redefining ownership, + middleware ordering, or provider-specific codec policy. **Prompt IR** - -**Prompt IR** is the internal representation ACG uses to model an LLM request as addressable prompt blocks for stability analysis and cache planning. +: Prompt IR is the internal representation ACG uses to model an LLM request as + addressable prompt blocks for stability analysis and cache planning. **Prompt-Cache Breakpoint** - -A **prompt-cache breakpoint** is a provider-specific location in a prompt where the cache governor suggests or applies cache behavior for stable prompt sections. +: A prompt-cache breakpoint is a provider-specific location in a prompt where + the cache governor suggests or applies cache behavior for stable prompt + sections. **Provider Adapter** - -A **provider adapter** is code that translates between a framework's model-call surface and a provider-specific API shape. Provider adapters often use request codecs when request intercepts or request-side middleware need normalized request semantics, and response codecs when events, subscribers, or exporters need normalized response annotations. +: A provider adapter is code that translates between a framework's model-call + surface and a provider-specific API shape. Provider adapters often use request + codecs when request intercepts or request-side middleware need normalized + request semantics, and response codecs when events, subscribers, or exporters + need normalized response annotations. **Provider Codec** +: A provider codec converts provider-specific LLM requests or responses into + normalized annotated data. Request codecs decode raw provider requests before + request intercepts run and encode edited annotations back into the provider + request before execution continues. Response codecs decode raw provider + responses for LLM end-event annotations, subscribers, exporters, and + diagnostics. -A **provider codec** converts provider-specific LLM requests or responses into normalized annotated data. Request codecs decode raw provider requests before request intercepts run and encode edited annotations back into the provider request before execution continues. Response codecs decode raw provider responses for LLM end-event annotations, subscribers, exporters, and diagnostics. +## R -**Request Intercept** +**Relay Gateway** +: The Relay gateway is the local NeMo Relay process that receives coding-agent + lifecycle payloads and can route model-provider traffic. Compatible MCP + clients share a user-level gateway for persistent installations; transparent + runs use a private gateway. -A **request intercept** rewrites the request before execution continues downstream. Use this when the real provider or tool implementation should receive modified input. +**Request Intercept** +: A request intercept rewrites the request before execution continues + downstream. Use this when the real provider or tool implementation should + receive modified input. **Response Codec** - -A **response codec** decodes a raw provider response into annotated response data for lifecycle events. It does not rewrite the value returned to the application unless the wrapper's typed value codec also does so. +: A response codec decodes a raw provider response into annotated response data + for lifecycle events. It does not rewrite the value returned to the + application unless the wrapper's typed value codec also does so. **Rollout Policy** - -A **rollout policy** is the configuration strategy for enabling adaptive behavior gradually, such as starting in observation mode, then injecting hints, then allowing scheduling or cache-planning behavior. +: A rollout policy is the configuration strategy for enabling adaptive behavior + gradually, such as starting in observation mode, then injecting hints, then + allowing scheduling or cache-planning behavior. **Root Scope** - -The **root scope** is the implicit base scope in every scope stack. All other scopes in that stack are descendants of the root. +: The root scope is the implicit base scope in every scope stack. All other + scopes in that stack are descendants of the root. **Root UUID** +: The root UUID is the identifier of the root scope for a scope stack. Events + carry this value as `root_uuid` so subscribers and exporters can group or + filter concurrent agent runs. -The **root UUID** is the identifier of the root scope for a scope stack. Events carry this value as ``root_uuid`` so subscribers and exporters can group or filter concurrent agent runs. +## S **Sanitize Request** - -A **sanitize-request guardrail** rewrites the payload recorded on the emitted start event. +: A sanitize-request guardrail rewrites the payload recorded on the emitted + start event. **Sanitize Response** - -A **sanitize-response guardrail** rewrites the payload recorded on the emitted end event. +: A sanitize-response guardrail rewrites the payload recorded on the emitted + end event. **Scope** - -A **scope** is a named unit of ownership in the runtime. Scopes create the parent-child structure that all emitted work attaches to. Scopes identify which request, task, or agent run owns the work, what the parent is for each tool or LLM call, and which scope-local middleware or subscribers are visible. +: A scope is a named unit of ownership in the runtime. Scopes create the + parent-child structure that all emitted work attaches to. Scopes identify + which request, task, or agent run owns the work, what the parent is for each + tool or LLM call, and which scope-local middleware or subscribers are visible. **Scope Handle** - -A **scope handle** is the runtime identifier returned by scope, tool, or LLM start helpers. Manual lifecycle APIs use handles to pair explicit start and end calls. +: A scope handle is the runtime identifier returned by scope, tool, or LLM + start helpers. Manual lifecycle APIs use handles to pair explicit start and + end calls. **Scope Stack** - -A **scope stack** is the active stack of scopes for the current task, thread, or request context. The stack always includes a root scope, and pushed scopes form the current parent chain for tools, LLM calls, marks, middleware visibility, and subscriber visibility. Use a fresh scope stack to isolate concurrent requests or agents. Propagate an existing scope stack only when detached work should remain part of the same logical trace. +: A scope stack is the active stack of scopes for the current task, thread, or + request context. The stack always includes a root scope, and pushed scopes + form the current parent chain for tools, LLM calls, marks, middleware + visibility, and subscriber visibility. Use a fresh scope stack to isolate + concurrent requests or agents. Propagate an existing scope stack only when + detached work should remain part of the same logical trace. **Scope Type** - -A **scope type** is the semantic category of a scope, such as ``Agent``, ``Function``, ``Tool``, ``Llm``, ``Retriever``, ``Embedder``, ``Reranker``, ``Guardrail``, ``Evaluator``, ``Custom``, or ``Unknown``. +: A scope type is the semantic category of a scope, such as `Agent`, + `Function`, `Tool`, `Llm`, `Retriever`, `Embedder`, `Reranker`, `Guardrail`, + `Evaluator`, `Custom`, or `Unknown`. **Stream Execution Intercept** - -A **stream execution intercept** is the streaming LLM variant of an execution intercept. It wraps the real stream lifecycle rather than a single request/response callback. +: A stream execution intercept is the streaming LLM variant of an execution + intercept. It wraps the real stream lifecycle rather than a single + request/response callback. **Subscriber** +: A subscriber is a consumer of emitted events. Subscribers receive the runtime + event stream and can use it for in-process analytics, forwarding, or export. -A **subscriber** is a consumer of emitted events. Subscribers receive the runtime event stream and can use it for in-process analytics, forwarding, or export. Examples include: - Custom event consumers - ATIF export - OpenTelemetry export - OpenInference export + Examples include custom event consumers, ATIF export, OpenTelemetry export, + and OpenInference export. -**Tool Call** +## T -A **tool call** is an instrumented invocation of a named tool or function-like operation. Managed tool calls emit start and end events, run tool middleware, and can carry an optional provider-specific ``tool_call_id``. +**Tool Call** +: A tool call is an instrumented invocation of a named tool or function-like + operation. Managed tool calls emit start and end events, run tool middleware, + and can carry an optional provider-specific `tool_call_id`. **Tool Parallelism** - -**Tool parallelism** is adaptive guidance about which tool calls may be run concurrently or scheduled differently based on observed dependency patterns. Supported modes include ``observe_only``, ``inject_hints``, and ``schedule``. +: Tool parallelism is adaptive guidance about which tool calls can run + concurrently or be scheduled differently based on observed dependency + patterns. Supported modes include `observe_only`, `inject_hints`, and + `schedule`. **Trace Span** - -A **trace span** is a timed observability record in a tracing backend. Exported NeMo Relay scopes, tool calls, LLM calls, and marks appear as spans when using OpenTelemetry or OpenInference export. +: A trace span is a timed observability record in a tracing backend. Exported + NeMo Relay scopes, tool calls, LLM calls, and marks appear as spans when using + OpenTelemetry or OpenInference export. **Trajectory** +: A trajectory is an ordered record of an agent run. In ATIF export, LLM events + become agent steps, tool events become tool calls and observations, and scope + nesting becomes lineage metadata. -A **trajectory** is an ordered record of an agent run. In ATIF export, LLM events become agent steps, tool events become tool calls and observations, and scope nesting becomes lineage metadata. +**Transparent Run** +: A transparent run is a no-install CLI launch that starts a private local + gateway, injects temporary hook and gateway configuration into one coding + agent process tree, and stops the gateway when that agent exits. Use an agent + shortcut such as `nemo-relay codex` or `nemo-relay run -- `. **Typed Value Codec** +: A typed value codec converts application-facing values to JSON before NeMo + Relay runs middleware or emits events, then converts JSON back into the type + expected by the framework callback or caller. + +## Related Documentation -A **typed value codec** converts application-facing values to JSON before NeMo Relay runs middleware or emits events, then converts JSON back into the type expected by the framework callback or caller. +- [NeMo Relay Overview](/about-nemo-relay/overview) +- [NeMo Relay Concepts](/about-nemo-relay/concepts) +- [API Reference](/reference/api) +- [Support and FAQs](/resources/support-and-faqs) diff --git a/docs/supported-integrations/about.mdx b/docs/supported-integrations/about.mdx index 3d193f12a..fe0323b17 100644 --- a/docs/supported-integrations/about.mdx +++ b/docs/supported-integrations/about.mdx @@ -19,10 +19,10 @@ security middleware, and optimization features. | Agent / Library | Observability | Security | Optimization | Notes | |:--|:--:|:--:|:--:|:--| -| LangChain | ✅ Yes | ✅ Yes | ✅ Yes | Wrapped tool and LLM calling. | -| LangGraph | ✅ Yes | ✅ Yes | ✅ Yes | Wrapped tool and LLM calling. | -| Deep Agents | ✅ Yes | ✅ Yes | ✅ Yes | Wrapped tool and LLM calling. | -| OpenClaw | ✅ Yes | ⚠️ Partial | ❌ No | Hook-backed telemetry with pre-tool guardrails. Public hooks do not expose managed execution rewrites. | +| LangChain | Yes | Yes | Yes | Wrapped tool and LLM calling. | +| LangGraph | Yes | Yes | Yes | Wrapped tool and LLM calling. | +| Deep Agents | Yes | Yes | Yes | Wrapped tool and LLM calling. | +| OpenClaw | Yes | Partial | No | Hook-backed telemetry with pre-tool guardrails. Public hooks do not expose managed execution rewrites. | ## Guides diff --git a/fern/docs.yml b/fern/docs.yml index 6c7d106b5..579f8eeee 100644 --- a/fern/docs.yml +++ b/fern/docs.yml @@ -21,6 +21,10 @@ redirects: # Release notes - source: /nemo/relay/about-nemo-relay/release-notes/related-topics destination: /nemo/relay/about-nemo-relay/release-notes +- source: /nemo/relay/about-nemo-relay/release-notes/highlights + destination: /nemo/relay/about-nemo-relay/release-notes +- source: /nemo/relay/about-nemo-relay/release-notes/known-issues + destination: /nemo/relay/about-nemo-relay/release-notes # Generated API references - source: /nemo/relay/reference/api/python-library-reference/pii-redaction