diff --git a/docs/about-nemo-relay/release-notes/index.mdx b/docs/about-nemo-relay/release-notes/index.mdx index f9495fca8..3f1b3137c 100644 --- a/docs/about-nemo-relay/release-notes/index.mdx +++ b/docs/about-nemo-relay/release-notes/index.mdx @@ -1,7 +1,7 @@ --- title: "Release Notes for NVIDIA NeMo Relay" sidebar-title: "Release Notes" -description: "Review highlights, compatibility updates, fixed known issues, and current known issues for NVIDIA NeMo Relay 0.7." +description: "Review highlights, compatibility updates, fixed known issues, and current known issues for NVIDIA NeMo Relay 0.8." template-library-version: "1.0.0" position: 6 --- @@ -25,155 +25,29 @@ This is the Release Notes template. Document one release version per page and us This page contains the release notes for [NVIDIA NeMo Relay](/about-nemo-relay/overview). -## Release 0.7 +## Release 0.8 -NVIDIA NeMo Relay 0.7 adds asynchronous middleware, codec-aware observability -sanitization, typed OpenTelemetry exporters, cross-boundary context -propagation, and opt-in exact-match LLM response caching. It also expands CLI -configuration, packaging, logging, and diagnostics workflows. +NVIDIA NeMo Relay 0.8 release notes are in preparation. ### Highlights -The following highlights summarize the most significant user-facing changes in -0.7: - -- **Asynchronous middleware and ordered event delivery:** Rust middleware - callbacks can await work directly, Python callbacks can return awaitables, - and Node.js callbacks can return Promises. Scope and mark APIs remain - synchronous while Relay publishes sanitized events through an ordered - background dispatcher. -- **Codec-aware privacy controls:** LLM sanitizers receive the active per-call - codec across in-process, native-plugin, and worker boundaries. The PII - redaction plugin can protect mixed OpenAI Chat, OpenAI Responses, and - Anthropic Messages traffic without a fixed provider codec. -- **Unified OpenTelemetry export:** One typed, multi-endpoint configuration - supports `full`, `gen_ai`, and `openinference` projections. The `gen_ai` - projection implements a pinned - [OpenTelemetry GenAI semantic-conventions v1.42-era snapshot](https://github.com/open-telemetry/semantic-conventions-genai/tree/43633a68ef8f8ed87a1d5eb205990311ca708bf1). -- **Portable Relay context:** Transport-neutral propagation can seed isolated - scope stacks across process and runtime boundaries. Python applications can - also fork the active scope stack for concurrent `asyncio` tasks. -- **Opt-in LLM response caching:** The Adaptive plugin can serve eligible - exact-match calls from in-memory or Redis stores. The cache remains disabled - unless `response_cache` is configured. -- **CLI and operational improvements:** The CLI adds structured configuration - editing, PyPI distribution, size-based log rotation, and offline `doctor` - probes. Python, Node.js, and Go bindings initialize default operational - logging from the same environment controls as the CLI. +- _Highlights will be added for the 0.8 release._ ### Support Matrix and Compatibility Updates - -LLM sanitizer callbacks require migration in 0.7. Callbacks now receive the -payload first and a directional context second, and they return an optional -payload. There is no one-argument compatibility adapter. Rust worker mark, -scope, tool, and LLM sanitizer callbacks are also asynchronous in 0.7. Native -plugins and workers must be rebuilt with the matching 0.7 SDK and protocol -definitions. - - The [Support Matrix](/reference/support-matrix) is the canonical reference for supported platforms and architectures, worker runtimes, coding agents, and integrations. It also records current limitations, including platform-specific worker requirements. -Before upgrading from 0.6, follow the -[0.7 Migration Guide](/reference/migration-guides#upgrade-to-nemo-relay-07). -For the new callback contract and codec operations, refer to -[Codec-Aware LLM Sanitizers](/about-nemo-relay/concepts/middleware#codec-aware-llm-sanitizers). - -Rust callers that exhaustively match `FlowError` must handle the new -`CallbackException` variant. It retains Python and JavaScript exception types -for OpenTelemetry projection while preserving the existing `internal_error` -classification. - -Observability configuration version 3 is a breaking change. It removes the -standalone `openinference` section and standalone OpenInference subscriber APIs, -and requires a type and endpoint for every OpenTelemetry exporter. Legacy mark -and attribute-alias controls remain available on typed `full` and -`openinference` endpoints. Refer to -[Migration Guides](/reference/migration-guides#observability-configuration-version-3) -for before-and-after configuration and binding API changes. - -Each `opentelemetry.endpoints` entry can resolve sensitive header values from -environment variables through `header_env`. Plugin activation rejects missing -variables, blank values, and duplicate header names, then snapshots each valid -value for the endpoint. -Process-global `OTEL_EXPORTER_OTLP_HEADERS` and -`OTEL_EXPORTER_OTLP_TRACES_HEADERS` now fail construction or activation because -their values cannot be isolated between endpoints. - -The experimental Relay-owned Switchyard plugin is deprecated in 0.7. It is -excluded from default CLI builds and is scheduled for removal in 0.8, when a -Switchyard-owned native plugin is expected to replace it. - -### Fixed Known Issues in 0.7 - -The following known issues are fixed in 0.7: - -- Sanitizer callback errors, panics, rejected awaitables or Promises, and - invalid return values now fail closed. Relay withholds the governed - observability payload instead of emitting the original unsanitized value. -- Relay removes known LLM credential headers before emitting request events, - and the PII redaction plugin rejects malformed RFC 6901 `target_paths` before - activation. -- Generated coding-agent enforcement hooks now fail closed for transport, - startup, authentication, evaluation, malformed-response, and policy-delivery - failures. Lifecycle and after-the-fact hooks continue to fail open. -- OpenTelemetry exporters now report finite batch-queue span loss through - operational logs and plugin runtime diagnostics, including an exact dropped - span count during graceful shutdown. -- Typed OpenTelemetry configuration now rejects colliding destinations after - canonical endpoint resolution, validates attribute mappings before - activation, and completes bare OTLP HTTP endpoints with `/v1/traces`. -- OpenTelemetry and OpenInference prefer response-side model attribution, - preserve binding callback exception types, and map structured `FlowError` - values to useful `error.type` attributes. OpenInference emits system content - as ordered input messages. -- ATIF no longer duplicates user steps for same-turn continuations. Model - fallback matches the span exporters, and metadata-routing and remote-delivery - failures remain visible and recoverable through flush and teardown. -- Explicit configuration editor targets are preserved, file layers follow one - documented priority model, disabled plugin entries do not contribute - configuration, and programmatic component enablement takes precedence over - discovered files. -- Library initialization reports inherited `plugins.toml` files without - exposing their values. `nemo-relay doctor` reports invalid explicit and - logging configuration, supports offline exporter validation, and reports - effective upstream authentication presence without exposing credentials. -- Transparent gateways preserve non-success streaming provider responses and - have bounded, interruptible shutdown behavior. -- Node.js preserves asynchronous propagation context, includes queued callback - delivery in flush completion, and isolates subscriber exceptions instead of - aborting the process. -- Python plugin teardown is safe from a running `asyncio` loop, cancelled - scopes are reported as errors, and completion callbacks no longer target a - closed event loop during cancellation. -- Prebuilt CLI binaries carry the intended release version, so an upgraded CLI - does not silently reuse a gateway started by a differently identified build. -- `object_store` is upgraded to `0.14.1`, which removes the temporary - `RUSTSEC-2026-0194` and `RUSTSEC-2026-0195` advisory exceptions. -- Embedded Python hosts can use - `plugin.load_dynamic_plugin_activation_specs(path)` to convert the standard - `[[plugins.dynamic]]` records in one explicitly selected `plugins.toml` into - the activation specs accepted by `initialize_with_dynamic_plugins()`. This - scoped 0.7 compatibility helper removes host-side TOML and manifest parsing; - a future unified file-backed initializer is expected to replace it. -- Programmatically declared plugin components now apply their `enabled` value - over discovered file configuration. When code re-enables a component that a - discovered file disabled, initialization reports a warning that names the - component and source file. -- Library plugin initialization now emits a warning for each discovered - `plugins.toml` file inherited by the caller configuration. The warning is - available in operational logs, initialization results, and the active plugin - report without exposing configuration values or credentials. -- LLM payload redaction now follows the codec active for each call instead of a - codec captured from plugin configuration. Codec-dependent policies omit the - observability payload and annotation when Relay cannot safely normalize the - payload. This change resolves the mixed-provider PII sanitization issue - [#526](https://github.com/NVIDIA/NeMo-Relay/issues/526). - -## Known Issues in 0.7 +Migration guidance for upgrading from 0.7 to 0.8 will be added to the +[Migration Guides](/reference/migration-guides) before the release. + +### Fixed Known Issues in 0.8 + +- _Fixed issues will be added for the 0.8 release._ + +## Known Issues in 0.8 - Go and the raw C FFI remain experimental and source-first. Generated API pages focus on Rust, Python, and Node.js. diff --git a/docs/reference/migration-guides.mdx b/docs/reference/migration-guides.mdx index 9d403d41c..52207b34f 100644 --- a/docs/reference/migration-guides.mdx +++ b/docs/reference/migration-guides.mdx @@ -1,574 +1,19 @@ --- title: "Migration Guides" -description: "Upgrade NeMo Relay integrations and migrate async middleware, plugins, workers, LLM sanitizers, and PII policy." +description: "Plan an upgrade from NeMo Relay 0.7 to 0.8." position: 6 --- {/* SPDX-FileCopyrightText: Copyright (c) 2026, NVIDIA CORPORATION & AFFILIATES. All rights reserved. SPDX-License-Identifier: Apache-2.0 */} -Use this page to plan an upgrade from NeMo Relay 0.6 to 0.7. It will group the +Use this page to plan an upgrade from NeMo Relay 0.7 to 0.8. It will group the actions from the release notes by the surface you operate. If you skip one or more releases, review the migration guides and release notes for every intervening release in sequence. -## Upgrade to NeMo Relay 0.7 +## Upgrade to NeMo Relay 0.8 -NeMo Relay 0.7 makes the Rust middleware callback contract asynchronous and -adds awaitable middleware support across the in-process bindings, native -plugins, and worker plugins. It also changes the LLM -observability sanitizer contract. Complete the following migrations before you -run existing middleware or a sanitizer with a 0.7 host. - - -A 0.6 native plugin can still load through the legacy v2 table fallback, but it -is not compatible with the changed middleware and LLM sanitizer callback -contracts or other changed ABI and schema behavior. Rebuild plugins and workers -for 0.7 before using those surfaces. NeMo Relay does not adapt synchronous Rust -middleware callbacks or one-argument LLM sanitizer callbacks. - - -Rust code that constructs `AdaptiveConfig` with an exhaustive struct literal -must add `response_cache: None`. Prefer `..AdaptiveConfig::default()` when the -literal should remain compatible with new optional fields. - -### Update Exhaustive FlowError Matches - -Rust callers that exhaustively match `FlowError` must add the new -`FlowError::CallbackException` variant or use a wildcard arm. The variant -retains the originating Python or JavaScript exception type for observability -while preserving the existing `internal_error` classification. - -### Reinstall Coding-Agent Enforcement Hooks - -Generated enforcement hooks now fail closed when Relay cannot start, -authenticate, evaluate, or deliver a response. Generated lifecycle and -after-the-fact hooks continue to fail open. - -After upgrading, reinstall every persistent coding-agent integration so its -generated hooks use the 0.7 policy: - -```bash -nemo-relay install claude-code --force -nemo-relay install codex --force -nemo-relay install hermes --force -``` - -Custom enforcement hooks must pass `--fail-closed` or set -`NEMO_RELAY_FAIL_CLOSED=1`. Do not apply fail-closed behavior to lifecycle or -after-the-fact hooks that cannot prevent the action they observe. - -### Migrate Middleware Callbacks - -The following callback families are now asynchronous: conditional execution -guardrails, request intercepts, execution intercepts, tool and LLM sanitizers, -and event sanitizers. Relay awaits each registered callback sequentially in -priority order. A callback that rejects or returns an error preserves the -existing error behavior for its middleware family. - -| Surface | 0.6 Callback | 0.7 Callback | -| --- | --- | --- | -| Rust | `Fn(...) -> Result` | `Fn(...) -> Pin> + Send>>` | -| Python | Direct return value | Direct return value or awaitable | -| Node.js | Direct return value | Direct return value or `Promise` | -| Go / raw C FFI | Synchronous callback | Synchronous callback | - -Conditional-execution guardrails and request or execution intercepts retain a -fail-closed exception contract in 0.7. Relay stops at the failing middleware -stage and surfaces the error to the managed caller. Conditional guardrail and -request-intercept failures prevent the real callback from running. This -behavior is the same for callbacks that return directly and callbacks that -complete asynchronously. If an execution intercept fails after invoking `next`, -Relay cannot undo completed downstream work. Relay rejects `next` calls that -remain unfinished and calls that begin after the interceptor settles. - -The experimental Go and raw C FFI callbacks remain synchronous. Relay waits -for each callback on a native thread, so blocking I/O and other long-running -callback work occupy that thread. No completion-based registration API is -provided for these bindings. - -Python's standalone middleware helpers preserve their direct synchronous -return when called without a running `asyncio` loop. In that mode, registered -callbacks must also return direct values; an awaitable callback raises a clear -runtime error. Call the helper from async Python and await its result when any -entry may return an awaitable. - -For Rust, wrap the existing result in a ready async future, or use an async -block when the callback needs to await work: - -```rust -use std::sync::Arc; - -use nemo_relay::api::registry::register_tool_conditional_execution_guardrail; - -register_tool_conditional_execution_guardrail( - "policy", - 10, - Arc::new(|_name, _args| { - Box::pin(async move { - // Await policy I/O here when needed. - Ok(None) // Return Some(reason) to block execution. - }) - }), -)?; -``` - -Rust middleware inputs that previously borrowed call data are now owned so -they can live for the duration of the returned future. Update explicitly typed -closures and named callback functions as follows: - -- Event sanitizers receive `Arc` instead of `&Event`. -- Tool sanitizers, conditional guardrails, and request intercepts receive an - owned `String` tool name; tool conditional guardrails also receive an owned - `Json` payload instead of `&Json`. -- LLM conditional guardrails receive an owned `LlmRequest` instead of - `&LlmRequest`, and LLM request intercepts receive an owned `String` call - name. - -Python and Node.js registration names are unchanged. Mark a Python callback -`async def`, or return a Promise from Node.js, only when it needs asynchronous -work; existing direct-value callbacks remain supported. - - -Do not introduce recursive middleware calls while converting callbacks to -async. Event sanitizers, conditional-execution guardrails, request intercepts, -execution intercepts, and subscribers must not call NeMo Relay APIs that run -middleware, flush subscribers, wait on exporters, or clear plugins. Within -these callbacks, scope operations remain supported at every nesting level, -including replacement of the active stack with an arbitrary scope stack. Event -emission is the only supported operation that can enqueue additional callback -work. An execution intercept may also invoke its supplied `next` continuation. - - -When queued Python middleware has no live captured event loop—including when -it was registered outside a loop or its registration loop has closed—the -fallback uses `asyncio.run` with a fresh loop. Bare coroutine results are -supported in that fallback; `Task` and `Future` instances bound to a different -loop are not. - -Scope, mark, and manual tool/LLM lifecycle APIs remain synchronous. This -includes `push_scope`, `pop_scope`, mark APIs, `tool_call`, `tool_call_end`, -`llm_call`, and `llm_call_end`. These APIs snapshot the event and visible -sanitizer/subscriber chain, then enqueue sanitization and publication on a -serial dispatcher. Event subscribers and exporters therefore receive sanitized -events later, in emission order. Do not add `await` to these lifecycle calls. -Only enqueue-time validation and runtime-state errors are returned directly. -Sanitizer errors discovered during queued publication are logged and fail -closed; codec errors retain their documented fallback behavior. - -Python subscriber flushing is context-sensitive. Continue to call -`nemo_relay.subscribers.flush()` from synchronous code. From a running -`asyncio` event loop, replace that call with -`await nemo_relay.subscribers.flush_async()` so queued async middleware can -continue running; the synchronous flush raises in that context. - -### Update LLM Sanitizer Callbacks - -The registration names remain unchanged for global, plugin-context, and -scope-local sanitizers. Update the callback itself as follows: - -| Surface | 0.6 Contract | 0.7 Contract | -| --- | --- | --- | -| Request | `(LlmRequest) -> LlmRequest` | `(LlmRequest, LlmSanitizeRequestContext) -> Option` | -| Response | `(Json) -> Json` | `(Json, LlmSanitizeResponseContext) -> Option` | - -The payload is always the first argument. The directional context is always the -second argument. Update request sanitizers with the following pattern: - - - - -```rust -use std::sync::Arc; - -use nemo_relay::api::registry::register_llm_sanitize_request_guardrail; - -register_llm_sanitize_request_guardrail( - "redact-request", - 10, - Arc::new(|request, context| { - Box::pin(async move { - let _active_codec = context.resolve_codec(); - // Apply policy, using _active_codec when normalized access is required. - Ok(Some(request)) - }) - }), -)?; -``` - - - - -```python -import nemo_relay -from nemo_relay import LLMRequest -from nemo_relay import guardrails - -def redact_request( - request: LLMRequest, - context: nemo_relay.LlmSanitizeRequestContext, -) -> LLMRequest | None: - _active_codec = context.resolve_codec() - # Apply policy, using _active_codec when normalized access is required. - return request - -guardrails.register_llm_sanitize_request( - "redact-request", - 10, - redact_request, -) -``` - - - - -```js -const relay = require("nemo-relay-node"); - -relay.registerLlmSanitizeRequestGuardrail( - "redact-request", - 10, - (request, context) => { - const _activeCodec = context.resolveCodec(); - // Apply policy, using _activeCodec when normalized access is required. - return request; - }, -); -``` - - - - -Apply the same change to response sanitizers with -`LlmSanitizeResponseContext`. Return the response payload to keep it in the -observability event. - -### Handle Payload Omission - -In 0.7, `None` or `null` has a specific fail-closed meaning for an LLM -sanitizer: Relay omits the observability payload and its annotation. Omission -short-circuits the remaining LLM sanitizer chain but does not change the -request or response returned to the client. - -Check every callback for an implicit empty return. In particular: - -- A Python function that reaches the end without `return` omits the payload. -- A JavaScript callback that returns `null` or `undefined` omits the payload. -- A Rust callback must return `Some(payload)` to retain the payload. -- A sanitizer error or panic fails closed: Relay omits the payload and - annotation, logs or records the callback error, and continues publication. - -Use omission only when recording the payload would be unsafe. - -### Resolve the Active Codec - -The request and response contexts expose `codec`, which has one of the -following identities: - -| Identity | Meaning | `resolve_codec()` | -| --- | --- | --- | -| `none` | No codec is active for this payload direction. | Returns no codec. | -| `builtin(id)` | A built-in codec is active. | Returns the active codec. | -| `runtime(id)` | A named runtime-registered codec is active. | Returns the active codec. | -| `opaque` | A codec without a public ID is active. | Returns the active codec. | - -Treat identity as descriptive and codec resolution as authoritative. Do not -infer a provider from the request shape. A request codec supports -`decode(request)` and `encode(annotated, original)`. A response codec supports -`decode_response(response)` in in-process Rust and Python, -`decode(response).await` in Rust workers, `await decode(response)` in Python -workers, and `decodeResponse(response)` in Node.js; response encoding is not -available. - -Worker codec proxies are scoped to the sanitizer callback and must not be -stored for later work. Worker codec operations are asynchronous because they -call the host over `grpc-v1`. In-process resolvers return the active codec -facade directly. - -For complete in-process examples, refer to -[Codec-Aware LLM Sanitizers](/about-nemo-relay/concepts/middleware#codec-aware-llm-sanitizers). - -### Migrate Worker Sanitizers - -All Rust worker middleware registrations now require callbacks that return -futures. This includes conditional guardrails, request and execution intercepts, -mark and scope event sanitizers, tool request/response sanitizers, and LLM -request/response sanitizers. Python worker middleware can return either an -immediate value or an awaitable. Python LLM sanitizers must still accept both -the payload and directional context. - -Update Rust worker callbacks to return `Box::pin(async move { ... })` and -resolve to `Result` from the future. - - - - -```rust -ctx.register_llm_sanitize_request_guardrail( - "redact-request", - 10, - |request, context| { - Box::pin(async move { - if let Some(codec) = context.resolve_codec() { - let annotated = codec.decode(&request).await?; - let request = codec.encode(&annotated, &request).await?; - return Ok(Some(request)); - } - Ok(Some(request)) - }) - }, -); -``` - - - - -```python -from nemo_relay_plugin import LlmSanitizeRequestContext - -async def redact_request(request, context: LlmSanitizeRequestContext): - codec = context.resolve_codec() - if codec is None: - return request - annotated = await codec.decode(request) - return await codec.encode(annotated, request) - -ctx.register_llm_sanitize_request_guardrail( - "redact-request", - redact_request, - priority=10, -) -``` - - - - -Regenerate worker bindings from the 0.7 -`nemo.relay.worker.v1` definition and deploy the 0.7 host and worker SDK -together. The protocol identifier remains `grpc-v1`, but the wire contract now -includes: - -- `LlmCodecIdentity` -- `LlmSanitizeRequestContext` -- `LlmSanitizeResponseContext` -- Host RPCs for request decode, request encode, and response decode - -The previous codec fields on `LlmInvocation` are reserved. Read codec identity -and capability data from the directional sanitizer context instead. - -The codec capability ID in the protocol is SDK-internal. Do not expose or -persist it in plugin code. The host rejects forged, expired, unauthorized, and -wrong-direction capability IDs. - -### Rebuild Native Plugins and Raw FFI Consumers - -NeMo Relay 0.7 uses native ABI v3. Recompile native plugins against the 0.7 -`nemo-relay-plugin` crate and rebuild raw FFI consumers against the generated -0.7 header. - -The v3 table preserves the v2 prefix, and Relay retries a legacy v2 table when -loading a plugin that rejects v3. That fallback supports loading, not -compatibility with changed middleware, LLM sanitizer, ABI, or schema contracts. -Rebuild native plugins that use the raw plugin ABI callbacks: native ABI v3 -adds completion-based async middleware registration, async execution -continuations, and explicit cancellation/late-settlement behavior. This is -separate from the synchronous `nemo-relay-ffi` middleware registration API. - -The plugin manifest value remains `compat.native_api = "1"`. This manifest -contract version is separate from the host ABI version; do not change it to -`"2"`. - -Request and response callbacks now receive distinct context structures. Each -structure contains structured codec identity and a borrowed directional codec -handle. The request handle supports decode and encode host operations. The -response handle supports decode. A successful null sanitizer output omits the -observability payload and annotation. - -Do not retain the codec handle, SDK facade, input pointers, or borrowed codec ID -after the callback returns. Release host-owned output strings with the standard -host string release operation. - -For the complete ABI contract, refer to -[Native ABI v3](/build-plugins/dynamic-plugins/native-dynamic/about#native-abi-v3). - -### Update PII Redaction Configuration - -The PII `codec` field is now an optional compatibility fallback. Relay uses it -only for a legacy or manual managed LLM call with no active codec. Any active -codec takes precedence, including runtime and opaque codecs. - -For a mixed-provider gateway, remove the fixed `codec` field: - - - - -```toml -[[components]] -kind = "pii_redaction" -enabled = true - -[components.config] - -[[components.config.profiles]] -mode = "builtin" -priority = 80 - -[components.config.profiles.builtin] -action = "redact" -detector = "email" -target_paths = ["/messages/0/content", "/message"] -``` - - - - -Provider-agnostic all-leaf and `trajectory_context` policies do not require -`codec`. A normalized `target_paths` policy fails closed when Relay has neither -a recognized active codec nor a compatible fallback: Relay omits the LLM -payload and annotation instead of recording unsanitized data. Normalized -`target_paths` response policies for runtime and opaque codecs also omit the -payload because response codec capabilities are decode-only. - -For the complete policy behavior, refer to -[PII Redaction Configuration](/configure-plugins/pii-redaction/configuration). - -### Verify the Upgrade - -Before deployment: - -1. Search for every LLM sanitize-request and sanitize-response registration. -2. Add the required directional context argument and optional payload result. -3. Check that no callback implicitly returns `None`, `null`, or `undefined`. -4. Convert every Rust worker sanitizer callback to an asynchronous callback. -5. Rebuild native plugins and raw FFI consumers against 0.7. -6. Regenerate and redeploy workers with the matching 0.7 protocol and SDK. -7. Remove fixed PII `codec` values from mixed-provider configurations. -8. Test buffered and streaming calls for every provider and custom codec that - your deployment uses. -### Observability Configuration Version 3 - -Observability configuration version 3 replaces the two version-2 OTLP sections -with one typed, multi-endpoint `opentelemetry` section. OpenTelemetry and -OpenInference support is always built in and no longer uses the `otel` or -`openinference` Cargo features. - - -Version 3 does not normalize version-2 OTLP fields. Migrate the complete -configuration before changing `components.config.version` to `3`. - - -The following example shows the version-2 shape: - -```toml -[components.config] -version = 2 - -[components.config.opentelemetry] -enabled = true -endpoint = "http://localhost:4318/v1/traces" -transport = "http_binary" -service_name = "relay-service" - -[components.config.opentelemetry.headers] -x-tenant-id = "demo" - -[components.config.openinference] -enabled = true -endpoint = "http://localhost:6006/v1/traces" -transport = "http_binary" -service_name = "relay-service" -``` - -Replace it with typed endpoints: - -```toml -[components.config] -version = 3 - -[components.config.opentelemetry] -enabled = true - -[[components.config.opentelemetry.endpoints]] -type = "full" -endpoint = "http://localhost:4318/v1/traces" -transport = "http_binary" -service_name = "relay-service" - -[components.config.opentelemetry.endpoints.headers] -x-tenant-id = "demo" - -[components.config.opentelemetry.endpoints.header_env] -authorization = "OTEL_AUTHORIZATION" - -[[components.config.opentelemetry.endpoints]] -type = "openinference" -endpoint = "http://localhost:6006/v1/traces" -transport = "http_binary" -service_name = "relay-service" -``` - -Set `OTEL_AUTHORIZATION` to the complete authorization header value before -activation. Process-global `OTEL_EXPORTER_OTLP_HEADERS` and -`OTEL_EXPORTER_OTLP_TRACES_HEADERS` are no longer accepted because their values -cannot be isolated between multiple exporters. Move non-sensitive version-2 headers to the -endpoint's `headers` map. Move `resource_attributes` and all service, -transport, instrumentation-scope, and timeout settings into the corresponding -endpoint. - -Use the following endpoint type for each projection: - -| Version-2 Source | Version-3 Endpoint Type | -|---|---| -| `opentelemetry` | `full` | -| `openinference` | `openinference` | -| New standardized GenAI-only export | `gen_ai` | - -Every endpoint requires a nonblank `endpoint` and explicit `type`. The defaults -also change: - -| Field | Version 2 | Version 3 | -|---|---|---| -| `service_name` | `nemo-relay` | `unknown_service` | -| `instrumentation_scope` | Exporter-specific or omitted | `opentelemetry` | -| `endpoint` | Exporter or environment default when omitted | Required | - -Remove `mark_projection`, `mark_exclude_names`, `attribute_mappings`, -`semantic_selector`, and `capture_content`. Projection behavior is fixed. The -`full` and `openinference` types retain their previous default mark behavior. -The `gen_ai` type omits marks, rerankers, and unsupported scope types, and does -not emit `nemo_relay.*` or opt-in content attributes. It targets the -[OpenTelemetry GenAI semantic-conventions v1.42-era snapshot](https://github.com/open-telemetry/semantic-conventions-genai/tree/43633a68ef8f8ed87a1d5eb205990311ca708bf1). - -NeMo Relay also upgrades to OpenTelemetry Rust `0.32`. - -Top-level component `config` lists now concatenate across layers, with -higher-precedence entries first. The observability destination lists -`atof.sinks`, `opentelemetry.endpoints`, and `atif.storage` follow the same -rule. Arbitrary lists nested inside structured values continue to use -replacement semantics. Edit the layer that declares an inherited entry when -you need to change or remove it. - -#### Binding API Changes - -The standalone OpenInference config and subscriber APIs are removed. Construct -one independently managed OpenTelemetry subscriber from one typed endpoint. - -| Surface | Version-2 API | Version-3 API | -|---|---|---| -| Rust | `OpenInferenceConfig`, `OpenInferenceSubscriber`, or untyped `OpenTelemetryConfig` constructors | `OpenTelemetryConfig::new(OpenTelemetryType::OpenInference, endpoint)` and `OpenTelemetrySubscriber` | -| Python | `OpenInferenceConfig`, `OpenInferenceSubscriber`, or `OpenTelemetryConfig()` | `OpenTelemetryConfig("openinference", endpoint)` and `OpenTelemetrySubscriber` | -| Node.js | `OpenInferenceSubscriber` or an OpenTelemetry config without a type | `new OpenTelemetrySubscriber({ type: "openinference", endpoint })` | -| Go | `NewOpenInferenceConfig`, `NewOpenInferenceSubscriber`, or `NewOpenTelemetryConfig()` | `NewOpenTelemetryConfig(OpenTelemetryTypeOpenInference, endpoint)` and `NewOpenTelemetrySubscriber` | -| C FFI | `nemo_relay_openinference_subscriber_*`, the untyped `nemo_relay_otel_subscriber_create`, and attribute-mapping constructors | `nemo_relay_otel_subscriber_create(otel_type, ..., endpoint, ...)` and `nemo_relay_otel_subscriber_*` lifecycle functions | - -For plugin-owned multi-endpoint configuration, use -`OpenTelemetryEndpointConfig` and `OpenTelemetrySectionConfig` in Python, -`openTelemetryEndpoint` and `openTelemetryConfig` in Node.js, or -`ObservabilityOpenTelemetryEndpointConfig` and -`ObservabilityOpenTelemetryConfig` in Go. - -Direct subscribers still own one tracer provider each. Register the subscriber -before instrumented work, then deregister, flush, and shut it down during -graceful teardown. +Migration guidance will be added before the 0.8 release. ## Related Release Information