diff --git a/docs/getting-started/configuration.mdx b/docs/getting-started/configuration.mdx index a77701e5d..1745dfbb5 100644 --- a/docs/getting-started/configuration.mdx +++ b/docs/getting-started/configuration.mdx @@ -8,7 +8,9 @@ position: 4 SPDX-License-Identifier: Apache-2.0 */} -NeMo Relay runtime behavior is configured through API objects and registration calls rather than a global configuration file. +Most NeMo Relay runtime behavior is configured through API objects and +registration calls rather than a global configuration file. Process-wide +operational logging also supports environment variables and TOML configuration. ## Core Runtime Setup @@ -21,6 +23,9 @@ Most applications configure NeMo Relay by: Use scope-local registration when behavior must be tied to one request, session, or agent run. +For process-wide stderr and file logging, refer to +[Operational Logging](/reference/operational-logging). + ## Plugin Setup Plugins use a structured plugin configuration with: @@ -50,9 +55,9 @@ plugin component to own standard exporter setup and teardown. Refer to and [Observability](/configure-plugins/observability/about) for the supported export paths. -NeMo Relay does not require application-level environment variables for normal -runtime use. Configure most behavior through API objects, registration calls, or -plugin configuration. +Outside operational logging, NeMo Relay does not require application-level +environment variables for normal runtime use. Configure most behavior through +API objects, registration calls, or plugin configuration. `OTEL_*` variables are only relevant when the underlying OpenTelemetry exporter reads endpoint settings from the environment. Prefer explicit config objects in diff --git a/docs/nemo-relay-cli/basic-usage.mdx b/docs/nemo-relay-cli/basic-usage.mdx index 82a0d656f..5a5166f97 100644 --- a/docs/nemo-relay-cli/basic-usage.mdx +++ b/docs/nemo-relay-cli/basic-usage.mdx @@ -142,6 +142,20 @@ leaving the plugin editor does not remove the saved base configuration. You can open the plugin editor again later with `nemo-relay plugins edit`, or use `nemo-relay plugins edit --project` for project configuration. +### Operational Logging + +The CLI initializes operational logging before operational commands run. +`nemo-relay config` and `nemo-relay plugins edit` skip initialization so invalid +logging settings do not prevent configuration repair. + +Configure temporary CLI settings with `--log-level` and +`--log-stderr-format`, or select an absolute TOML file with +`--log-config-path`. + +For defaults, source precedence, environment variables, TOML file sinks, and +Rust APIs, refer to +[Operational Logging](/reference/operational-logging). + ## Add Model Pricing for Cost Estimates Model pricing is configured with the same `plugins.toml` discovery path as diff --git a/docs/reference/operational-logging.mdx b/docs/reference/operational-logging.mdx new file mode 100644 index 000000000..2202fccde --- /dev/null +++ b/docs/reference/operational-logging.mdx @@ -0,0 +1,126 @@ +--- +title: "Operational Logging" +description: "Configure NeMo Relay operational logs through CLI options, environment variables, TOML, or Rust APIs." +position: 4 +--- +{/* SPDX-FileCopyrightText: Copyright (c) 2026, NVIDIA CORPORATION & AFFILIATES. All rights reserved. +SPDX-License-Identifier: Apache-2.0 */} + +Operational logging records diagnostics about the Relay process, including +startup, configuration, plugins, gateway behavior, and runtime failures. It is +separate from agent observability through ATOF, ATIF, OpenTelemetry, or +OpenInference. + +Relay writes operational logs to stderr. Optional file sinks receive additional +copies of those records. Each record includes a root Relay ID for correlation. + +## Defaults + +Without configuration, Relay uses: + +- `info` as the minimum log level +- Human-readable stderr output +- No file sinks + +## Choose a Configuration Source + +Use the source that matches how Relay is launched: + +| Use Case | Configuration Source | +| --- | --- | +| Run the Relay CLI with temporary settings | `--log-*` options | +| Configure a process without a Relay config file | `NEMO_RELAY_LOG*` environment variables | +| Reuse logging settings across runs | `[logging]` in TOML | +| Embed Relay in a Rust application | `LoggingConfig` and `LoggingRuntime` | + +For CLI processes, Relay selects one source in this order: + +1. `--log-*` options or `--log-config-path` +2. `NEMO_RELAY_LOG*` environment variables +3. `[logging]` in the resolved Relay `config.toml` +4. Built-in defaults + +Sources are selected rather than merged. Rust applications explicitly choose +which `LoggingRuntime` initialization method to use and do not apply the CLI +precedence rules. + +## CLI Options + +Configure the minimum level and stderr format directly: + +```bash +nemo-relay --log-level debug --log-stderr-format jsonl +``` + +Use an absolute TOML path when file sinks or other logging settings are needed: + +```bash +nemo-relay --log-config-path /absolute/path/to/logging.toml +``` + +Do not combine `--log-config-path` with `--log-level` or +`--log-stderr-format`. + +## Environment Variables + +Set these variables for the CLI or a Rust application that initializes logging +with `LoggingRuntime::configure_from_environment()`: + +```bash +export NEMO_RELAY_LOG=debug +export NEMO_RELAY_LOG_STDERR_FORMAT=jsonl +``` + +Supported values are: + +- `NEMO_RELAY_LOG`: `error`, `warn`, `info`, `debug`, or `trace` +- `NEMO_RELAY_LOG_STDERR_FORMAT`: `human` or `jsonl` + +Alternatively, select an absolute TOML file: + +```bash +export NEMO_RELAY_LOG_CONFIG_PATH=/absolute/path/to/logging.toml +``` + +`NEMO_RELAY_LOG_CONFIG_PATH` cannot be combined with the other logging +environment variables. + +## TOML Configuration + +Logging settings use a `[logging]` table: + +```toml +[logging] +level = "info" +stderr_format = "human" +flush_interval_millis = 1000 + +[[logging.sinks]] +path = ".nemo-relay/logs/relay.log.jsonl" +format = "jsonl" +level = "debug" +queue_capacity = 1024 +``` + +File sink paths are resolved relative to the process working directory. File +sinks use asynchronous queues, and `queue_capacity` cannot exceed 8,192 entries +per sink. + +## Rust Library API + +Initialize operational logging once during application startup by choosing one +of these public APIs: + +- `LoggingRuntime::configure(config)` for a constructed `LoggingConfig` +- `LoggingRuntime::configure_from_environment()` for environment configuration +- `LoggingRuntime::configure_from_file_path(path)` for an absolute TOML path + +For example: + +```rust +let _logging_runtime = + nemo_relay::logging::LoggingRuntime::configure_from_environment()?; +``` + +Keep the returned runtime alive until application shutdown so pending file +records can be flushed.