Skip to content
Merged
13 changes: 9 additions & 4 deletions docs/getting-started/configuration.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,9 @@ position: 4
SPDX-License-Identifier: Apache-2.0 */}


NeMo Relay runtime behavior is configured through API objects and registration calls rather than a global configuration file.
Most NeMo Relay runtime behavior is configured through API objects and
registration calls rather than a global configuration file. Process-wide
operational logging also supports environment variables and TOML configuration.

## Core Runtime Setup

Expand All @@ -21,6 +23,9 @@ Most applications configure NeMo Relay by:

Use scope-local registration when behavior must be tied to one request, session, or agent run.

For process-wide stderr and file logging, refer to
[Operational Logging](/reference/operational-logging).

## Plugin Setup

Plugins use a structured plugin configuration with:
Expand Down Expand Up @@ -50,9 +55,9 @@ plugin component to own standard exporter setup and teardown. Refer to
and [Observability](/configure-plugins/observability/about)
for the supported export paths.

NeMo Relay does not require application-level environment variables for normal
runtime use. Configure most behavior through API objects, registration calls, or
plugin configuration.
Outside operational logging, NeMo Relay does not require application-level
environment variables for normal runtime use. Configure most behavior through
API objects, registration calls, or plugin configuration.

`OTEL_*` variables are only relevant when the underlying OpenTelemetry exporter
reads endpoint settings from the environment. Prefer explicit config objects in
Expand Down
14 changes: 14 additions & 0 deletions docs/nemo-relay-cli/basic-usage.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -142,6 +142,20 @@ leaving the plugin editor does not remove the saved base configuration. You can
open the plugin editor again later with `nemo-relay plugins edit`, or use
`nemo-relay plugins edit --project` for project configuration.

### Operational Logging

The CLI initializes operational logging before operational commands run.
`nemo-relay config` and `nemo-relay plugins edit` skip initialization so invalid
logging settings do not prevent configuration repair.

Configure temporary CLI settings with `--log-level` and
`--log-stderr-format`, or select an absolute TOML file with
`--log-config-path`.

For defaults, source precedence, environment variables, TOML file sinks, and
Rust APIs, refer to
[Operational Logging](/reference/operational-logging).

## Add Model Pricing for Cost Estimates

Model pricing is configured with the same `plugins.toml` discovery path as
Expand Down
126 changes: 126 additions & 0 deletions docs/reference/operational-logging.mdx
Comment thread
willkill07 marked this conversation as resolved.
Original file line number Diff line number Diff line change
@@ -0,0 +1,126 @@
---
title: "Operational Logging"
description: "Configure NeMo Relay operational logs through CLI options, environment variables, TOML, or Rust APIs."
position: 4
---
{/* SPDX-FileCopyrightText: Copyright (c) 2026, NVIDIA CORPORATION & AFFILIATES. All rights reserved.
SPDX-License-Identifier: Apache-2.0 */}

Operational logging records diagnostics about the Relay process, including
startup, configuration, plugins, gateway behavior, and runtime failures. It is
separate from agent observability through ATOF, ATIF, OpenTelemetry, or
OpenInference.

Relay writes operational logs to stderr. Optional file sinks receive additional
copies of those records. Each record includes a root Relay ID for correlation.

## Defaults

Without configuration, Relay uses:

- `info` as the minimum log level
- Human-readable stderr output
- No file sinks

## Choose a Configuration Source

Use the source that matches how Relay is launched:

| Use Case | Configuration Source |
| --- | --- |
| Run the Relay CLI with temporary settings | `--log-*` options |
| Configure a process without a Relay config file | `NEMO_RELAY_LOG*` environment variables |
| Reuse logging settings across runs | `[logging]` in TOML |
| Embed Relay in a Rust application | `LoggingConfig` and `LoggingRuntime` |

For CLI processes, Relay selects one source in this order:

1. `--log-*` options or `--log-config-path`
2. `NEMO_RELAY_LOG*` environment variables
3. `[logging]` in the resolved Relay `config.toml`
4. Built-in defaults

Sources are selected rather than merged. Rust applications explicitly choose
which `LoggingRuntime` initialization method to use and do not apply the CLI
precedence rules.

## CLI Options

Configure the minimum level and stderr format directly:

```bash
nemo-relay --log-level debug --log-stderr-format jsonl
```

Use an absolute TOML path when file sinks or other logging settings are needed:

```bash
nemo-relay --log-config-path /absolute/path/to/logging.toml
```

Do not combine `--log-config-path` with `--log-level` or
`--log-stderr-format`.

## Environment Variables

Set these variables for the CLI or a Rust application that initializes logging
with `LoggingRuntime::configure_from_environment()`:

```bash
export NEMO_RELAY_LOG=debug
export NEMO_RELAY_LOG_STDERR_FORMAT=jsonl
```

Supported values are:

- `NEMO_RELAY_LOG`: `error`, `warn`, `info`, `debug`, or `trace`
- `NEMO_RELAY_LOG_STDERR_FORMAT`: `human` or `jsonl`

Alternatively, select an absolute TOML file:

```bash
export NEMO_RELAY_LOG_CONFIG_PATH=/absolute/path/to/logging.toml
```

`NEMO_RELAY_LOG_CONFIG_PATH` cannot be combined with the other logging
environment variables.

## TOML Configuration

Logging settings use a `[logging]` table:

```toml
[logging]
level = "info"
stderr_format = "human"
flush_interval_millis = 1000

[[logging.sinks]]
path = ".nemo-relay/logs/relay.log.jsonl"
format = "jsonl"
level = "debug"
queue_capacity = 1024
```

File sink paths are resolved relative to the process working directory. File
sinks use asynchronous queues, and `queue_capacity` cannot exceed 8,192 entries
per sink.

## Rust Library API

Initialize operational logging once during application startup by choosing one
of these public APIs:

- `LoggingRuntime::configure(config)` for a constructed `LoggingConfig`
- `LoggingRuntime::configure_from_environment()` for environment configuration
- `LoggingRuntime::configure_from_file_path(path)` for an absolute TOML path

For example:

```rust
let _logging_runtime =
nemo_relay::logging::LoggingRuntime::configure_from_environment()?;
```

Keep the returned runtime alive until application shutdown so pending file
records can be flushed.
Loading