Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
19 commits
Select commit Hold shift + click to select a range
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 5 additions & 5 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -32,7 +32,7 @@ evaluation products, see [the Ecosystem guide](https://docs.nvidia.com/nemo/rela
| Instrument app-owned LLM or tool calls | [Quick Start Application](https://docs.nvidia.com/nemo/relay/getting-started/quick-start) |
| Use LangChain, LangGraph, Deep Agents, or OpenClaw | [Supported Integrations](https://docs.nvidia.com/nemo/relay/supported-integrations/about) |
| Build a framework or provider integration | [Integrate into Frameworks](https://docs.nvidia.com/nemo/relay/integrate-into-frameworks/about) |
| Export ATOF, ATIF, or typed OpenTelemetry projections | [Observability Plugin](https://docs.nvidia.com/nemo/relay/configure-plugins/observability/about) |
| Export ATOF events, ATIF trajectories, or OpenTelemetry traces, logs, and metrics | [Observability Plugin](https://docs.nvidia.com/nemo/relay/configure-plugins/observability/about) |
| Package reusable middleware or exporters | [Build Plugins](https://docs.nvidia.com/nemo/relay/build-plugins/about) |
| Develop or test this repository from source | [CONTRIBUTING.md](CONTRIBUTING.md) |

Expand Down Expand Up @@ -205,8 +205,8 @@ and agent-specific diagnostics.
> [!TIP]
> Start by trusting the raw Agent Trajectory Observability Format (ATOF) JSONL.
> It shows the lifecycle events Relay actually captured before anything is
> translated into Agent Trajectory Interchange Format (ATIF) or typed
> OpenTelemetry output, including the OpenInference projection.
> translated into Agent Trajectory Interchange Format (ATIF) or OpenTelemetry
> traces, logs, and metrics, including the OpenInference projection.

## Quick Start Applications

Expand Down Expand Up @@ -256,8 +256,8 @@ Relay gives those systems:
route, retry, or replace execution.
- **Plugins** so reusable observability, guardrail, adaptive, and exporter
behavior can be turned on from configuration.
- **Events and subscribers** so raw ATOF, normalized ATIF, and typed
OpenTelemetry projections all come from the same runtime stream.
- **Events and subscribers** so raw ATOF, normalized ATIF, and OpenTelemetry
traces, logs, and metrics all come from the same runtime stream.

Relay does not replace frameworks, model provider, application logic,
observability backend, or guardrail authoring system. It gives those systems a
Expand Down
100 changes: 98 additions & 2 deletions crates/core/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -32,7 +32,7 @@ Node.js bindings mirror the semantics exposed by this crate.
sanitize observability payloads, rewrite requests, or wrap execution.
- **Emit one lifecycle stream**: Subscribers can consume canonical runtime
events in-process or export them to Agent Trajectory Interchange Format
(ATIF) or typed OpenTelemetry projections.
(ATIF) or OpenTelemetry traces, logs, and metrics.
- **Integrate without changing orchestration**: Wrap framework and provider
callbacks while leaving scheduling, retries, memory, and result handling in
the owning application.
Expand All @@ -47,7 +47,7 @@ Node.js bindings mirror the semantics exposed by this crate.
one shared plugin system.
- **Built-in observability plugin**: Configure first-party Agent Trajectory
Observability Format (ATOF), Agent Trajectory Interchange Format (ATIF),
and typed OpenTelemetry exporters from the core crate.
and OpenTelemetry trace, log, and metric exporters from the core crate.
- **Codec and typed helpers**: Normalize provider requests and responses for
framework integrations.
- **Binding source of truth**: Use the runtime semantics mirrored by the
Expand Down Expand Up @@ -110,6 +110,102 @@ fn main() -> Result<(), Box<dyn std::error::Error>> {
}
```

## OTLP Logs and Metrics

Configure plugin-managed trace, log, and metric export with an observability
version-4 component. When log and metric endpoints are omitted, Relay derives
`/v1/logs` and `/v1/metrics` from each trace endpoint:

```toml
# plugins.toml
version = 1

[[components]]
kind = "observability"

[components.config]
version = 4

[components.config.opentelemetry]
enabled = true

[[components.config.opentelemetry.endpoints]]
type = "gen_ai"
endpoint = "http://localhost:4318/v1/traces"

[components.config.opentelemetry.logs]
enabled = true

[components.config.opentelemetry.metrics]
enabled = true
```

Emit a schema-tagged log mark with `event` and an atomically validated metric
group with `metric`:

```rust
use nemo_relay::api::{
event::{DataSchema, LogSeverity, MetricKind, MetricMeasurement, MetricValueType},
scope::{self, EmitMarkEventParams, EmitMetricEventParams},
};
use serde_json::json;

scope::event(
EmitMarkEventParams::builder()
.name("cache-nearly-full")
.data(json!({"entries": 900}))
.data_schema(DataSchema::builder().name("example.cache").version("1").build())
.severity(LogSeverity::Warn)
.build(),
)?;
scope::metric(
EmitMetricEventParams::builder()
.name("cache-entries")
.measurements(vec![
MetricMeasurement::builder()
.name("example.cache.entries")
.kind(MetricKind::Gauge)
.value_type(MetricValueType::U64)
.value(json!(900_u64))
.build(),
])
.build(),
)?;
```

Direct subscribers are independently managed. Register them before emitting
marks, then deregister, force-flush, and shut them down during graceful
teardown. The metric subscriber has the same lifecycle:

```rust
use nemo_relay::observability::{
otel_logs::{OpenTelemetryLogConfig, OpenTelemetryLogSubscriber},
otel_metrics::{OpenTelemetryMetricConfig, OpenTelemetryMetricSubscriber},
};

// Equivalent explicit OTLP/HTTP paths are /v1/logs and /v1/metrics, respectively.
let logs = OpenTelemetryLogSubscriber::new(OpenTelemetryLogConfig::new(
"http://localhost:4318",
))?;
let metrics = OpenTelemetryMetricSubscriber::new(OpenTelemetryMetricConfig::new(
"http://localhost:4318",
))?;
logs.register("otlp-logs")?;
metrics.register("otlp-metrics")?;

let diagnostics = logs.runtime_diagnostics();
for diagnostic in diagnostics.entries() {
eprintln!("{}: {}", diagnostic.code, diagnostic.message);
}

logs.deregister("otlp-logs")?;
logs.force_flush()?;
logs.shutdown()?;
metrics.deregister("otlp-metrics")?;
metrics.force_flush()?;
metrics.shutdown()?;
```

## Documentation

NeMo Relay Documentation: https://docs.nvidia.com/nemo/relay
115 changes: 113 additions & 2 deletions crates/ffi/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -46,8 +46,13 @@ binding consumes it through CGo.
isolation.
- **Typed OpenTelemetry export**:
`nemo_relay_otel_subscriber_create` constructs one `full`, `gen_ai`, or
`openinference` endpoint subscriber. The endpoint and projection type are
required.
`openinference` trace subscriber. Independently managed log and metric
subscribers use `nemo_relay_otel_log_subscriber_create` and
`nemo_relay_otel_metric_subscriber_create`.
- **Structured marks and metrics**: `nemo_relay_event_v2` adds optional data
schema JSON and `NemoRelayLogSeverity` to the compatible mark API.
`nemo_relay_metric_json` and `nemo_relay_metric` emit atomically validated
Relay metric measurements.
- **Generated header**: A committed `nemo_relay.h` file for C-compatible
consumers.
- **Native library outputs**: Shared and static libraries for platform
Expand All @@ -63,6 +68,112 @@ long-running callback work therefore occupy that thread and can reduce
middleware throughput. The FFI does not expose completion-based middleware
registration.

## OTLP Logs and Metrics

The raw C ABI remains experimental and source-first. Configure plugin-managed
export with the same version-4 JSON document used by every binding. Leaving
the log and metric endpoint lists absent derives their destinations from the
trace endpoint:

```c
const char *config_json =
"{\"version\":1,\"components\":[{\"kind\":\"observability\",\"config\":{"
"\"version\":4,\"opentelemetry\":{\"enabled\":true,\"endpoints\":[{"
"\"type\":\"gen_ai\",\"endpoint\":\"http://localhost:4318/v1/traces\"}],"
"\"logs\":{\"enabled\":true},\"metrics\":{\"enabled\":true}}}}]}";
char *report_json = NULL;
if (nemo_relay_initialize_plugins(config_json, &report_json) != NEMO_RELAY_STATUS_OK) {
/* inspect nemo_relay_last_error() */
}
nemo_relay_string_free(report_json);
/* Call nemo_relay_clear_plugin_configuration() during process teardown. */
```

Use `nemo_relay_event_v2` for a schema-tagged log mark and
`nemo_relay_metric_json` for an atomically validated metric group. The legacy
`nemo_relay_event` function remains valid for untyped marks:

```c
NemoRelayLogSeverity severity = NEMO_RELAY_LOG_SEVERITY_WARN;
nemo_relay_event_v2(
"cache-nearly-full", NULL, "{\"entries\":900}",
"{\"name\":\"example.cache\",\"version\":\"1\"}", NULL, &severity, NULL
);
nemo_relay_metric_json(
"cache-entries", NULL,
"[{\"name\":\"example.cache.entries\",\"kind\":\"gauge\","
"\"value_type\":\"u64\",\"value\":900}]",
NULL, NULL
);
```

Create direct log and metric subscribers independently. Register each before
emitting marks, then deregister, force-flush, shut down, and free it during
graceful teardown. Runtime diagnostics are a caller-owned, bounded JSON array
of `{"code":"...","message":"...","count":N}` entries; release the result
with `nemo_relay_string_free` after parsing it. Check every returned status
before continuing. On failure, read `nemo_relay_last_error()` immediately,
before another FFI call clears the thread-local message:

```c
#include <stdbool.h>
#include <stdio.h>

static bool relay_ok(NemoRelayStatus status) {
if (status == NEMO_RELAY_STATUS_OK) {
return true;
}
const char *message = nemo_relay_last_error();
fprintf(stderr, "nemo-relay: %s\n", message != NULL ? message : "unknown error");
return false;
}

struct FfiOpenTelemetryLogSubscriber *logs = NULL;
struct FfiOpenTelemetryMetricSubscriber *metrics = NULL;
bool logs_registered = false;
bool metrics_registered = false;

/* Equivalent explicit OTLP/HTTP paths are /v1/logs and /v1/metrics, respectively. */
if (!relay_ok(nemo_relay_otel_log_subscriber_create(
"http_binary", "http://localhost:4318", NULL, NULL, NULL, NULL, NULL, NULL,
0, "info", 0, 0, 0, &logs
))) goto cleanup;
if (!relay_ok(nemo_relay_otel_metric_subscriber_create(
"http_binary", "http://localhost:4318", NULL, NULL, NULL, NULL, NULL, NULL,
0, 0, "cumulative", 0, 0, &metrics
))) goto cleanup;
if (!relay_ok(nemo_relay_otel_log_subscriber_register(logs, "otlp-logs"))) goto cleanup;
logs_registered = true;
if (!relay_ok(nemo_relay_otel_metric_subscriber_register(metrics, "otlp-metrics"))) goto cleanup;
metrics_registered = true;

char *diagnostics_json = NULL;
if (relay_ok(nemo_relay_otel_log_subscriber_runtime_diagnostics_json(
logs, &diagnostics_json
))) {
/* Parse diagnostics_json, then release it. */
nemo_relay_string_free(diagnostics_json);
}

cleanup:
if (metrics_registered) {
(void)relay_ok(nemo_relay_otel_metric_subscriber_deregister("otlp-metrics"));
}
if (metrics != NULL) {
(void)relay_ok(nemo_relay_otel_metric_subscriber_force_flush(metrics));
(void)relay_ok(nemo_relay_otel_metric_subscriber_shutdown(metrics));
nemo_relay_otel_metric_subscriber_free(metrics);
}
if (logs_registered) {
(void)relay_ok(nemo_relay_otel_log_subscriber_deregister("otlp-logs"));
}
if (logs != NULL) {
(void)relay_ok(nemo_relay_otel_log_subscriber_force_flush(logs));
(void)relay_ok(nemo_relay_otel_log_subscriber_shutdown(logs));
nemo_relay_otel_log_subscriber_free(logs);
}
```

## Installation

Build the FFI library from a repository checkout:
Expand Down
69 changes: 69 additions & 0 deletions crates/node/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -127,6 +127,75 @@ event('initialized', handle, { binding: 'node' }, null, null, null, LogSeverity.
Call `metric()` with `MetricMeasurement` objects for metrics; Relay validates
the complete measurement group before publishing it.

## OTLP Logs and Metrics

For plugin-managed export, the `nemo-relay-node/observability` helpers create a
version-4 component. Enabling logs and metrics without signal endpoints derives
`/v1/logs` and `/v1/metrics` from the trace endpoint:

```js
const observability = require('nemo-relay-node/observability');

const component = observability.ComponentSpec({
version: 4,
opentelemetry: observability.openTelemetryConfig({
enabled: true,
endpoints: [observability.openTelemetryEndpoint({
type: 'gen_ai', endpoint: 'http://localhost:4318/v1/traces',
})],
logs: observability.openTelemetryLogConfig({ enabled: true }),
metrics: observability.openTelemetryMetricConfig({ enabled: true }),
}),
});
```

Use the final `dataSchema` and `severity` arguments for a typed log mark, and
use `metric()` for an atomically validated metric group:

```js
const {
event, metric, LogSeverity, MetricKind, MetricValueType,
} = require('nemo-relay-node');

event(
'cache-nearly-full', null, { entries: 900 }, null, null,
{ name: 'example.cache', version: '1' }, LogSeverity.Warn,
);
metric('cache-entries', [{
name: 'example.cache.entries', kind: MetricKind.Gauge,
valueType: MetricValueType.U64, value: 900,
}]);
```

Direct log and metric subscribers are independently managed. Register each
before emitting marks, then deregister, force-flush, and shut it down during
graceful teardown. `runtimeDiagnostics()` returns bounded `code`, `message`,
and `count` entries:

```js
const {
OpenTelemetryLogSubscriber, OpenTelemetryMetricSubscriber,
} = require('nemo-relay-node');

// Equivalent explicit OTLP/HTTP paths are /v1/logs and /v1/metrics, respectively.
const logs = new OpenTelemetryLogSubscriber({ endpoint: 'http://localhost:4318' });
const metrics = new OpenTelemetryMetricSubscriber({ endpoint: 'http://localhost:4318' });
logs.register('otlp-logs');
metrics.register('otlp-metrics');
try {
for (const diagnostic of logs.runtimeDiagnostics()) {
console.error(diagnostic.code, diagnostic.message);
}
} finally {
logs.deregister('otlp-logs');
logs.forceFlush();
logs.shutdown();
metrics.deregister('otlp-metrics');
metrics.forceFlush();
metrics.shutdown();
}
```

Native subscriber delivery is asynchronous. Awaiting `flushSubscribers()` drains
the native dispatcher and waits for managed terminal publications registered
before the call and the JavaScript subscriber callbacks they queue, without
Expand Down
16 changes: 16 additions & 0 deletions crates/plugin/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -53,6 +53,12 @@ the dynamic-library boundary on the stable C-compatible ABI.
continuations use a pull-based host handle.
- **Canonical tool results**: `ToolNext` returns `ToolExecutionResult`, keeping
application results and opaque annotations adjacent across native API 1.
- **Typed telemetry marks**: `PluginRuntime::emit_mark_with_options` carries a
`DataSchema` and `LogSeverity`; `emit_metric` validates and emits typed
`MetricMeasurement` values through the reserved Relay metric schema.
- **Runtime diagnostics**: `PluginRuntime::runtime_diagnostics()` reads the
active host-level `RuntimeDiagnostics` snapshot, with ordered entries and
`get(code)` lookup.

## Installation

Expand Down Expand Up @@ -151,6 +157,16 @@ can override that method when they need different configuration rules.
During plugin teardown, the SDK stops accepting new callbacks and drains
already accepted typed middleware before the plugin library unloads.

## Runtime Diagnostics

`PluginRuntime::runtime_diagnostics()` returns `RuntimeDiagnostic { code,
message, count }` entries for the active host configuration. Relay aggregates
repeated codes, retains the latest message, saturates counts, orders entries by
code, and caps the snapshot at 32 entries. The result is host-level and does
not attribute an entry to the plugin that emitted it. ABI v4 requires the
complete finalized host table. Plugins loaded through the frozen ABI-v3 or
ABI-v2 compatibility tables cannot use runtime diagnostics.

Relay scope context is restored around every poll of a registered middleware
future. Child tasks created with `tokio::spawn` do not automatically inherit
that scope context.
Expand Down
Loading
Loading