Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
34 changes: 32 additions & 2 deletions Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 2 additions & 0 deletions Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -27,6 +27,8 @@ members = [
# NVTX integration crates
"integrations/nvtx/events",
"integrations/nvtx/injection",
"integrations/nvtx/bridge",
"integrations/nvtx/example",
# Domain-specific crates
"domains/query_engine/analyzer",
"domains/query_engine/model",
Expand Down
10 changes: 10 additions & 0 deletions crates/instrumentation/src/observer.rs
Original file line number Diff line number Diff line change
Expand Up @@ -117,6 +117,16 @@ impl<T> Observer<T> {
pub fn emit(&self, id: Uuid, event: impl Into<T>) {
self.events_sender.emit(id, event);
}

/// A cloned [`EventSender`] feeding this observer's pipeline.
///
/// Lets a `'static` producer emit into the observer while the caller keeps
/// ownership (and still flushes on drop). The sender does not keep the
/// observer alive; sends after it is dropped are discarded (the first logs
/// an error via `tracing`, then further ones are suppressed).
pub fn sender(&self) -> EventSender<T> {
self.events_sender.clone()
}
}

impl<T> Drop for Observer<T> {
Expand Down
166 changes: 98 additions & 68 deletions integrations/nvtx/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,100 +6,130 @@ SPDX-License-Identifier: Apache-2.0
# NVTX integration

Captures NVTX events (ranges, marks, domains, registered strings, categories,
thread names, resources, and the core payload union) from an instrumented
application and turns them into structured events — with **no application code
changes**. NVTX loads the capture library at runtime via the
`NVTX_INJECTION64_PATH` environment variable.
thread names, resources, and the core payload union) from an application and
turns them into Quent events.

**Linux 64-bit only.** Injection relies on the ELF weak-symbol /
`NVTX_INJECTION64_PATH` mechanism, which excludes Windows and 32-bit targets;
`nvtx-injection` enforces this with a `compile_error!`.
The application owns its Quent `Context` and exporter, annotates its code with
the NVTX Rust API, and links a small shim so NVTX initializes capture
**in-process** — no cdylib, no `NVTX_INJECTION64_PATH`.

**Linux 64-bit only** — capture relies on the ELF weak-symbol mechanism
(`nvtx-injection` enforces this with a `compile_error!`).

## Contents

- [Crates](#crates)
- [How capture works](#how-capture-works)
- [Using it](#using-it)
- [Captured surface](#captured-surface)
- [NVTX bindings (generated by bindgen)](#nvtx-bindings-generated-by-bindgen)
- [Upstreaming](#upstreaming)

## Crates

| Crate | Path | Role |
|-------|------|------|
| `nvtx-events` | `events/` | The verbatim, application-agnostic NVTX event **vocabulary** (`NvtxEvent` + attribute/payload types). Pure Rust — no C, no NVTX headers, and nothing product-specific — so it can be offered upstream to the NVTX Rust crates later. |
| `nvtx-injection` | `injection/` | The application-agnostic injection **cdylib** NVTX loads. Exports `InitializeInjectionNvtx2`, fills the NVTX callback tables, converts each call into a verbatim `NvtxEvent`, and hands it to a sink-agnostic `Fn(NvtxEvent)` hook. This is the only crate bound to the NVTX C ABI. |

> A third crate — the bridge that wires captured events into a consumer's event
> pipeline and provides the self-configuring capture cdylib you actually load —
> follows in a later change.
| `nvtx-events` | `events/` | The application-agnostic NVTX event **vocabulary** (`NvtxEvent` + attribute/payload types). Pure Rust, upstreamable to the NVTX Rust crates. |
| `nvtx-injection` | `injection/` | The **NVTX C ABI layer**. Fills NVTX's callback tables, converts each call into a verbatim `NvtxEvent`, and hands it to a sink-agnostic `Fn(NvtxEvent)` hook. Attach in-process via the `static-injection` feature, or at runtime as a cdylib via `NVTX_INJECTION64_PATH`. |
| `nvtx-bridge` | `bridge/` | The **bridge**: `NvtxEventEntity`, a newtype over `NvtxEvent` implementing Quent's `EntityEvent`. The orphan rule forces the impl here; the only crate depending on Quent internals. |
| `nvtx-example` | `example/` | A runnable, self-verifying example. |

## How capture works

1. NVTX `dlopen`s the injection cdylib (via `NVTX_INJECTION64_PATH`) and calls
its exported `InitializeInjectionNvtx2` exactly once, before the first NVTX
call.
2. Injection claims NVTX's callback tables and installs its own `extern "C"`
functions — so **our code becomes NVTX's implementation** of the subscribed
calls.
3. Each callback reads the raw NVTX ABI struct into a verbatim `NvtxEvent`
(bounded by the struct's declared `size`, strings copied in, panics contained
at the C boundary) and dispatches it to the installed hook.
4. Handles, ids, and push/pop nesting levels that NVTX would normally return to
the app are synthesized so the observed application still behaves correctly.

Injection is **sink-agnostic** — it does not know where events go. A downstream
crate installs the hook (into an ndjson file, a consumer's pipeline, an
in-memory collector for tests, etc.).

### Captured surface

Injection subscribes both NVTX ASCII surfaces:

- **Domain-scoped (CORE2)** — mark, range start/end/push/pop, domain /
register-string / name-category / resource.
- **Classic default-domain (CORE)** — `nvtxMarkA`/`Ex`, `nvtxRangePushA`/`Ex`,
`nvtxRangePop`, `nvtxRangeStartA`/`Ex`/`nvtxRangeEnd`, `nvtxNameCategoryA`,
captured on the default domain (`0`) — plus OS thread naming
(`nvtxNameOsThreadA`).

Wide-char (`*W` / Unicode) variants are subscribed with warn-once stubs:
Unicode capture is still deferred, but such a call emits a one-time diagnostic
and keeps range nesting/ids valid rather than silently becoming a no-op.
1. The application links `nvtx-injection` with its **`static-injection`**
feature, publishing a *strong* `InitializeInjectionNvtx2_fnptr` that
overrides the *weak* one in NVTX's header-only client. NVTX then initializes
injection **in-process** at the first NVTX call.
2. Injection claims NVTX's callback tables — our `extern "C"` functions become
NVTX's implementation of the subscribed calls.
3. Each callback converts the raw NVTX ABI struct into a verbatim `NvtxEvent`
and dispatches it to the installed hook.
4. The application's hook forwards each event (wrapped in `NvtxEventEntity`)
into its `Observer`. Handles, ids, and nesting levels are synthesized so the
app still behaves correctly.

## Using it

The app owns the pipeline; wiring capture is two steps — build an observer, then
install the hook:

```rust
// 1. The app owns its Quent context and picks the exporter.
let ctx = Context::try_new(session)?;
let options = FileSystemExporterOptions::new(FileSystemFormat::Ndjson, out_dir);
let observer = ctx.block_on(async { ctx.observer::<NvtxEventEntity>(options).await })?;

// 2. Forward captured NVTX events into it, before the first NVTX call.
let sender = observer.sender();
nvtx_injection::install_hook(move |event| sender.emit(session, event))?;

// 3. Ordinary app code, annotated with the NVTX Rust API.
nvtx::mark!("startup");
let range = nvtx::range!("phase-1");
drop(range); // end the range before flushing

// 4. Flush by dropping the observer.
drop(observer);
```

`static-injection` is requested in the manifest:

## NVTX bindings (generated by bindgen)
```toml
nvtx-injection = { path = "…/injection", features = ["static-injection"] }
```

The bundled example uses a **callback exporter** to debug-print each captured
event (a real app would swap in ndjson, the collector, …). Run it — no GPU:

```sh
pixi run cargo run -p nvtx-example
```

It prints one line per event — entity `id`, capture `timestamp` (ns), and the
`NvtxEvent`:

`injection/build.rs` runs `bindgen` over the NVTX injection ABI on every build
and writes the result to `$OUT_DIR/bindings.rs`, which `injection/src/lib.rs`
`include!`s. Nothing is checked in — there is no `bindings.rs` in the source
tree to keep in sync.
```text
[019f… @ 1784726504601037528] Mark { domain: 0, attributes: { message: Some(String("startup")), .. } }
```

### Test

`example/tests/capture.rs` reuses the same capture routine in-process with a
collecting callback exporter and asserts every core NVTX kind is captured — no
subprocess, no files:

```sh
pixi run cargo test -p nvtx-example
```

## Captured surface

The headers and `libclang` come from the pixi env, pinned in the workspace
`pixi.toml` for `linux-64` (the only target this crate supports):
Both NVTX ASCII surfaces: **domain-scoped (CORE2)** — mark, range
start/end/push/pop, domain/register-string/name-category/resource — and the
**classic default domain (CORE)** on domain `0`, plus OS thread naming.

Wide-char (`*W` / Unicode) variants get warn-once stubs: capture is deferred,
but a call emits a one-time diagnostic and keeps nesting/ids valid instead of
silently no-oping.

## NVTX bindings (generated by bindgen)

`injection/build.rs` runs `bindgen` over the NVTX injection ABI into
`$OUT_DIR/bindings.rs` on every build (nothing checked in). Headers and
`libclang` come from the pixi env, pinned for `linux-64`:

```toml
[target.linux-64.dependencies]
nvtx-c = ">=3.5.0" # NVTX C headers at $CONDA_PREFIX/include/nvtx3
libclang = ">=18" # libclang for bindgen at $CONDA_PREFIX/lib
```

So a `pixi run cargo …` build is hermetic: `build.rs` reads `CONDA_PREFIX`, adds
`$CONDA_PREFIX/include` to the clang include path, and points `LIBCLANG_PATH` at
`$CONDA_PREFIX/lib`. Bumping the captured NVTX version is a one-line `nvtx-c`
version bump in `pixi.toml` — no regeneration step, no committed file to update.

Building outside the pixi env fails fast: `build.rs` requires `CONDA_PREFIX` and
the pinned `nvtx-c` / `libclang` packages on the path.
`pixi run cargo …` is hermetic; bumping the NVTX version is a one-line `nvtx-c`
bump. Building outside pixi fails fast (needs `CONDA_PREFIX`).

## Upstreaming

`nvtx-events` and `nvtx-injection` are kept deliberately application-agnostic —
the vocabulary depends only on `serde`, and injection depends only on the
vocabulary — because both are **very likely to be contributed upstream to the
NVTX Rust crates**: the event vocabulary and a consumer/`tools` surface (the
injection ABI the current `nvtx-sys` producer-only crate does not expose). If
and when that lands upstream, a consumer would depend on those crates instead
of vendoring these two here.

This is a parallel effort and **not a blocker** for landing NVTX capture in this
repo; the crates are structured now so that switching over later is a clean
dependency swap.
`nvtx-events` and `nvtx-injection` are kept application-agnostic so they can be
contributed upstream to the NVTX Rust crates (`nvtx-sys` is producer-only
today). If that lands, this repo depends on them instead of vendoring — a clean
swap. Parallel effort, not a blocker.
15 changes: 15 additions & 0 deletions integrations/nvtx/bridge/Cargo.toml
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
# SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
# SPDX-License-Identifier: Apache-2.0

[package]
name = "nvtx-bridge"
version.workspace = true
edition.workspace = true
publish.workspace = true

[dependencies]
# The bridge is a single adapter type. It needs the NVTX event vocabulary, the
# Quent `EntityEvent` trait it implements, and serde for the transparent derive.
nvtx-events = { path = "../events" }
quent-events = { path = "../../../crates/events" }
serde = { workspace = true }
33 changes: 33 additions & 0 deletions integrations/nvtx/bridge/src/lib.rs
Original file line number Diff line number Diff line change
@@ -0,0 +1,33 @@
// SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
// SPDX-License-Identifier: Apache-2.0

//! The thin bridge that lets captured NVTX events flow through Quent's typed
//! event pipeline.
//!
//! It is one adapter: [`NvtxEventEntity`], a newtype over [`NvtxEvent`]
//! implementing Quent's [`EntityEvent`]. The orphan rule forbids that impl in
//! either of *their* crates — the events crate stays Quent-agnostic for
//! upstreaming — so it lives here.
//!
//! See `integrations/nvtx/example` for a complete, runnable capture.

use nvtx_events::NvtxEvent;
use quent_events::EntityEvent;
use serde::{Deserialize, Serialize};

/// A `#[serde(transparent)]` newtype over [`NvtxEvent`] implementing
/// [`EntityEvent`], naming the `"NvtxEvent"` entity stream. Transparent, so its
/// serialized form is identical to a bare [`NvtxEvent`].
#[derive(Debug, Clone, PartialEq, Deserialize, Serialize)]
#[serde(transparent)]
pub struct NvtxEventEntity(pub NvtxEvent);

impl EntityEvent for NvtxEventEntity {
const NAME: &'static str = "NvtxEvent";
}

impl From<NvtxEvent> for NvtxEventEntity {
fn from(event: NvtxEvent) -> Self {
Self(event)
}
}
29 changes: 29 additions & 0 deletions integrations/nvtx/example/Cargo.toml
Original file line number Diff line number Diff line change
@@ -0,0 +1,29 @@
# SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
# SPDX-License-Identifier: Apache-2.0

[package]
name = "nvtx-example"
version.workspace = true
edition.workspace = true
publish = false

[[bin]]
name = "nvtx_capture"
path = "src/main.rs"

[dependencies]
nvtx-injection = { path = "../injection", features = ["static-injection"] }
nvtx-bridge = { path = "../bridge" }
# `io-callback` selects the callback exporter used to print (bin) and collect
# (test) captured events.
quent-instrumentation = { path = "../../../crates/instrumentation", features = [
"io-callback",
] }
uuid = { workspace = true }
# The NVTX producer: a header-only Rust binding that makes the NVTX calls the
# injection layer captures. Self-contained (vendors the NVTX v3 headers).
nvtx = "1.3"

[dev-dependencies]
# The test matches on `NvtxEvent` variants to assert coverage.
nvtx-events = { path = "../events" }
Loading