Skip to content

Cyclic buffer telemetry exporter moved to use plug-in infrastructure - #1088

Merged
ovidiusm merged 9 commits into
ai-dynamo:mainfrom
alexanderbilk:cyclic
Dec 10, 2025
Merged

ovidiusm merged 9 commits into
ai-dynamo:mainfrom
alexanderbilk:cyclic

Conversation

@alexanderbilk

@alexanderbilk alexanderbilk commented Dec 8, 2025

Copy link
Copy Markdown
Contributor

Introducing shared memory cyclic buffer telemetry exporter to work as static built-in plugin.
This telemetry exporter is a default choice in case no other exporter is specified

@alexanderbilk
alexanderbilk requested a review from a team as a code owner December 8, 2025 00:02
@copy-pr-bot

copy-pr-bot Bot commented Dec 8, 2025

Copy link
Copy Markdown

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

@github-actions

github-actions Bot commented Dec 8, 2025

Copy link
Copy Markdown

👋 Hi alexanderbilk! Thank you for contributing to ai-dynamo/nixl.

Your PR reviewers will review your contribution then trigger the CI to test your changes.

🚀

Signed-off-by: Aleksandr Bilkovskii <alexanderb@nvidia.com>
Comment thread src/core/telemetry/buffer_exporter.cpp Outdated
Comment thread src/core/telemetry/buffer_exporter.cpp Outdated
Comment thread src/core/telemetry/buffer_exporter.cpp Outdated
Comment thread src/core/telemetry/buffer_exporter.h Outdated
Comment thread src/core/telemetry/buffer_plugin.cpp
Comment thread src/core/telemetry/telemetry.cpp Outdated
Comment thread src/core/telemetry/telemetry.cpp Outdated
Comment thread src/core/telemetry/buffer_exporter.cpp Outdated
Comment thread src/core/telemetry/telemetry.cpp Outdated
Signed-off-by: Aleksandr Bilkovskii <alexanderb@nvidia.com>
Comment thread src/core/nixl_agent.cpp
Comment thread src/api/cpp/telemetry/telemetry_plugin.h Outdated
@rakhmets

rakhmets commented Dec 8, 2025

Copy link
Copy Markdown
Contributor

Please add a description to the PR, mentioning that the telemetry exporter based on shared memory buffer was added as a static plugin, and is also used as the default telemetry plugin.

Comment thread src/core/telemetry/telemetry.cpp Outdated
Comment thread src/core/telemetry/buffer_exporter.cpp Outdated
Comment thread src/core/telemetry/buffer_exporter.h Outdated
Comment thread src/core/telemetry/telemetry.cpp Outdated
Signed-off-by: Aleksandr Bilkovskii <alexanderb@nvidia.com>
Signed-off-by: Aleksandr Bilkovskii <alexanderb@nvidia.com>
Comment thread src/core/telemetry/telemetry.cpp Outdated
Comment thread src/core/telemetry/telemetry.cpp Outdated
Comment thread src/core/telemetry/telemetry.cpp Outdated
Signed-off-by: Aleksandr Bilkovskii <alexanderb@nvidia.com>
ovidiusm
ovidiusm previously approved these changes Dec 9, 2025
@ovidiusm

ovidiusm commented Dec 9, 2025

Copy link
Copy Markdown
Contributor

/build

@ovidiusm

ovidiusm commented Dec 9, 2025

Copy link
Copy Markdown
Contributor

/ok to test bfe3671

@brminich

brminich commented Dec 9, 2025

Copy link
Copy Markdown
Contributor

/ok to test e6e371f

Signed-off-by: Aleksandr Bilkovskii <alexanderb@nvidia.com>
@alexanderbilk
alexanderbilk dismissed stale reviews from rakhmets and brminich via 673bc58 December 9, 2025 18:44
@brminich

brminich commented Dec 9, 2025

Copy link
Copy Markdown
Contributor

/ok to test 673bc58

@brminich

brminich commented Dec 9, 2025

Copy link
Copy Markdown
Contributor

/build

@brminich
brminich enabled auto-merge (squash) December 9, 2025 20:46
brminich
brminich previously approved these changes Dec 9, 2025
…s not set

Signed-off-by: Aleksandr Bilkovskii <alexanderb@nvidia.com>
auto-merge was automatically disabled December 10, 2025 00:08

Head branch was pushed to by a user without write access

@ovidiusm

Copy link
Copy Markdown
Contributor

/build

@ovidiusm

Copy link
Copy Markdown
Contributor

/ok to test c344d6a

@ovidiusm
ovidiusm enabled auto-merge (squash) December 10, 2025 00:15
@ovidiusm
ovidiusm merged commit 60a12a4 into ai-dynamo:main Dec 10, 2025
33 checks passed
iyastreb pushed a commit that referenced this pull request Jul 9, 2026
## What?
Remove the unused `nixlTelemetry::buffer_` member and its unreachable
destructor branch (the `if (buffer_)` flush/reset). Also drop the
now-unneeded `#include "common/cyclic_buffer.h"` from `telemetry.h`, and
add it directly to `telemetry_test.cpp`, which uses `sharedRingBuffer`
and had been relying on the transitive include through `telemetry.h`.

## Why?
The core `buffer_` member was live when telemetry was introduced (#562):
`nixlTelemetry` itself owned the shared-memory ring buffer, pushed
events to it, and flushed on destruction. #1088 moved the BUFFER sink
into the plugin-based `nixlTelemetryBufferExporter` (which owns its own
buffer) but left the core member and destructor branch orphaned. Since
#1088 the core `buffer_` is never assigned, so it is always null and the
destructor branch is dead code. Removing it keeps the core telemetry
owner aligned with the current exporter model and avoids implying a
shutdown flush that does not actually happen.

Behavior-preserving: no change to event ordering, drop accounting,
exporter error handling, BUFFER cyclic-ring behavior, or the periodic
`flushPendingEvents()` task.

## How?
- `src/core/telemetry/telemetry.h`: remove the `buffer_` member and the
`common/cyclic_buffer.h` include.
- `src/core/telemetry/telemetry.cpp`: remove the dead destructor branch;
keep `#include "buffer_exporter.h"` (still needed for `telemetryDirVar`)
and the shutdown ordering (disable task, stop/join pool, cancel timer).
- `test/gtest/telemetry_test.cpp`: include `common/cyclic_buffer.h`
directly.

## Testing
- `ninja -C build` — clean build.
- Full gtest suite: 111/111 passed, including all 17 `telemetryTest`
cases.

<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->

## Summary by CodeRabbit

* **Bug Fixes**
* Improved telemetry shutdown behavior to avoid extra work during
teardown.
* Added more resilient handling when background scheduling cannot be
canceled, helping prevent shutdown-related issues.

<!-- end of auto-generated comment: release notes by coderabbit.ai -->

Signed-off-by: Efraim Eygin <eeygin@nvidia.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants