Skip to content

test/gtest: stop the inert-tracer test depending on NIXL_PLUGIN_DIR - #2092

Merged
ovidiusm merged 1 commit into
ai-dynamo:mainfrom
e-eygin:nix-1710-tracing-inert-test-env-independent
Aug 18, 2026
Merged

ovidiusm merged 1 commit into
ai-dynamo:mainfrom
e-eygin:nix-1710-tracing-inert-test-env-independent

Conversation

@e-eygin

@e-eygin e-eygin commented Aug 14, 2026

Copy link
Copy Markdown
Contributor

What?

Tracing.RequestedBackendWithoutPluginIsInert asked makeTracer for the nvtx backend and expected a null tracer. It now asks for a backend name no plugin can provide, matching what the sibling test Tracing.MakeTracerUnknownBackendReturnsNull already did; the name lives in one kUnloadableBackend constant used by both.

Why?

The old assertion rested on a property of the environment, not of the code: its own comment said "no libtrace_backend_*.so is registered in this unit binary". Whenever a real libtrace_backend_nvtx.so is discoverable, the plugin loads, makeTracer correctly returns a live tracer, and the test fails. That is the case in the dev container, where NIXL_PLUGIN_DIR points at an install tree containing the NVTX trace plugin, and it would be the case in any CI leg that installs NIXL before running the unit suite.

The test has been permanently red locally for weeks and was written off as "pre-existing, unrelated" in the validation notes of three telemetry PRs (#1952, #2054, #2086). Beyond the recurring explanation, a genuine regression in the null-tracer / inert-Span path would have been indistinguishable from the known-red state.

Coverage is unchanged: this test exists to prove that a null tracer leaves call sites on the safe default-constructed Span path, which never required NVTX specifically. Real NVTX behaviour stays covered by the e2e TestTransferTracing tests, which load the actual plugin.

Tracking: NIX-1710.

Rejected alternative, and verification

Clearing NIXL_PLUGIN_DIR inside the test via the existing gtest::ScopedEnv helper does not work reliably: getPluginDir() is read once in the nixlPluginManager constructor and cached in plugin_dirs_ for the process lifetime, so the outcome would depend on whether an earlier test in the binary already touched the plugin manager.

Verified both directions with the container's default NIXL_PLUGIN_DIR, i.e. with no workaround applied:

  • Baseline (this change stashed, rebuilt): exactly one failure, RequestedBackendWithoutPluginIsInert, reporting a live tracer pointer where nullptr was expected.
  • With the change: all 17 Tracing.* tests pass, and the full unit suite is 157 passed / 2 skipped / 3 failed — the three failures being objCrtTestFixture.TransferBelowThreshold and the two ObjClientTests/objParamTestFixture.ReadTransfer params, which need an object-storage endpoint this container does not provide.

Summary by CodeRabbit

  • Tests
    • Improved tracing plugin-loading tests to consistently cover environments where the requested backend is unavailable.
    • Updated inert-tracer coverage to validate behavior when no tracing plugin can be loaded.

Tracing.RequestedBackendWithoutPluginIsInert asked makeTracer for the
"nvtx" backend and expected a null tracer, on the premise that no
libtrace_backend_*.so is registered in the unit binary. That is a
property of the environment rather than of the code under test: when a
real libtrace_backend_nvtx.so is discoverable -- e.g. NIXL_PLUGIN_DIR
pointing at an install tree, as in the dev container, or any CI leg that
installs NIXL before running the unit suite -- the plugin loads,
makeTracer correctly returns a live tracer, and the test fails.

Request a name no plugin can provide instead, as the sibling test
MakeTracerUnknownBackendReturnsNull already does, and lift that name into
one kUnloadableBackend constant so a real backend name cannot creep back
in. The test's subject is unchanged: a null tracer must leave call sites
on the safe default-constructed Span path. Real NVTX behaviour stays
covered by the e2e TestTransferTracing tests, which load the plugin.

Clearing NIXL_PLUGIN_DIR from inside the test was rejected: getPluginDir()
is read once in the nixlPluginManager constructor and cached in
plugin_dirs_ for the process lifetime, so the result would depend on
whether an earlier test already touched the plugin manager.

With the container's default NIXL_PLUGIN_DIR, all 17 Tracing tests now
pass (previously 16, and green only with the variable pointed at an empty
directory); the unit suite drops to 157 passed / 2 skipped / 3 failed,
the remainder being object-storage tests that need an S3 endpoint.

Signed-off-by: Efraim Eygin <eeygin@nvidia.com>
@e-eygin
e-eygin requested a review from a team as a code owner August 14, 2026 14:26
@copy-pr-bot

copy-pr-bot Bot commented Aug 14, 2026

Copy link
Copy Markdown

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

@github-actions

Copy link
Copy Markdown

👋 Hi e-eygin! Thank you for contributing to ai-dynamo/nixl.

Your PR reviewers will review your contribution then trigger the CI to test your changes.

🚀

@coderabbitai

coderabbitai Bot commented Aug 14, 2026

Copy link
Copy Markdown
Contributor

Review Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: ASSERTIVE

Plan: Enterprise

Run ID: f7321b40-b17c-4547-9512-17feef0bc7bd

📥 Commits

Reviewing files that changed from the base of the PR and between 21afd55 and ee80261.

📒 Files selected for processing (1)
  • test/gtest/unit/tracing/tracing_test.cpp

📝 Walkthrough

Walkthrough

The tracing tests add a shared guaranteed-unloadable backend constant. Plugin-loading tests reuse the constant, and the inert-tracer test now covers the unavailable-plugin runtime path instead of requesting nvtx.

Changes

Tracing test stabilization

Layer / File(s) Summary
Guaranteed-unloadable backend test path
test/gtest/unit/tracing/tracing_test.cpp
Defines kUnloadableBackend, replaces duplicated unknown-backend literals, and updates the inert-tracer test to use the unavailable-plugin path.

Estimated code review effort: 2 (Simple) | ~10 minutes

Merge Risk: ⚪ Minimal · up to ee802

The change makes the tracing unit test independent of the runtime plugin environment without changing production behavior or coverage; no actionable merge-blocking risk remains after normal checks and review.

Suggested reviewers: aranadive

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Description check ✅ Passed The description includes complete What and Why sections and provides optional design and verification details in How-related content.
Title check ✅ Passed The title clearly and concisely describes removing the inert-tracer test's dependency on NIXL_PLUGIN_DIR.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Comment @coderabbitai help to get the list of available commands.

@e-eygin e-eygin self-assigned this Aug 14, 2026
@e-eygin

e-eygin commented Aug 14, 2026

Copy link
Copy Markdown
Contributor Author

/build

@e-eygin

e-eygin commented Aug 14, 2026

Copy link
Copy Markdown
Contributor Author

/ok to test ee80261

@ovidiusm
ovidiusm merged commit 4e50bc3 into ai-dynamo:main Aug 18, 2026
20 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants