Skip to content

[NV TensorRT RTX EP] Fix external tensorrt_plugins load path #26814

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
yuslepukhin merged 4 commits into from
Jan 15, 2026
Merged

[NV TensorRT RTX EP] Fix external tensorrt_plugins load path #26814

Changes from 3 commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
fc2e203
Fix TRT RTX plugin load path
keshavv27 Dec 16, 2025
ab26c6f
Add header files, fix error output
keshavv27 Dec 16, 2025
fe607b4
fix linting
keshavv27 Dec 17, 2025
aedf409
use pathstring ctor, abstract fn and platform defs
keshavv27 Dec 19, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
52 changes: 50 additions & 2 deletions onnxruntime/core/providers/nv_tensorrt_rtx/nv_execution_provider_custom_ops.cc
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,12 @@
#include "nv_execution_provider_custom_ops.h"
#include "nv_execution_provider.h"

#ifdef _WIN32
#include <windows.h>
Comment thread
yuslepukhin marked this conversation as resolved.
Outdated
#else
#include <dlfcn.h>
#endif

// The filename extension for a shared library is different per platform
#ifdef _WIN32
#define LIBRARY_PREFIX
Expand All @@ -23,6 +29,48 @@
namespace onnxruntime {
extern TensorrtLogger& GetTensorrtLogger(bool verbose);

/// @brief Gets the directory path of the EP library that contains the current function
/// @return The directory path containing the EP library, or an empty string if the path cannot be determined.
static PathString GetEPLibraryDirectory() {
#ifdef _WIN32
HMODULE hModule = NULL;
// Get handle to the DLL executing this code
if (!GetModuleHandleExW(GET_MODULE_HANDLE_EX_FLAG_FROM_ADDRESS |
GET_MODULE_HANDLE_EX_FLAG_UNCHANGED_REFCOUNT,
reinterpret_cast<LPCWSTR>(&GetEPLibraryDirectory),

Copilot AI Dec 18, 2025

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The reinterpret_cast to LPCWSTR is incorrect. GetModuleHandleExW expects the lpModuleName parameter to be a string (LPCWSTR) when GET_MODULE_HANDLE_EX_FLAG_FROM_ADDRESS is not used, but when this flag is set, it expects a raw address (LPCVOID). The address of the function should be cast to LPCWSTR without going through the pointer, i.e., reinterpret_cast(GetEPLibraryDirectory) should be used, not reinterpret_cast(&GetEPLibraryDirectory).

Copilot uses AI. Check for mistakes.
&hModule)) {
return PathString(L"");
Comment thread
yuslepukhin marked this conversation as resolved.
Outdated
}

wchar_t buffer[MAX_PATH];
DWORD len = GetModuleFileNameW(hModule, buffer, MAX_PATH);
if (len == 0 || len >= MAX_PATH) {
return PathString(L"");
Comment thread
yuslepukhin marked this conversation as resolved.
Outdated
}

std::wstring path(buffer);
size_t lastSlash = path.find_last_of(L"\\/");
if (lastSlash != std::wstring::npos) {
return PathString(path.substr(0, lastSlash + 1));
}
return PathString(path);

Copilot AI Dec 18, 2025

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

On line 56, when lastSlash is not found (npos), the function returns the entire path string. This would be incorrect because if there's no slash in the path, the function should return an empty string instead of the full path, since the purpose is to return the directory containing the library, not the library path itself. Consider returning an empty PathString when lastSlash == std::wstring::npos.

Copilot uses AI. Check for mistakes.
#else
// Linux and other Unix-like platforms
Dl_info dl_info;

if (dladdr((void*)&GetEPLibraryDirectory, &dl_info) == 0 || dl_info.dli_fname == nullptr) {

Check warning on line 61 in onnxruntime/core/providers/nv_tensorrt_rtx/nv_execution_provider_custom_ops.cc

View workflow job for this annotation

GitHub Actions / Optional Lint C++ 

[cpplint] reported by reviewdog 🐶
 Using C-style cast.  Use reinterpret_cast<void*>(...) instead  [readability/casting] [4]

Raw Output:
onnxruntime/core/providers/nv_tensorrt_rtx/nv_execution_provider_custom_ops.cc:61:  Using C-style cast.  Use reinterpret_cast<void*>(...) instead  [readability/casting] [4]
return PathString("");
}

std::string so_path(dl_info.dli_fname);
size_t last_slash = so_path.find_last_of('/');
if (last_slash != std::string::npos) {
return PathString(so_path.substr(0, last_slash + 1));
}
return PathString(so_path);

Copilot AI Dec 18, 2025

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

On line 70, when last_slash is not found (npos), the function returns the entire path string. This would be incorrect because if there's no slash in the path, the function should return an empty string instead of the full path, since the purpose is to return the directory containing the library, not the library path itself. Consider returning an empty PathString when last_slash == std::string::npos.

Copilot uses AI. Check for mistakes.
#endif
}

/*
* Create custom op domain list for TRT plugins.
*
Expand Down Expand Up @@ -76,14 +124,14 @@
// This library contains GroupQueryAttention and RotaryEmbedding plugins for transformer models
try {
const auto& env = onnxruntime::GetDefaultEnv();
auto external_plugin_path = env.GetRuntimePath() +
auto external_plugin_path = GetEPLibraryDirectory() +
PathString(LIBRARY_PREFIX ORT_TSTR("tensorrt_plugins") LIBRARY_EXTENSION);
void* external_plugin_handle = nullptr;
auto status = env.LoadDynamicLibrary(external_plugin_path, false, &external_plugin_handle);
if (status.IsOK()) {
LOGS_DEFAULT(INFO) << "[NvTensorRTRTX EP] External plugins loaded: tensorrt_plugins (GQA + RotaryEmbedding)";
} else {
LOGS_DEFAULT(VERBOSE) << "[NvTensorRTRTX EP] tensorrt_plugins library not found in runtime path (optional)";
LOGS_DEFAULT(VERBOSE) << "[NvTensorRTRTX EP] tensorrt_plugins library not found in EP library path (optional)";
}
} catch (const std::exception& e) {
LOGS_DEFAULT(VERBOSE) << "[NvTensorRTRTX EP] tensorrt_plugins library not available: " << e.what();
Expand Down
Loading