Skip to content

[NV TensorRT RTX EP] Fix external tensorrt_plugins load path #26814

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
yuslepukhin merged 4 commits into from
Jan 15, 2026
Merged

[NV TensorRT RTX EP] Fix external tensorrt_plugins load path #26814

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
fc2e203
Fix TRT RTX plugin load path
keshavv27 Dec 16, 2025
ab26c6f
Add header files, fix error output
keshavv27 Dec 16, 2025
fe607b4
fix linting
keshavv27 Dec 17, 2025
aedf409
use pathstring ctor, abstract fn and platform defs
keshavv27 Dec 19, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
17 changes: 3 additions & 14 deletions onnxruntime/core/providers/nv_tensorrt_rtx/nv_execution_provider_custom_ops.cc
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -7,18 +7,7 @@
#include "core/framework/provider_options.h"
#include "nv_execution_provider_custom_ops.h"
#include "nv_execution_provider.h"

// The filename extension for a shared library is different per platform
#ifdef _WIN32
#define LIBRARY_PREFIX
#define LIBRARY_EXTENSION ORT_TSTR(".dll")
#elif defined(__APPLE__)
#define LIBRARY_PREFIX "lib"
#define LIBRARY_EXTENSION ".dylib"
#else
#define LIBRARY_PREFIX "lib"
#define LIBRARY_EXTENSION ".so"
#endif
#include "nv_platform_utils.h"

Check warning on line 10 in onnxruntime/core/providers/nv_tensorrt_rtx/nv_execution_provider_custom_ops.cc

View workflow job for this annotation

GitHub Actions / Optional Lint C++ 

[cpplint] reported by reviewdog 🐶
 Include the directory when naming header files  [build/include_subdir] [4]

Raw Output:
onnxruntime/core/providers/nv_tensorrt_rtx/nv_execution_provider_custom_ops.cc:10:  Include the directory when naming header files  [build/include_subdir] [4]

namespace onnxruntime {
extern TensorrtLogger& GetTensorrtLogger(bool verbose);
Expand Down Expand Up @@ -76,14 +65,14 @@
// This library contains GroupQueryAttention and RotaryEmbedding plugins for transformer models
try {
const auto& env = onnxruntime::GetDefaultEnv();
auto external_plugin_path = env.GetRuntimePath() +
auto external_plugin_path = GetEPLibraryDirectory() +
PathString(LIBRARY_PREFIX ORT_TSTR("tensorrt_plugins") LIBRARY_EXTENSION);
void* external_plugin_handle = nullptr;
auto status = env.LoadDynamicLibrary(external_plugin_path, false, &external_plugin_handle);
if (status.IsOK()) {
LOGS_DEFAULT(INFO) << "[NvTensorRTRTX EP] External plugins loaded: tensorrt_plugins (GQA + RotaryEmbedding)";
} else {
LOGS_DEFAULT(VERBOSE) << "[NvTensorRTRTX EP] tensorrt_plugins library not found in runtime path (optional)";
LOGS_DEFAULT(VERBOSE) << "[NvTensorRTRTX EP] tensorrt_plugins library not found in EP library path (optional)";
}
} catch (const std::exception& e) {
LOGS_DEFAULT(VERBOSE) << "[NvTensorRTRTX EP] tensorrt_plugins library not available: " << e.what();
Expand Down
67 changes: 67 additions & 0 deletions onnxruntime/core/providers/nv_tensorrt_rtx/nv_platform_utils.h
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,67 @@
// Copyright (c) Microsoft Corporation. All rights reserved.
// SPDX-FileCopyrightText: Copyright (c) 2025 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
// Licensed under the MIT License.
#pragma once

#include <string>
#include "core/common/path_string.h"

#ifdef _WIN32
#include <windows.h>
#else
#include <dlfcn.h>
#endif

// The filename extension for a shared library is different per platform
#ifdef _WIN32
#define LIBRARY_PREFIX
#define LIBRARY_EXTENSION ORT_TSTR(".dll")
#elif defined(__APPLE__)
#define LIBRARY_PREFIX "lib"
#define LIBRARY_EXTENSION ".dylib"
#else
#define LIBRARY_PREFIX "lib"
#define LIBRARY_EXTENSION ".so"
#endif

namespace onnxruntime {
inline PathString GetEPLibraryDirectory() {
#ifdef _WIN32
HMODULE hModule = NULL;
// Get handle to the DLL executing this code
if (!GetModuleHandleExW(GET_MODULE_HANDLE_EX_FLAG_FROM_ADDRESS |
GET_MODULE_HANDLE_EX_FLAG_UNCHANGED_REFCOUNT,
reinterpret_cast<LPCWSTR>(GetEPLibraryDirectory),
&hModule)) {
return PathString();
}

wchar_t buffer[MAX_PATH];
DWORD len = GetModuleFileNameW(hModule, buffer, MAX_PATH);
if (len == 0 || len >= MAX_PATH) {
return PathString();
}

std::wstring path(buffer);
size_t lastSlash = path.find_last_of(L"\\/");
if (lastSlash != std::wstring::npos) {
return PathString(path.substr(0, lastSlash + 1));
}
return PathString();
#else
// Linux and other Unix-like platforms
Dl_info dl_info;

if (dladdr((void*)&GetEPLibraryDirectory, &dl_info) == 0 || dl_info.dli_fname == nullptr) {

Check warning on line 55 in onnxruntime/core/providers/nv_tensorrt_rtx/nv_platform_utils.h

View workflow job for this annotation

GitHub Actions / Optional Lint C++ 

[cpplint] reported by reviewdog 🐶
 Using C-style cast.  Use reinterpret_cast<void*>(...) instead  [readability/casting] [4]

Raw Output:
onnxruntime/core/providers/nv_tensorrt_rtx/nv_platform_utils.h:55:  Using C-style cast.  Use reinterpret_cast<void*>(...) instead  [readability/casting] [4]
return PathString();
}

std::string so_path(dl_info.dli_fname);
size_t last_slash = so_path.find_last_of('/');
if (last_slash != std::string::npos) {
return PathString(so_path.substr(0, last_slash + 1));
}
return PathString();
#endif
}
} // namespace onnxruntime
Loading