Skip to content

Conversation

@nieubank
Copy link
Contributor

@nieubank nieubank commented May 8, 2025

Description

Enables automatic selection of NvTensorRTRTX EP for PREFER_GPU policy similar to #24629

Sample code from unit test:

  auto env = Ort::Env();

  env.RegisterExecutionProviderLibrary(kNvTensorRTRTXExecutionProvider, ORT_TSTR("onnxruntime_providers_nv_tensorrt_rtx.dll"));

  Ort::SessionOptions so;
  so.SetEpSelectionPolicy(OrtExecutionProviderDevicePolicy_PREFER_GPU);
  Ort::Session session_object(env, model_name_ctx.c_str(), so);
  EXPECT_TRUE(SessionHasEp(session_object, kNvTensorRTRTXExecutionProvider));

  env.UnregisterExecutionProviderLibrary(kNvTensorRTRTXExecutionProvider);
Note: Google Test filter = *NvExecutionProviderTest*AutoEp_PreferGpu*
[==========] Running 1 test from 1 test suite.
[----------] Global test environment set-up.
[----------] 1 test from NvExecutionProviderTest
[ RUN      ] NvExecutionProviderTest.AutoEp_PreferGpu
[       OK ] NvExecutionProviderTest.AutoEp_PreferGpu (1136 ms)
[----------] 1 test from NvExecutionProviderTest (1137 ms total)

[----------] Global test environment tear-down
[==========] 1 test from 1 test suite ran. (1138 ms total)
[  PASSED  ] 1 test.

Motivation and Context

A recent feature allows ORT to automatically select an EP according to policies set by the user (e.g., prefer npu or prefer gpu). This PR allows NvTensorRTRTX EP to be potentially selected when the user sets the PREFER_GPU policy.

@adrianlizarraga adrianlizarraga requested a review from skottmckay May 8, 2025 16:28
@nieubank nieubank marked this pull request as ready for review May 8, 2025 16:59
adrianlizarraga
adrianlizarraga previously approved these changes May 8, 2025
@nieubank nieubank force-pushed the nieubank/nvrtx-support-autoep branch from 4edbfe6 to 4d84acb Compare May 21, 2025 17:59
@nieubank
Copy link
Contributor Author

@adrianlizarraga - I've updated the PR on the latest main with the change we took for Windows ML to fix multi-device path. When you get a chance, please review and let me know if there is anything else that needs to be done to unblock merging. Thanks!

@snnn snnn closed this Jul 3, 2025
@nieubank nieubank deleted the nieubank/nvrtx-support-autoep branch January 10, 2026 00:10
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants