Add EP Device Compatibility APIs #26922
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
xhan65
force-pushed
the
user/xiha/compat_api
branch
2 times, most recently
from
ad78f56 to
80f9953
Compare
Contributor
|
/azp run Linux QNN CI Pipeline, Win_TRT_Minimal_CUDA_Test_CI, Windows ARM64 QNN CI Pipeline, Windows GPU Doc Gen CI Pipeline |
|
Azure Pipelines successfully started running 4 pipeline(s). |
edgchen1
reviewed
Jan 10, 2026
![github-advanced-security[bot]](https://avatars.githubusercontent.com/in/57789?s=60&v=4){kind=small}
edgchen1
reviewed
Jan 13, 2026
Contributor
|
/azp run Linux QNN CI Pipeline, Win_TRT_Minimal_CUDA_Test_CI, Windows ARM64 QNN CI Pipeline, Windows GPU Doc Gen CI Pipeline |
|
Azure Pipelines successfully started running 4 pipeline(s). |
adrianlizarraga
previously approved these changes
Jan 13, 2026
javier-intel
approved these changes
Jan 13, 2026
edgchen1
previously approved these changes
Jan 13, 2026
adrastogi
previously approved these changes
Jan 13, 2026
Contributor
Author
|
/azp run Linux QNN CI Pipeline, Win_TRT_Minimal_CUDA_Test_CI, Windows ARM64 QNN CI Pipeline, Windows GPU Doc Gen CI Pipeline |
|
Commenter does not have sufficient privileges for PR 26922 in repo microsoft/onnxruntime |
Contributor
|
/azp run Linux QNN CI Pipeline, Win_TRT_Minimal_CUDA_Test_CI, Windows ARM64 QNN CI Pipeline, Windows GPU Doc Gen CI Pipeline |
|
Azure Pipelines successfully started running 4 pipeline(s). |
…ons APIs and tests
xhan65
dismissed stale reviews from adrastogi, edgchen1, and adrianlizarraga
via
62fdfe3
xhan65
force-pushed
the
user/xiha/compat_api
branch
from
9de6ce8 to
62fdfe3
Compare
adrianlizarraga
approved these changes
Jan 14, 2026
Contributor
|
/azp run Linux QNN CI Pipeline, Win_TRT_Minimal_CUDA_Test_CI, Windows ARM64 QNN CI Pipeline, Windows GPU Doc Gen CI Pipeline |
|
Azure Pipelines successfully started running 4 pipeline(s). |
alex-spacemit
pushed a commit
to spacemit-com/onnxruntime
that referenced
this pull request
Jan 20, 2026
### Description <!-- Describe your changes. --> This PR adds new C API for querying hardware device and execution provider (EP) compatibility/incompatibility reasons. The API allows applications to determine why a specific EP may not be compatible with a given hardware device before attempting to use it. The EP implementor decides how device compatibility is defined for their stack and communicates this via the possible states in the OrtDeviceEpIncompatibilityDetails struct. New API Functions - GetHardwareDeviceEPIncompatibilityReasons - Query incompatibility reasons between an EP and a hardware device - DeviceEpIncompatibilityDetails_GetReasonsBitmask - Get a bitmask of incompatibility reasons - DeviceEpIncompatibilityDetails_GetErrorCode - Get an EP-specific error code - DeviceEpIncompatibilityDetails_GetNotes - Get optional human-readable notes about incompatibility - ReleaseDeviceEpIncompatibilityDetails - Release the details object New Types - OrtDeviceEpIncompatibilityDetails - Opaque type holding incompatibility information - OrtDeviceEpIncompatibilityReason - Enum defining standard incompatibility reason flags Testing - Added unit tests in hardware_device_compatibility_test.cc for the new C API functions - Added plugin EP compatibility tests in test_execution.cc . Tests verify both compatible (CPU device with CPU-supporting EP) and incompatible (GPU device with CPU-only EP) scenarios - Updated example plugin EP to implement the compatibility checking interface ### Motivation and Context Applications using ONNX Runtime need a way to understand why an execution provider cannot run on a particular hardware device. This is especially important for: - User feedback - Providing meaningful error messages when hardware is incompatible - EP selection - Choosing the best available EP based on compatibility - Diagnostics - Understanding driver/dependency requirements Previously, there was no standardized way to query this information. EPs would simply fail at session creation/model load/inference time without providing actionable feedback.
|
Determining the optimal EP for execution requires inputs from the application, the OS, and the EP driver. To establish priority, the application must provide the OS with a deadline. While the OS retains exclusive control over thread scheduling, the hardware manages frequency and thermal constraints. Ultimately, the API should be simple yet provide enough data for each component to make the correct decision. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
6 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description
This PR adds new C API for querying hardware device and execution provider (EP) compatibility/incompatibility reasons. The API allows applications to determine why a specific EP may not be compatible with a given hardware device before attempting to use it. The EP implementor decides how device compatibility is defined for their stack and communicates this via the possible states in the OrtDeviceEpIncompatibilityDetails struct.
New API Functions
New Types
Testing
. Tests verify both compatible (CPU device with CPU-supporting EP) and incompatible (GPU device with CPU-only EP) scenarios
Motivation and Context
Applications using ONNX Runtime need a way to understand why an execution provider cannot run on a particular hardware device. This is especially important for:
Previously, there was no standardized way to query this information. EPs would simply fail at session creation/model load/inference time without providing actionable feedback.