Add OrtExternalResourceImporter API for D3D12 shared resource import #26828

nieubank · 2025-12-18T21:04:26Z

Description

Introduces the OrtExternalResourceImporter API enabling execution providers to import D3D12 shared resources and timeline fences for zero-copy GPU-to-GPU data sharing with ORT inference.

Public API additions:

OrtExternalResourceImporter capability object
OrtExternalMemoryHandle for imported D3D12 allocations
OrtExternalSemaphoreHandle for imported D3D12 timeline fences
SessionGetEpDeviceForOutputs to query output EP device placement

EP Plugin API:

OrtExternalResourceImporterImpl interface for EP implementations
OrtEpFactory::CreateExternalResourceImporterForDevice extension

Design:

No GPU virtual addresses in public API
EP-agnostic design allows any EP to implement import
Capability discovery with explicit ORT_NOT_IMPLEMENTED
Follows existing patterns (Allocator, DataTransfer, SyncStream)

Includes example_plugin_ep mock implementation and autoep tests.

Motivation and Context

#26821

Introduces the OrtExternalResourceImporter API enabling execution providers to import D3D12 shared resources and timeline fences for zero-copy GPU-to-GPU data sharing with ORT inference. Public API additions: - OrtExternalResourceImporter capability object - OrtExternalMemoryHandle for imported D3D12 allocations - OrtExternalSemaphoreHandle for imported D3D12 timeline fences - SessionGetEpDeviceForOutputs to query output EP device placement - RunOptions_SetSyncStream to associate sync stream for async execution EP Plugin API: - OrtExternalResourceImporterImpl interface for EP implementations - OrtEpFactory::CreateExternalResourceImporterForDevice extension Design: - No GPU virtual addresses in public API - EP-agnostic design allows any EP to implement import - Capability discovery with explicit ORT_NOT_IMPLEMENTED - Follows existing patterns (Allocator, DataTransfer, SyncStream) Includes example_plugin_ep mock implementation and autoep tests.

include/onnxruntime/core/session/onnxruntime_c_api.h

include/onnxruntime/core/framework/run_options.h

include/onnxruntime/core/session/onnxruntime_ep_c_api.h

onnxruntime/core/session/onnxruntime_c_api.cc

onnxruntime/core/session/plugin_ep/ep_factory_provider_bridge.h

- Deleted the sync_stream member from OrtRunOptions structure. - Removed the RunOptions_SetSyncStream API and its implementation. - Updated related C++ API and example implementations to reflect the removal of sync stream functionality. - Adjusted tests to remove references to RunOptions_SetSyncStream. - Introduced new structures for external memory and semaphore handles to improve resource management. - Ensured backward compatibility by checking EP version support for external resource import.

include/onnxruntime/core/session/onnxruntime_ep_c_api.h

…l resource structs

onnxruntime/test/autoep/library/example_plugin_ep/ep_external_resource_importer.h

yuslepukhin

…lMemoryHandle

nieubank · 2026-01-05T17:58:18Z

@skottmckay / @yuslepukhin - any concerns with getting this in this week prior to the 1.24 snap?

I think this provides a sufficient baseline for achieving the core of the asks around shared resources and we continue iterating on specific EP implementations with this foundation.

yuslepukhin · 2026-01-05T18:11:54Z

Looks great to me

skottmckay · 2026-01-06T22:12:48Z

@gaugarg-nv and @gedoensmax can you please take a look and let us know if there are any gaps/issues with this approach?

- Added `ep_interop_api.h` to define the Interop API for external resource importers. - Implemented functions for creating and managing external resource importers, including memory and semaphore import capabilities. - Updated `onnxruntime_c_api.cc` to integrate the new Interop API, replacing previous external resource importer implementations. - Modified `ort_apis.h` to declare the new Interop API functions. - Refactored tests in `test_external_resource_importer.cc` to utilize the new Interop API for external resource importer operations.

Resolved API conflicts by placing KernelInfo APIs before Interop APIs

onnxruntime/core/session/ep_interop_api.cc

- Return ORT_NOT_IMPLEMENTED status instead of nullptr when EP doesn't support external resource import - Rename ep_interop_api.{cc,h} to interop_api.{cc,h} to match the generic OrtInteropApi naming - Update documentation to reflect the new error handling behavior

…eExternalResourceImporterForDevice Capability discovery APIs should return success with nullptr output when a feature is unsupported, rather than an error status. This allows simple "if (out != nullptr)" checks without needing to distinguish ORT_NOT_IMPLEMENTED from real errors. - Update tests to assert status is nullptr and skip when importer is nullptr

include/onnxruntime/core/session/onnxruntime_c_api.h

onnxruntime/test/autoep/library/example_plugin_ep/ep_external_resource_importer.cc

onnxruntime/test/autoep/library/example_plugin_ep/ep_factory.cc

onnxruntime/test/autoep/test_external_resource_importer.cc

…ted comments

onnxruntime/test/autoep/library/example_plugin_ep/ep_external_resource_importer.h

gedoensmax · 2026-01-08T20:08:44Z

@skottmckay thanks for tagging, this looks good to me. One question i would have is if we could handle e.g. overallocation of allocation with a callback could work. Usually one can simply bin a memory info and rely on ORT to handle the correct allocation size. With this approach it will require preallocation and importing the memory beforehand with the correct shape i assume.
I think this is an edge case just wanted to hear your thoughts.

nieubank · 2026-01-08T22:00:07Z

@skottmckay thanks for tagging, this looks good to me. One question i would have is if we could handle e.g. overallocation of allocation with a callback could work. Usually one can simply bin a memory info and rely on ORT to handle the correct allocation size. With this approach it will require preallocation and importing the memory beforehand with the correct shape i assume. I think this is an edge case just wanted to hear your thoughts.

Right, good callout, the current design is intentionally kept simple and focused on the core import-existing-memory scenario for basic zero-copy interop.

The dual offset pattern OrtExternalMemoryDescriptor::offset_bytes + OrtExternalTensorDescriptor::offset_bytes does allow importing a larger buffer and carving out regions, but I agree it's not quite the same as ORT-driven binning strategies.

This is definitely something we can build on top of in the future, through allocator callbacks or other extensions.

skottmckay

…icrosoft#26828)

The OrtExternalResourceImporter types added in #26828 were wrapped in a Doxygen group block, but this section of the file is already within an existing Global group scope (opened at line 289, closed at end of file). This caused Doxygen to fail with: error: Refusing to add group Global to itself Remove the redundant group markers since the new types correctly inherit the existing Global group context. Verified locally with Doxygen 1.16.1.

Fixes multiple Doxygen errors introduced by recent API additions that cause the nightly documentation build to fail with WARN_AS_ERROR=YES. Root causes by PR: PR #26828 (c54be3c - OrtExternalResourceImporter API for D3D12): - Incorrect \param names for ORT_CLASS_RELEASE macros: the macro expands to use 'input' as the parameter name, but docs used 'importer' and 'handle' PR #26927 (1ed8fd9 - Control flow kernels API): - Used ::CreateLoopKernel() and ::CreateScanKernel() syntax which Doxygen cannot resolve as links PR #26971 (3874516 - CreateEnvWithOptions API): - Used ::SetSessionLogSeverityLevel() and ::RunOptionsSetRunLogSeverityLevel() syntax which Doxygen cannot resolve - Used <ep_name> in documentation which Doxygen interprets as HTML tag - Used incorrect \param[out] name 'out' instead of 'config_entries' Fixes: - Qualify method references with struct name (e.g., OrtApi::Method) - Escape angle brackets as \<ep_name\> - Use 'input' as param name for ORT_CLASS_RELEASE documentation - Use actual param name 'config_entries' for GetEnvConfigEntries Verified locally with Doxygen 1.9.8 (matches CI configuration).

# Fix Doxygen documentation build errors from recent PRs Fixes multiple Doxygen errors introduced by recent API additions that cause the nightly documentation build to fail (`WARN_AS_ERROR=YES`). ## Root Cause Analysis | Error | File | Line | Introduced By | Commit | Fix | |-------|------|------|---------------|--------|-----| | Duplicate `\addtogroup Global` | onnxruntime_c_api.h | 973 | PR #26828 - OrtExternalResourceImporter API | c54be3c | Remove redundant group markers | | Unresolved `::SetSessionLogSeverityLevel()` | onnxruntime_c_api.h | 1065 | PR #26971 - CreateEnvWithOptions API | 3874516 | Use `OrtApi::SetSessionLogSeverityLevel` | | Unresolved `::RunOptionsSetRunLogSeverityLevel()` | onnxruntime_c_api.h | 1066 | PR #26971 - CreateEnvWithOptions API | 3874516 | Use `OrtApi::RunOptionsSetRunLogSeverityLevel` | | `<ep_name>` interpreted as HTML | onnxruntime_c_api.h | 1119 | PR #26971 - CreateEnvWithOptions API | 3874516 | Escape as `\<ep_name\>` | | `\param[in] importer` not found | onnxruntime_c_api.h | 7982 | PR #26828 - OrtExternalResourceImporter API | c54be3c | Use `\param[in] input` (macro expands to `input`) | | `\param[in] handle` not found | onnxruntime_c_api.h | 8025 | PR #26828 - OrtExternalResourceImporter API | c54be3c | Use `\param[in] input` | | `\param[in] handle` not found | onnxruntime_c_api.h | 8091 | PR #26828 - OrtExternalResourceImporter API | c54be3c | Use `\param[in] input` | | Unresolved `::CreateLoopKernel()` | onnxruntime_ep_c_api.h | 667 | PR #26927 - Control flow kernels API | 1ed8fd9 | Use `OrtEpApi::CreateLoopKernel` | | Unresolved `::CreateScanKernel()` | onnxruntime_ep_c_api.h | 710 | PR #26927 - Control flow kernels API | 1ed8fd9 | Use `OrtEpApi::CreateScanKernel` | | `<ep_name>` interpreted as HTML | onnxruntime_ep_c_api.h | 1434 | PR #26971 - CreateEnvWithOptions API | 3874516 | Escape as `\<ep_name\>` | | `\param[out] out` not found | onnxruntime_ep_c_api.h | 1440 | PR #26971 - CreateEnvWithOptions API | 3874516 | Use `\param[out] config_entries` | ## Summary by PR | PR | Issues | |----|--------| | **#26828** (c54be3c) - OrtExternalResourceImporter API for D3D12 | Duplicate Doxygen group, incorrect `\param` names for `ORT_CLASS_RELEASE` macros | | **#26927** (1ed8fd9) - Control flow kernels API | `::Method()` syntax unresolvable by Doxygen | | **#26971** (3874516) - CreateEnvWithOptions API | `::Method()` syntax, `<ep_name>` HTML interpretation, incorrect param name | ## Technical Details ### `ORT_CLASS_RELEASE` Macro Issue The `ORT_CLASS_RELEASE(X)` macro at line 164 expands to: ```cpp void(ORT_API_CALL * Release##X)(_Frees_ptr_opt_ Ort##X * input) ``` The parameter is always named `input`, but the documentation in PR #26828 used semantic names like `importer` and `handle`. Doxygen validates `\param` names against actual parameter names in the expanded code. ### Doxygen Link Resolution Doxygen 1.9.8 cannot resolve `::MethodName()` as a link to a method. The correct syntax is to qualify with the struct name: `OrtApi::MethodName`. ## Testing Verified locally with Doxygen 1.9.8 (matches CI configuration).

# Fix Doxygen documentation build errors from recent PRs Fixes multiple Doxygen errors introduced by recent API additions that cause the nightly documentation build to fail (`WARN_AS_ERROR=YES`). ## Root Cause Analysis | Error | File | Line | Introduced By | Commit | Fix | |-------|------|------|---------------|--------|-----| | Duplicate `\addtogroup Global` | onnxruntime_c_api.h | 973 | PR #26828 - OrtExternalResourceImporter API | c54be3c | Remove redundant group markers | | Unresolved `::SetSessionLogSeverityLevel()` | onnxruntime_c_api.h | 1065 | PR #26971 - CreateEnvWithOptions API | 3874516 | Use `OrtApi::SetSessionLogSeverityLevel` | | Unresolved `::RunOptionsSetRunLogSeverityLevel()` | onnxruntime_c_api.h | 1066 | PR #26971 - CreateEnvWithOptions API | 3874516 | Use `OrtApi::RunOptionsSetRunLogSeverityLevel` | | `<ep_name>` interpreted as HTML | onnxruntime_c_api.h | 1119 | PR #26971 - CreateEnvWithOptions API | 3874516 | Escape as `\<ep_name\>` | | `\param[in] importer` not found | onnxruntime_c_api.h | 7982 | PR #26828 - OrtExternalResourceImporter API | c54be3c | Use `\param[in] input` (macro expands to `input`) | | `\param[in] handle` not found | onnxruntime_c_api.h | 8025 | PR #26828 - OrtExternalResourceImporter API | c54be3c | Use `\param[in] input` | | `\param[in] handle` not found | onnxruntime_c_api.h | 8091 | PR #26828 - OrtExternalResourceImporter API | c54be3c | Use `\param[in] input` | | Unresolved `::CreateLoopKernel()` | onnxruntime_ep_c_api.h | 667 | PR #26927 - Control flow kernels API | 1ed8fd9 | Use `OrtEpApi::CreateLoopKernel` | | Unresolved `::CreateScanKernel()` | onnxruntime_ep_c_api.h | 710 | PR #26927 - Control flow kernels API | 1ed8fd9 | Use `OrtEpApi::CreateScanKernel` | | `<ep_name>` interpreted as HTML | onnxruntime_ep_c_api.h | 1434 | PR #26971 - CreateEnvWithOptions API | 3874516 | Escape as `\<ep_name\>` | | `\param[out] out` not found | onnxruntime_ep_c_api.h | 1440 | PR #26971 - CreateEnvWithOptions API | 3874516 | Use `\param[out] config_entries` | ## Summary by PR | PR | Issues | |----|--------| | **#26828** (c54be3c) - OrtExternalResourceImporter API for D3D12 | Duplicate Doxygen group, incorrect `\param` names for `ORT_CLASS_RELEASE` macros | | **#26927** (1ed8fd9) - Control flow kernels API | `::Method()` syntax unresolvable by Doxygen | | **#26971** (3874516) - CreateEnvWithOptions API | `::Method()` syntax, `<ep_name>` HTML interpretation, incorrect param name | ## Technical Details ### `ORT_CLASS_RELEASE` Macro Issue The `ORT_CLASS_RELEASE(X)` macro at line 164 expands to: ```cpp void(ORT_API_CALL * Release##X)(_Frees_ptr_opt_ Ort##X * input) ``` The parameter is always named `input`, but the documentation in PR #26828 used semantic names like `importer` and `handle`. Doxygen validates `\param` names against actual parameter names in the expanded code. ### Doxygen Link Resolution Doxygen 1.9.8 cannot resolve `::MethodName()` as a link to a method. The correct syntax is to qualify with the struct name: `OrtApi::MethodName`. ## Testing Verified locally with Doxygen 1.9.8 (matches CI configuration). (cherry picked from commit 39f966e)

…icrosoft#26828)

…ft#27083) # Fix Doxygen documentation build errors from recent PRs Fixes multiple Doxygen errors introduced by recent API additions that cause the nightly documentation build to fail (`WARN_AS_ERROR=YES`). ## Root Cause Analysis | Error | File | Line | Introduced By | Commit | Fix | |-------|------|------|---------------|--------|-----| | Duplicate `\addtogroup Global` | onnxruntime_c_api.h | 973 | PR microsoft#26828 - OrtExternalResourceImporter API | c54be3c | Remove redundant group markers | | Unresolved `::SetSessionLogSeverityLevel()` | onnxruntime_c_api.h | 1065 | PR microsoft#26971 - CreateEnvWithOptions API | 3874516 | Use `OrtApi::SetSessionLogSeverityLevel` | | Unresolved `::RunOptionsSetRunLogSeverityLevel()` | onnxruntime_c_api.h | 1066 | PR microsoft#26971 - CreateEnvWithOptions API | 3874516 | Use `OrtApi::RunOptionsSetRunLogSeverityLevel` | | `<ep_name>` interpreted as HTML | onnxruntime_c_api.h | 1119 | PR microsoft#26971 - CreateEnvWithOptions API | 3874516 | Escape as `\<ep_name\>` | | `\param[in] importer` not found | onnxruntime_c_api.h | 7982 | PR microsoft#26828 - OrtExternalResourceImporter API | c54be3c | Use `\param[in] input` (macro expands to `input`) | | `\param[in] handle` not found | onnxruntime_c_api.h | 8025 | PR microsoft#26828 - OrtExternalResourceImporter API | c54be3c | Use `\param[in] input` | | `\param[in] handle` not found | onnxruntime_c_api.h | 8091 | PR microsoft#26828 - OrtExternalResourceImporter API | c54be3c | Use `\param[in] input` | | Unresolved `::CreateLoopKernel()` | onnxruntime_ep_c_api.h | 667 | PR microsoft#26927 - Control flow kernels API | 1ed8fd9 | Use `OrtEpApi::CreateLoopKernel` | | Unresolved `::CreateScanKernel()` | onnxruntime_ep_c_api.h | 710 | PR microsoft#26927 - Control flow kernels API | 1ed8fd9 | Use `OrtEpApi::CreateScanKernel` | | `<ep_name>` interpreted as HTML | onnxruntime_ep_c_api.h | 1434 | PR microsoft#26971 - CreateEnvWithOptions API | 3874516 | Escape as `\<ep_name\>` | | `\param[out] out` not found | onnxruntime_ep_c_api.h | 1440 | PR microsoft#26971 - CreateEnvWithOptions API | 3874516 | Use `\param[out] config_entries` | ## Summary by PR | PR | Issues | |----|--------| | **microsoft#26828** (c54be3c) - OrtExternalResourceImporter API for D3D12 | Duplicate Doxygen group, incorrect `\param` names for `ORT_CLASS_RELEASE` macros | | **microsoft#26927** (1ed8fd9) - Control flow kernels API | `::Method()` syntax unresolvable by Doxygen | | **microsoft#26971** (3874516) - CreateEnvWithOptions API | `::Method()` syntax, `<ep_name>` HTML interpretation, incorrect param name | ## Technical Details ### `ORT_CLASS_RELEASE` Macro Issue The `ORT_CLASS_RELEASE(X)` macro at line 164 expands to: ```cpp void(ORT_API_CALL * Release##X)(_Frees_ptr_opt_ Ort##X * input) ``` The parameter is always named `input`, but the documentation in PR microsoft#26828 used semantic names like `importer` and `handle`. Doxygen validates `\param` names against actual parameter names in the expanded code. ### Doxygen Link Resolution Doxygen 1.9.8 cannot resolve `::MethodName()` as a link to a method. The correct syntax is to qualify with the struct name: `OrtApi::MethodName`. ## Testing Verified locally with Doxygen 1.9.8 (matches CI configuration).

nieubank requested a review from skottmckay December 18, 2025 21:04

skottmckay reviewed Dec 19, 2025

View reviewed changes

skottmckay mentioned this pull request Dec 19, 2025

Graphics ORT interop design and implementation #26543

Draft

nieubank marked this pull request as ready for review December 19, 2025 23:04

skottmckay reviewed Dec 22, 2025

View reviewed changes

include/onnxruntime/core/session/onnxruntime_ep_c_api.h Show resolved Hide resolved

Use ORT_API_VERSION instead of separate version constants for externa…

2d49c05

…l resource structs

yuslepukhin reviewed Dec 22, 2025

View reviewed changes

onnxruntime/test/autoep/library/example_plugin_ep/ep_external_resource_importer.h Outdated Show resolved Hide resolved

yuslepukhin previously approved these changes Dec 22, 2025

View reviewed changes

Use std::unique_ptr for simulated memory management in ExampleExterna…

c94db3e

…lMemoryHandle

nieubank dismissed yuslepukhin’s stale review via c94db3e December 23, 2025 02:24

nieubank enabled auto-merge (squash) January 5, 2026 23:39

nieubank requested a review from skottmckay January 5, 2026 23:40

nieubank disabled auto-merge January 6, 2026 22:38

nieubank added 2 commits January 6, 2026 19:44

Merge branch 'main' into nieubank/ext_importer

53e29f9

Resolved API conflicts by placing KernelInfo APIs before Interop APIs

yuslepukhin reviewed Jan 7, 2026

View reviewed changes

onnxruntime/core/session/ep_interop_api.cc Show resolved Hide resolved

nieubank added 2 commits January 6, 2026 20:24

skottmckay reviewed Jan 7, 2026

View reviewed changes

include/onnxruntime/core/session/onnxruntime_c_api.h Show resolved Hide resolved

skottmckay reviewed Jan 7, 2026

View reviewed changes

include/onnxruntime/core/session/onnxruntime_c_api.h Outdated Show resolved Hide resolved

skottmckay reviewed Jan 7, 2026

View reviewed changes

include/onnxruntime/core/session/onnxruntime_c_api.h Show resolved Hide resolved

skottmckay reviewed Jan 7, 2026

View reviewed changes

Address feedback, remove access mode from descriptors and update rela…

89ad6c7

…ted comments

github-advanced-security bot found potential problems Jan 8, 2026

View reviewed changes

onnxruntime/test/autoep/library/example_plugin_ep/ep_external_resource_importer.h Fixed Show fixed Hide fixed

Add TODO for Windows Vulkan interop

bd715ae

nieubank added 2 commits January 8, 2026 09:08

Fix lint issue

af29375

Merge branch 'main' into nieubank/ext_importer

b850c3a

skottmckay approved these changes Jan 9, 2026

View reviewed changes

nieubank merged commit c54be3c into main Jan 9, 2026
102 of 103 checks passed

nieubank deleted the nieubank/ext_importer branch January 9, 2026 08:04

skottmckay mentioned this pull request Jan 19, 2026

[webgpu] Support CopyTensors with offsets #27004

Draft

alex-spacemit pushed a commit to spacemit-com/onnxruntime that referenced this pull request Jan 20, 2026

Add OrtExternalResourceImporter API for D3D12 shared resource import (m…

689bc5d

…icrosoft#26828)

nieubank mentioned this pull request Jan 20, 2026

Fix Doxygen documentation build error in onnxruntime_c_api.h #27083

Merged

alex-spacemit pushed a commit to spacemit-com/onnxruntime that referenced this pull request Jan 27, 2026

Add OrtExternalResourceImporter API for D3D12 shared resource import (m…

1fe2c4e

…icrosoft#26828)

Add OrtExternalResourceImporter API for D3D12 shared resource import #26828

Add OrtExternalResourceImporter API for D3D12 shared resource import #26828

Uh oh!

Conversation

nieubank commented Dec 18, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Motivation and Context

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

yuslepukhin left a comment

Choose a reason for hiding this comment

Uh oh!

nieubank commented Jan 5, 2026

Uh oh!

yuslepukhin commented Jan 5, 2026

Uh oh!

skottmckay commented Jan 6, 2026

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

gedoensmax commented Jan 8, 2026

Uh oh!

nieubank commented Jan 8, 2026

Uh oh!

skottmckay left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

nieubank commented Dec 18, 2025 •

edited

Loading