-
Notifications
You must be signed in to change notification settings - Fork 4.1k
Experimental C API support #28746
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
Experimental C API support #28746
Changes from 22 commits
Commits
Show all changes
28 commits
Select commit
Hold shift + click to select a range
87799ee
add readme
edgchen1 bb4965d
Add Experimental C API design doc
edgchen1 e206370
Update naming to ExpSinceV<ver> prefix, merge headers, remove retired…
edgchen1 bf0dc83
make API version a suffix. some minor refinement.
edgchen1 69e40da
answers to open questions
edgchen1 6da8cbc
Add experimental C API mechanism with name-based function lookup
edgchen1 6058cd5
Update design doc to match implemented naming
edgchen1 27774df
fix Ort::Status usage
edgchen1 c5dc853
fix formatting
edgchen1 1400fbd
Rename ORT_EXPERIMENTAL_FUNC to ORT_EXPERIMENTAL_API
edgchen1 fddfa82
Update design doc: rename ORT_EXPERIMENTAL_FUNC to ORT_EXPERIMENTAL_API
edgchen1 e6d0e8c
Replace design doc reference with .inc file reference in experimental…
edgchen1 819e945
update comments
edgchen1 f13c1a3
lint
edgchen1 4a3509a
fix spacing
edgchen1 15519c3
minor updates
edgchen1 2796cf1
Address PR review: shorten names, reorder macro params, add UB comment
edgchen1 ba792d0
Add .github/copilot-instructions.md with C API versioning guidance
edgchen1 1bcf51c
Rename experimental API suffix from _V to _SinceV
edgchen1 e2ea7fb
Merge remote-tracking branch 'origin/main' into edgchen1/experimental…
edgchen1 35e2664
add some newlines around macros
edgchen1 b2d893e
Update experimental API test function to SinceV28
edgchen1 7006004
Address review feedback: comments and ORT_API_T usage
edgchen1 3f7b884
update comment, use std::array for function table
edgchen1 7db5c42
cmake: sort files alphabetically, remove trailing whitespace
edgchen1 f1fd062
update experimental header comments
edgchen1 9e0e9a4
Apply suggestion from @edgchen1
edgchen1 3e4ae3c
fix doxygen issues
edgchen1 File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,15 @@ | ||
| # Copilot Instructions for ONNX Runtime | ||
|
|
||
| For detailed codebase conventions, architecture, and coding standards, see [AGENTS.md](../AGENTS.md). | ||
|
|
||
| ## Code Review | ||
|
|
||
| ### No C API Version Bump Needed for API Additions | ||
|
|
||
| `ORT_API_VERSION`, the `ort_api_1_to_N` function pointer table, and the version-boundary `static_assert` checks in | ||
| `onnxruntime/core/session/onnxruntime_c_api.cc` are updated only during release preparation — not each time a new API | ||
| is added. See [`docs/Versioning.md`](../docs/Versioning.md) for the full release versioning process. | ||
|
|
||
| During development, new API function pointers are appended to the **current** `ort_api_1_to_N` table. This is the | ||
| expected workflow and does **not** require a version bump, a new table, or new `static_assert` entries. Do not flag | ||
| PRs that append new function pointers to the current table as needing a version bump. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,258 @@ | ||
| # Experimental C API Design | ||
|
|
||
| ## Problem Statement | ||
|
|
||
| The ORT C API (`OrtApi` struct) provides a stable binary interface with strict backward compatibility guarantees: functions are append-only, never reordered or removed, and versioned so that older clients work against newer libraries. | ||
|
|
||
| This stability comes at a cost: new public APIs cannot be iterated on. Once a function is added to the stable struct, its signature and slot are permanent. We need a mechanism to expose new APIs for experimental/preview usage before committing them to the stable surface. | ||
|
|
||
| Requirements: | ||
| - Allow test usage of new public APIs before promotion to the stable API | ||
| - No stability guarantee for experimental functions | ||
| - Minimal impact on the stable API surface | ||
| - Reasonable ergonomics for C and C++ consumers | ||
| - Functions may persist across multiple releases if unchanged | ||
| - Clean promotion path to the stable API | ||
|
|
||
| ## Approaches Considered | ||
|
|
||
| ### Approach A: Versioned Experimental Struct (Exact Match) | ||
|
|
||
| Add a stable API entry point that returns a `const OrtExperimentalApi*` struct, similar to the existing sub-API pattern (`GetCompileApi`, `GetEpApi`, etc.). The struct would require an exact version match—any change to the struct layout bumps the version, and the runtime only satisfies the exact version it was built with. | ||
|
|
||
| **Pros:** | ||
| - Same ergonomics as the existing stable API (typed struct, IDE autocomplete) | ||
| - Full type safety with no casts after the initial retrieval | ||
| - Familiar pattern for existing ORT API consumers | ||
|
|
||
| **Cons:** | ||
| - A struct version bump due to *any* function changing breaks *all* consumers, even those only using unchanged functions | ||
| - Mimics the stable API pattern but intentionally breaks its contract—semantically confusing | ||
| - Requires either per-version headers or a single "latest only" header | ||
| - Doesn't naturally support functions that persist unchanged across releases | ||
| - If a user doesn't know the runtime ORT version, they'd have to probe multiple versions | ||
|
|
||
| ### Approach B: Name-Based Function Pointer Lookup | ||
|
|
||
| Add a single stable API entry point that retrieves an experimental function pointer by name. A companion header provides typedefs, name constants, and (for C++) typed accessor helpers. | ||
|
|
||
| **Pros:** | ||
| - Each function is independently addressable—unchanged functions keep resolving across releases | ||
| - Signature changes use a new name (API version-introduced suffix guarantees uniqueness); old name can be removed independently | ||
| - Minimal stable API cost (one slot) | ||
| - The instability contract is semantically clear: "is this specific thing available?" | ||
| - Promotion to stable is clean: move to `OrtApi`, optionally keep the name as a redirect | ||
| - Adding/removing experimental functions doesn't affect unrelated consumers | ||
|
|
||
| **Cons:** | ||
| - Requires one cast from the generic function pointer to the correct function pointer type | ||
| - Less discoverable without the companion header | ||
| - String-based lookup has minor runtime cost (irrelevant—done once at init) | ||
|
|
||
| ## Chosen Approach: Name-Based Lookup with Typed C++ Helpers | ||
|
|
||
| The name-based approach (B) is the better fit because: | ||
|
|
||
| 1. **Individual function longevity**: Experimental functions may persist unchanged for several releases before promotion. The struct approach breaks all consumers on any change; name-based lookup only affects the specific function that changed. | ||
|
|
||
| 2. **Honest contract**: A struct *looks* stable but isn't. A per-function lookup makes instability semantically obvious. | ||
|
|
||
| 3. **Simpler maintenance**: No struct layout to track, no version bumps to coordinate. Just add/remove entries from a registration table. | ||
|
|
||
| 4. **Ergonomics are solvable**: The C++ wrapper with macro-generated typed accessors provides essentially the same user experience as a struct, minus one initial cast. | ||
|
|
||
| ## Design Details | ||
|
|
||
| ### Stable API Addition | ||
|
|
||
| One function pointer slot added to `OrtApi`: | ||
|
|
||
| ```c | ||
| // Generic function pointer type used as an opaque handle. | ||
| // Cast to the correct function pointer type before calling. | ||
| typedef void (ORT_API_CALL* OrtExperimentalFnPtr)(void); | ||
|
|
||
| // Returns nullptr if the named function is not available in this build. | ||
| OrtExperimentalFnPtr(ORT_API_CALL* GetExperimentalFunction)(_In_ const char* name) NO_EXCEPTION; | ||
| ``` | ||
|
|
||
| Using a function pointer type (rather than `void*`) ensures that casting to the correct | ||
| function pointer type and back is well-defined in both C and C++ per the standard. | ||
| Consumers must cast the returned value to the exact typedef before calling—calling through | ||
| any other type is undefined behavior. Including `ORT_API_CALL` in the typedef matches the | ||
| calling convention of all ORT API functions, which avoids compiler warnings when casting | ||
| between the generic and typed pointers. | ||
|
|
||
| ### Single Source of Truth: The `.inc` File | ||
|
|
||
| All experimental functions are declared in one [X-macro](https://en.wikipedia.org/wiki/X_macro) include file. | ||
| The first argument is the ORT API version in which the function was introduced. The macro | ||
| mechanically constructs the lookup name as `<Name>_SinceV<API Version>`, guaranteeing uniqueness | ||
| by construction—no two entries can collide unless they share both the same version and the | ||
| same base name, which is trivially avoided during review. | ||
|
|
||
| ```c | ||
| // onnxruntime_experimental_c_api.inc | ||
| // | ||
| // ORT_EXPERIMENTAL_API(SinceVersion, ReturnType, Name, Params...) | ||
|
|
||
| ORT_EXPERIMENTAL_API(22, OrtStatusPtr, OrtApi_SomeNewThing, | ||
| _In_ const OrtSession* session, _Out_ int64_t* result) | ||
|
|
||
| ORT_EXPERIMENTAL_API(22, OrtStatusPtr, OrtApi_AnotherThing, | ||
| _In_ const OrtEnv* env, _In_ const char* name, _Out_ OrtValue** out) | ||
| ``` | ||
|
|
||
| ### Experimental Consumer Header (generated from `.inc`) | ||
|
|
||
| A single header serves both C and C++ experimental API consumers. The C section provides typedefs and name | ||
| constants; the C++ section (guarded by `#ifdef __cplusplus`) adds typed inline accessors in the `Ort::Experimental` | ||
| namespace. | ||
|
|
||
| ```c | ||
| // onnxruntime_experimental_c_api.h | ||
| #pragma once | ||
|
|
||
| // Declare any new, auxiliary opaque types required by the experimental APIs in this header too. | ||
|
|
||
| // --- C: function pointer typedefs and name constants --- | ||
| #define ORT_EXPERIMENTAL_API(VER, RET, NAME, ...) \ | ||
| typedef RET(ORT_API_CALL* OrtExperimental_##NAME##_SinceV##VER##_Fn)(__VA_ARGS__) NO_EXCEPTION; \ | ||
| static const char* const kOrtExperimental_##NAME##_SinceV##VER##_FnName = #NAME "_SinceV" #VER; | ||
| #include "onnxruntime_experimental_c_api.inc" | ||
| #undef ORT_EXPERIMENTAL_API | ||
|
|
||
| // Produces (for SinceVersion=22, Name=OrtApi_SomeNewThing): | ||
| // typedef OrtStatusPtr(ORT_API_CALL* OrtExperimental_OrtApi_SomeNewThing_SinceV22_Fn)( | ||
| // ...) NO_EXCEPTION; | ||
| // static const char* const kOrtExperimental_OrtApi_SomeNewThing_SinceV22_FnName = | ||
| // "OrtApi_SomeNewThing_SinceV22"; | ||
|
|
||
| #ifdef __cplusplus | ||
| namespace Ort { | ||
| namespace Experimental { | ||
|
|
||
| // --- C++: typed inline accessors (reuses the C typedefs above) --- | ||
| #define ORT_EXPERIMENTAL_API(VER, RET, NAME, ...) \ | ||
| inline OrtExperimental_##NAME##_SinceV##VER##_Fn Get_##NAME##_SinceV##VER##_Fn( \ | ||
| const OrtApi* api) { \ | ||
| return reinterpret_cast<OrtExperimental_##NAME##_SinceV##VER##_Fn>( \ | ||
| api->GetExperimentalFunction(kOrtExperimental_##NAME##_SinceV##VER##_FnName)); \ | ||
| } | ||
| #include "onnxruntime_experimental_c_api.inc" | ||
| #undef ORT_EXPERIMENTAL_API | ||
|
|
||
| } // namespace Experimental | ||
| } // namespace Ort | ||
|
|
||
| // Produces (for SinceVersion=22, Name=OrtApi_SomeNewThing): | ||
| // namespace Ort { | ||
| // namespace Experimental { | ||
| // inline OrtExperimental_OrtApi_SomeNewThing_SinceV22_Fn | ||
| // Get_OrtApi_SomeNewThing_SinceV22_Fn(const OrtApi* api) { | ||
| // return reinterpret_cast<OrtExperimental_OrtApi_SomeNewThing_SinceV22_Fn>( | ||
| // api->GetExperimentalFunction(kOrtExperimental_OrtApi_SomeNewThing_SinceV22_FnName)); | ||
| // } | ||
| // } | ||
| // } | ||
| #endif // __cplusplus | ||
| ``` | ||
|
|
||
| C usage: | ||
|
|
||
| ```c | ||
| OrtExperimental_OrtApi_SomeNewThing_SinceV22_Fn fn = | ||
| (OrtExperimental_OrtApi_SomeNewThing_SinceV22_Fn)api->GetExperimentalFunction( | ||
| kOrtExperimental_OrtApi_SomeNewThing_SinceV22_FnName); | ||
| if (fn) { | ||
| OrtStatusPtr status = fn(session, &result); | ||
| } | ||
| ``` | ||
|
|
||
| C++ usage: | ||
|
|
||
| ```cpp | ||
| if (auto* fn = Ort::Experimental::Get_OrtApi_SomeNewThing_SinceV22_Fn(api)) { | ||
| Ort::Status status(fn(session, &result)); | ||
| } | ||
| ``` | ||
|
|
||
| ### Implementation Side (generated from `.inc`) | ||
|
|
||
| ```cpp | ||
| // experimental_c_api.cc | ||
|
|
||
| // Function implementations use the full constructed name. | ||
| ORT_API_STATUS_IMPL(OrtExperimentalApis::OrtApi_SomeNewThing_SinceV22, | ||
| _In_ const OrtSession* session, _Out_ int64_t* result) { | ||
| API_IMPL_BEGIN | ||
| // ... | ||
| API_IMPL_END | ||
| } | ||
|
|
||
| // Registration table (auto-generated from .inc) | ||
| struct ExperimentalEntry { | ||
| std::string_view name; | ||
| OrtExperimentalFnPtr fn; | ||
| }; | ||
|
|
||
| static const ExperimentalEntry kExperimentalFunctions[] = { | ||
| #define ORT_EXPERIMENTAL_API(VER, RET, NAME, ...) \ | ||
| { #NAME "_SinceV" #VER, reinterpret_cast<OrtExperimentalFnPtr>(&OrtExperimentalApis::NAME##_SinceV##VER) }, | ||
| #include "onnxruntime_experimental_c_api.inc" | ||
| #undef ORT_EXPERIMENTAL_API | ||
| }; | ||
|
|
||
| // Lookup implementation | ||
| ORT_API(OrtExperimentalFnPtr, OrtApis::GetExperimentalFunction, _In_ const char* name) { | ||
| if (name == nullptr) return nullptr; | ||
| std::string_view target(name); | ||
| for (const auto& entry : kExperimentalFunctions) { | ||
| if (entry.name == target) return entry.fn; | ||
| } | ||
| return nullptr; | ||
| } | ||
| ``` | ||
|
|
||
| ### Lifecycle Rules | ||
|
|
||
| 1. **Adding an experimental function**: Add one line to the `.inc` file with the current ORT API version, implement it. | ||
| 2. **Removing a function**: Delete the line from the `.inc`. No retirement tracking is needed—the versioned name is inherently unique and cannot be accidentally reused. | ||
| 3. **Changing a signature**: Add a new entry with the current ORT API version (producing a new unique name) and optionally delete the old entry. Both can coexist if the old signature is still supported. | ||
| 4. **Promoting to stable**: Add the function to the stable API struct (append-only, name drops the `_SinceV<ver>` suffix). Delete the experimental entry. Optionally keep the experimental entry for a transitional period, resolving it as a redirect. | ||
|
|
||
| ### Naming Convention | ||
|
|
||
| Experimental function names follow the pattern `<TargetStruct>_<Name>_SinceV<API Version>`. | ||
| The API version is the ORT API version in which the function was first introduced. The target | ||
| struct prefix indicates the intended promotion destination. This naming scheme guarantees | ||
| uniqueness by construction—a signature change requires a new `.inc` entry at the current | ||
| API version, which produces a distinct name. | ||
|
|
||
| Examples: | ||
|
|
||
| - `OrtApi_SomeNewThing_SinceV22` — introduced in API v22, destined for `OrtApi` | ||
| - `OrtApi_SomeNewThing_SinceV23` — signature changed in API v23, replaces the v22 entry | ||
| - `OrtEpApi_SomeNewEpThing_SinceV22` — destined for `OrtEpApi` | ||
| - `OrtCompileApi_SomeNewCompileThing_SinceV22` — destined for `OrtCompileApi` | ||
|
|
||
| At promotion, the stable struct member drops the `_SinceV<API Version>` suffix (e.g., the stable | ||
| slot is named `SomeNewThing` in `OrtApi`). | ||
|
|
||
| Names are flat strings matched exactly. No separate retirement tracking is needed because | ||
| the version suffix makes accidental name reuse impossible. | ||
|
|
||
| ### Rejected: Enumeration Helper | ||
|
|
||
| A runtime enumeration API (`GetExperimentalFunctionNames`) was considered but rejected. | ||
| Any consumer that wants to *call* an experimental function already needs the header for the | ||
| typedef—and the header *is* the enumeration. A consumer without the header could discover | ||
| names at runtime but couldn't safely call them (no type information). This would cost a | ||
| stable API slot for no practical benefit. | ||
|
|
||
| ## Open Questions | ||
|
|
||
| - Should the Python/C#/Java bindings expose experimental functions, or keep them C/C++ only initially? | ||
| - We can start with C/C++ and prove it out first. | ||
| - Should we document an "epoch" expectation (e.g., "experimental functions are expected to be promoted or removed within 2 releases")? | ||
| - We can set a general expectation without enforcing it. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1 @@ | ||
| This directory contains ORT implementation design documents. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.