[Plugin EP] Add plugin EP APIs to retrieve ONNX operator schemas by adrianlizarraga · Pull Request #27713 · microsoft/onnxruntime

adrianlizarraga · 2026-03-17T20:48:55Z

Description

Adds C/C++ APIs to the OrtEpApi that allow plugin EPs to query ONNX operator schemas from ORT's global schema registry. This enables EPs to programmatically discover operator metadata (input/output names, type constraints, allowed types, since_version) needed to correctly build kernel definitions with proper type constraints.

Motivation

Resolves #27680. Plugin EPs must provide exact type constraint names (e.g., "T", "T1") and allowed types when calling KernelDefBuilder::AddTypeConstraint(). Without schema access, EPs must either hard-code these names or skip type constraints entirely, leading to potentially incorrect kernel selection and data type mismatches at runtime.

Why can't an EP library just link to its own ONNX library? The ONNX OpSchemaRegistry is a Meyers singleton (static local in Instance()). Each shared library gets its own copy of that static variable: on Windows each DLL is isolated by default, on macOS two-level namespaces have the same effect, and on Linux behavior depends on dlopen flags (RTLD_LOCAL isolates, RTLD_GLOBAL creates unpredictable interposition). Even when isolation doesn't occur, the EP's registry would lack ORT's contrib and internal schemas, and version mismatches between the EP's ONNX library and ORT's vendored copy could cause silent divergence. A C API through ORT is the only reliable, portable way to query the schemas ORT actually uses.

Changes

New opaque types:

OrtOpSchema — owning opaque struct wrapping an onnx::OpSchema* with precomputed type constraint data. Allocated by GetOpSchema, released by ReleaseOpSchema.
OrtOpSchemaTypeConstraint — non-owning opaque entity representing a single type constraint (e.g., "T"). Lifetime is tied to the parent OrtOpSchema. Each constraint carries its name, allowed types, and input/output index mappings.

New C APIs added to OrtEpApi (Version 1.25, 15 functions):

Function	Description
`GetOpSchema`	Look up a schema by name, max opset version, and domain. Accepts `""` or `"ai.onnx"` for standard ONNX ops, `"ai.onnx.ml"` for ML ops, `"com.microsoft"` for contrib ops.
`ReleaseOpSchema`	Release an `OrtOpSchema` allocated by `GetOpSchema`.
`OpSchema_GetSinceVersion`	Get the opset version that introduced the schema.
`OpSchema_GetNumInputs` / `GetNumOutputs`	Input/output counts.
`OpSchema_GetInputName` / `GetOutputName`	Formal parameter names.
`OpSchema_GetInputTypeConstraint` / `GetOutputTypeConstraint`	Get the type constraint for a given input/output (O(1) lookup). Returns `nullptr` if the input/output has no type constraint. Shared constraints return the same pointer (pointer identity = shared type).
`OpSchema_GetTypeConstraintCount`	Number of unique type constraints.
`OpSchema_GetTypeConstraint`	Get the i-th type constraint by index.
`OpSchemaTypeConstraint_GetTypeParamName`	Get the type parameter name (e.g., `"T"`, `"T1"`).
`OpSchemaTypeConstraint_GetAllowedTypes`	Get the allowed type strings (e.g., `"tensor(float)"`).
`OpSchemaTypeConstraint_GetInputIndices`	Get input indices using this constraint.
`OpSchemaTypeConstraint_GetOutputIndices`	Get output indices using this constraint.

C++ wrappers:

Ort::OpSchema — owning wrapper around OrtOpSchema* (move-only, auto-releases).
Ort::ConstOpSchemaTypeConstraint — non-owning wrapper around const OrtOpSchemaTypeConstraint*.
Ort::GetOpSchema() — free function to query the registry.

Design highlights:

Type constraints are eagerly precomputed during GetOpSchema — all subsequent accessors are O(1) with no allocation.
GetInputTypeConstraint/GetOutputTypeConstraint return the full constraint object directly (not just a string), enabling a 2-call workflow: GetInputTypeConstraint(0) → GetAllowedTypes().
Pointer identity: inputs sharing a constraint (e.g., both inputs of Add use "T") return the same OrtOpSchemaTypeConstraint*.
Domain "ai.onnx" is normalized to "" (the canonical ONNX domain) for transparent lookup.

Tests: 14 unit tests covering known/unknown ops, version boundaries, wrong domains, "ai.onnx" alias, schema properties (Add, Relu, LSTM), type constraint access, pointer identity for shared constraints, out-of-range errors, and the input→constraint→allowed-types workflow.

Files

File	Description
`onnxruntime/core/session/abi_opschema.h`	Internal struct definitions for `OrtOpSchemaTypeConstraint` and `OrtOpSchema`.
`include/.../onnxruntime_ep_c_api.h`	Public C API: function signatures, doc comments, opaque type declarations.
`onnxruntime/core/session/plugin_ep/ep_api.h`	Internal function declarations.
`onnxruntime/core/session/plugin_ep/ep_api.cc`	Implementation of all 15 functions + API struct initializer.
`include/.../onnxruntime_cxx_api.h`	C++ wrapper class declarations.
`include/.../onnxruntime_cxx_inline.h`	C++ wrapper inline implementations.
`onnxruntime/test/framework/ep_plugin_provider_test.cc`	Unit tests.

…information

Copilot

Pull request overview

Adds plugin Execution Provider (EP) APIs (C and C++) to query ONNX operator schemas from the global ONNX schema registry, along with unit tests validating the new wrapper behavior.

Changes:

Extend OrtEpApi with functions to look up an ONNX OpSchema and query basic schema metadata (since_version, inputs/outputs, type constraints).
Add C++ wrapper types/functions (Ort::ConstOpSchema, Ort::GetOpSchema) over the new OrtEpApi schema functions.
Add unit tests covering successful/unsuccessful schema lookup and basic schema property queries.

Reviewed changes

Copilot reviewed 8 out of 8 changed files in this pull request and generated 3 comments.

Show a summary per file

File	Description
onnxruntime/test/framework/ep_plugin_provider_test.cc	Adds unit tests for the new C++ OpSchema wrapper and lookup function.
onnxruntime/core/session/plugin_ep/ep_api.h	Declares new plugin EP API entry points for OpSchema lookup and metadata access.
onnxruntime/core/session/plugin_ep/ep_api.cc	Implements the OpSchema APIs using the ONNX schema registry and appends them to `OrtEpApi`.
onnxruntime/core/framework/error_code_helper.h	Introduces `ORT_API_RETURN_IF` macro for concise `OrtStatus*` error returns.
include/onnxruntime/core/session/onnxruntime_ep_c_api.h	Exposes the new OpSchema APIs in the public EP C API surface with documentation.
include/onnxruntime/core/session/onnxruntime_cxx_inline.h	Implements the inline C++ wrappers for `ConstOpSchema` and `GetOpSchema`.
include/onnxruntime/core/session/onnxruntime_cxx_api.h	Declares the new C++ wrapper type and free function in the public C++ API.
include/onnxruntime/core/session/onnxruntime_c_api.h	Adds the opaque runtime type declaration `OrtOpSchema`.

Comments suppressed due to low confidence (1)

onnxruntime/core/session/plugin_ep/ep_api.cc:1005

The OrtEpApi struct has grown for version 1.25, but the slot/offset static_asserts are only updated up through version 24. Add a new static_assert for the new version boundary (e.g., validate the expected slot index for GetOpSchema or the size/offset at the end of the 1.25 additions) to keep ABI/slot stability checks complete.

    &OrtExecutionProviderApi::OpSchema_HasTypeConstraint,
};

// checks that we don't violate the rule that the functions must remain in the slots they were originally assigned
static_assert(offsetof(OrtEpApi, ReleaseEpDevice) / sizeof(void*) == 1,

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

You can also share your feedback on Copilot code review. Take the survey.

Copilot

Pull request overview

This PR introduces new Plugin EP (OrtEpApi) APIs to retrieve and query ONNX operator schemas from the global ONNX schema registry, and adds a C++ wrapper plus tests to validate the behavior.

Changes:

Added new OrtEpApi C APIs to fetch an operator schema (GetOpSchema) and query schema metadata (since_version, IO names/types, type constraints).
Added C++ convenience wrappers (Ort::ConstOpSchema, Ort::GetOpSchema) over the new EP APIs.
Added unit tests validating schema lookup and basic schema properties for well-known ONNX ops.

Reviewed changes

Copilot reviewed 8 out of 8 changed files in this pull request and generated 1 comment.

Show a summary per file

File	Description
onnxruntime/test/framework/ep_plugin_provider_test.cc	Adds tests for the new C++ OpSchema wrapper and lookup API.
onnxruntime/core/session/plugin_ep/ep_api.h	Declares new OpSchema-related EP API entrypoints.
onnxruntime/core/session/plugin_ep/ep_api.cc	Implements OpSchema EP APIs and appends them to the `OrtEpApi` function table.
onnxruntime/core/framework/error_code_helper.h	Adds `ORT_API_RETURN_IF` helper macro for concise OrtStatus error returns.
include/onnxruntime/core/session/onnxruntime_ep_c_api.h	Extends the public `OrtEpApi` struct with the new OpSchema APIs and documentation.
include/onnxruntime/core/session/onnxruntime_cxx_inline.h	Provides inline implementations for the new `ConstOpSchema` C++ wrapper methods and `Ort::GetOpSchema`.
include/onnxruntime/core/session/onnxruntime_cxx_api.h	Declares the `ConstOpSchema` wrapper type and `Ort::GetOpSchema` API.
include/onnxruntime/core/session/onnxruntime_c_api.h	Adds the opaque runtime type declaration for `OrtOpSchema`.

Comments suppressed due to low confidence (1)

onnxruntime/core/session/plugin_ep/ep_api.cc:1011

The OrtEpApi struct is append-only and has per-version slot static_asserts below. Since new function pointers were added after the “End of Version 24” marker, please add/update the corresponding static_assert for Version 1.25 (e.g., based on the last new field) so ABI slot stability continues to be enforced for the new version.

// checks that we don't violate the rule that the functions must remain in the slots they were originally assigned
static_assert(offsetof(OrtEpApi, ReleaseEpDevice) / sizeof(void*) == 1,
              "Size of version 22 API cannot change");  // initial version in ORT 1.22
static_assert(offsetof(OrtEpApi, GetSyncIdForLastWaitOnSyncStream) / sizeof(void*) == 15,
              "Size of version 23 API cannot change");
static_assert(offsetof(OrtEpApi, GetEnvConfigEntries) / sizeof(void*) == 49,
              "Size of version 24 API cannot change");

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

You can also share your feedback on Copilot code review. Take the survey.

Replace the UB reinterpret_cast<onnx::OpSchema*> approach with a proper opaque struct OrtOpSchema that owns: - A pointer to the onnx::OpSchema from the global registry - Eagerly-built OrtOpSchemaTypeConstraints data Key changes: - GetOpSchema now allocates and returns an owning OrtOpSchema* - Add ReleaseOpSchema; remove ReleaseOpSchemaTypeConstraints - OpSchema_GetTypeConstraints returns a non-owning view (no allocation) - All OpSchema_Get* functions access schema->onnx_schema (no casts) - C++ wrapper: ConstOpSchema -> OpSchema (owning RAII) - C++ wrapper: OpSchemaTypeConstraints -> ConstOpSchemaTypeConstraints (non-owning view) - Rename abi_opschema_type_constraints.h -> abi_opschema.h Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Replace OrtOpSchemaTypeConstraints (plural container + index-based accessors) with OrtOpSchemaTypeConstraint (singular opaque entity per constraint). Key changes: - GetInputTypeStr/GetOutputTypeStr -> GetInputTypeConstraint/GetOutputTypeConstraint returning the constraint object directly (O(1) via lookup table) - Container-level APIs (TypeConstraints_GetName(container, i)) replaced by per-entity APIs (TypeConstraint_GetName(tc)) - OrtOpSchema gains input_to_constraint/output_to_constraint vectors for O(1) input/output -> constraint mapping - Pointer identity: shared constraints return same pointer (e.g., both inputs of Add return the same OrtOpSchemaTypeConstraint*) - Dropped GetTypeConstraints (array return) and FindByName - 15 API slots (was 16) Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Aligns with ONNX terminology where the field is type_param_str in TypeConstraintParam struct. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot

Pull request overview

This PR extends the EP Plugin ABI (OrtEpApi) with new APIs that let plugin execution providers query ONNX operator schemas (including type constraints) from ORT’s global schema registry, plus adds corresponding C++ wrapper types and unit tests.

Changes:

Added OrtEpApi v1.25 OpSchema query/release + schema/type-constraint accessor APIs.
Implemented the new APIs in the plugin EP bridge layer with eager precomputation of type-constraint metadata for O(1) access.
Added C++ RAII wrappers (Ort::OpSchema, Ort::ConstOpSchemaTypeConstraint) and unit tests validating schema lookup and type-constraint workflows.

Reviewed changes

Copilot reviewed 8 out of 8 changed files in this pull request and generated 3 comments.

Show a summary per file

File	Description
onnxruntime/core/session/abi_opschema.h	Introduces internal structs backing the opaque OpSchema/type-constraint handles.
include/onnxruntime/core/session/onnxruntime_ep_c_api.h	Adds new public EP C APIs and documentation for schema/type-constraint access.
onnxruntime/core/session/plugin_ep/ep_api.h	Declares internal implementations for the new EP OpSchema APIs.
onnxruntime/core/session/plugin_ep/ep_api.cc	Implements schema lookup + accessors and appends them to the v1.25 EP API table.
include/onnxruntime/core/session/onnxruntime_cxx_api.h	Adds C++ wrapper types and release wiring for `OrtOpSchema`.
include/onnxruntime/core/session/onnxruntime_cxx_inline.h	Implements the new C++ wrapper methods and `Ort::GetOpSchema()` helper.
onnxruntime/test/framework/ep_plugin_provider_test.cc	Adds unit tests for the new C++ wrapper + schema/type-constraint behavior.
onnxruntime/core/framework/error_code_helper.h	Adds `ORT_API_RETURN_IF` helper macro for concise OrtStatus returns.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Copilot

Pull request overview

Copilot reviewed 8 out of 8 changed files in this pull request and generated 2 comments.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Copilot

Pull request overview

Copilot reviewed 8 out of 8 changed files in this pull request and generated no new comments.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

### Description Address some good leftover comments from PR that added EP APIs to retrieve operator schemas: #27713 ### Motivation and Context Clean up as promised

adrianlizarraga added 8 commits March 16, 2026 23:49

[Plugin EP] Add C APIs to add OpSchemas for custom ops, get opschema …

89d414f

…information

Add OpSchema APIs to get SinceVersion, NumInputs, and NumOutputs

c57cc57

CXX apis

df4058a

Add unit tests

c79cad5

null checks

a052871

Update OpSchema_HasTypeConstraint to return a status

183ea29

Update C++ API to return std::string

a05ade9

Remove unnecessary forward declaration

5b64fec

adrianlizarraga requested a review from Copilot March 17, 2026 20:58

Copilot started reviewing on behalf of adrianlizarraga March 17, 2026 20:59 View session

Copilot AI reviewed Mar 17, 2026

View reviewed changes

Comment thread include/onnxruntime/core/session/onnxruntime_ep_c_api.h Outdated

Comment thread onnxruntime/test/framework/ep_plugin_provider_test.cc Outdated

Comment thread onnxruntime/core/framework/error_code_helper.h

Review comments

bfe6853

adrianlizarraga requested a review from Copilot March 17, 2026 21:54

Copilot started reviewing on behalf of adrianlizarraga March 17, 2026 21:55 View session

Copilot AI reviewed Mar 17, 2026

View reviewed changes

Comment thread onnxruntime/core/session/plugin_ep/ep_api.cc

adrianlizarraga mentioned this pull request Mar 18, 2026

[Feature Request] Expose OpSchema API for EP Plugin to enable proper TypeConstraint registration #27680

Closed

Merge branch 'main' into adrianl/PluginEp_Kernels_GetOpSchema

b2b67f9

adrianlizarraga marked this pull request as ready for review March 26, 2026 20:51

adrianlizarraga changed the title ~~DRAFT: Add plugin EP APIs to retrieve ONNX operator schemas~~ [Plugin EP] Add plugin EP APIs to retrieve ONNX operator schemas Mar 26, 2026

adrianlizarraga requested a review from tianleiwu March 26, 2026 20:58

mingyueliuh approved these changes Mar 27, 2026

View reviewed changes

adrianlizarraga requested a review from edgchen1 March 27, 2026 17:07

tianleiwu reviewed Mar 27, 2026

View reviewed changes

Comment thread include/onnxruntime/core/session/onnxruntime_cxx_api.h Outdated

adrianlizarraga added 3 commits March 27, 2026 12:09

Merge branch 'main' into adrianl/PluginEp_Kernels_GetOpSchema

b0491bb

Add static_assert for ep_api

419735c

First attempt at getting more info about type constraints

4f58b99

edgchen1 reviewed Mar 27, 2026

View reviewed changes

Comment thread include/onnxruntime/core/session/onnxruntime_ep_c_api.h Outdated

Comment thread onnxruntime/core/session/plugin_ep/ep_api.cc Outdated

Comment thread onnxruntime/test/framework/ep_plugin_provider_test.cc Outdated

adrianlizarraga and others added 3 commits March 27, 2026 18:01

Remove HasTypeConstraint

29f1f7b

adrianlizarraga and others added 6 commits March 30, 2026 14:30

Rename OpSchemaTypeConstraint_GetName to GetTypeParamName

09d5ecd

Aligns with ONNX terminology where the field is type_param_str in TypeConstraintParam struct. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Mention domains and handle 'ai.onnx' alias

87dc453

Update docs and move ReleaseOpSchema

50a6541

Add comment about there only being one type param per input or output

742fd26

Add doc comment about use of OpSchema_GetTypeConstraint

061b39c

Merge branch 'main' into adrianl/PluginEp_Kernels_GetOpSchema

2cf04fc

adrianlizarraga requested a review from Copilot March 30, 2026 22:46

Copilot started reviewing on behalf of adrianlizarraga March 30, 2026 22:47 View session

Copilot AI reviewed Mar 30, 2026

View reviewed changes

Comment thread onnxruntime/test/framework/ep_plugin_provider_test.cc

Comment thread include/onnxruntime/core/session/onnxruntime_cxx_api.h Outdated

Comment thread onnxruntime/core/session/plugin_ep/ep_api.cc

adrianlizarraga added 3 commits March 30, 2026 16:08

Review comments

8437870

Make tests test what comments say

a195d9a

Consolidate tests

82d5322

adrianlizarraga requested a review from Copilot March 30, 2026 23:43

Copilot started reviewing on behalf of adrianlizarraga March 30, 2026 23:44 View session

Copilot AI reviewed Mar 30, 2026

View reviewed changes

Comment thread include/onnxruntime/core/session/onnxruntime_cxx_inline.h

Comment thread include/onnxruntime/core/session/onnxruntime_cxx_inline.h

Address review comments

a3a60fb

adrianlizarraga requested a review from Copilot March 31, 2026 00:03

Copilot started reviewing on behalf of adrianlizarraga March 31, 2026 00:04 View session

Copilot AI reviewed Mar 31, 2026

View reviewed changes

adrianlizarraga requested review from edgchen1 and tianleiwu March 31, 2026 16:51

edgchen1 approved these changes Mar 31, 2026

View reviewed changes

Comment thread include/onnxruntime/core/session/onnxruntime_cxx_api.h

Comment thread onnxruntime/test/framework/ep_plugin_provider_test.cc

Comment thread onnxruntime/test/framework/ep_plugin_provider_test.cc

adrianlizarraga merged commit 048e7dc into main Mar 31, 2026
111 of 116 checks passed

adrianlizarraga deleted the adrianl/PluginEp_Kernels_GetOpSchema branch March 31, 2026 21:03

adrianlizarraga mentioned this pull request Apr 1, 2026

Cleanup for op schema API tests for plugin EPs #27921

Merged

BrewTestBot mentioned this pull request Apr 20, 2026

onnxruntime 1.25.0 Homebrew/homebrew-core#278543

Merged

Conversation

adrianlizarraga commented Mar 17, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Motivation

Changes

Files

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

adrianlizarraga commented Mar 17, 2026 •

edited

Loading