From 45c0b4712229ccef9ba12545000dd10cd08861d8 Mon Sep 17 00:00:00 2001 From: edgchen1 <18449977+edgchen1@users.noreply.github.com> Date: Thu, 8 Jan 2026 16:23:08 -0800 Subject: [PATCH 1/7] Add initial plugin EP packaging guidance doc. --- .../plugin-ep-packaging.md | 31 +++++++++++++++++++ 1 file changed, 31 insertions(+) create mode 100644 docs/execution-providers/plugin-ep-packaging.md diff --git a/docs/execution-providers/plugin-ep-packaging.md b/docs/execution-providers/plugin-ep-packaging.md new file mode 100644 index 0000000000000..54911e48f19f7 --- /dev/null +++ b/docs/execution-providers/plugin-ep-packaging.md @@ -0,0 +1,31 @@ +# ONNX Runtime Plugin Execution Provider Packaging Guidance + +## Overview + +This document aims to provide guidance for ONNX Runtime (ORT) plugin Execution Provider (EP) implementers to consider with regards to packaging for a plugin EP. + +## General Guidance + +### Usage + +Users are expected to call the ORT API `RegisterExecutionProviderLibrary()` to register the plugin EP library. Then, they may either choose to use the auto EP selection mechanism or manually call ORT API `SessionOptionsAppendExecutionProvider_V2()` to explicitly use the plugin EP. + +### Structure + +#### Contents + +A plugin EP package should contain the plugin EP shared library file and any other files that need to be distributed with it. + +A plugin EP package should NOT contain the ORT shared library or other core ORT libraries (e.g., onnxruntime.dll or libonnxruntime.so). Users should obtain the ORT library separately, most likely via installing the separate ONNX Runtime package. + +TODO: Should a plugin EP package have a dependency on the ORT package or be independent? + +#### Additional Information to Provide + +There should be a way to get the package's plugin EP library path. The user will need the plugin EP library path to call the ORT API `RegisterExecutionProviderLibrary()`. For example, the package may provide a helper function that returns the path to the plugin EP library. + +There should be a way to get the package's plugin EP name. The user may require the plugin EP name to select the appropriate `OrtEpDevice` instances to provide to the ORT API `SessionOptionsAppendExecutionProvider_V2()`. For example, the plugin EP name may be well-documented or made available with a helper function provided by the package. + +#### Naming + +The name of the package should indicate that the package contains a plugin EP. A special prefix like "onnxruntime-ep" may be used to identify a plugin EP. As an example, for the fictional ContosoAI plugin EP, the Python package might be named "onnxruntime-ep-contosoai". From 2ec8b20b5498f9cc61aa981b2676d4cbd8de2b51 Mon Sep 17 00:00:00 2001 From: edgchen1 <18449977+edgchen1@users.noreply.github.com> Date: Thu, 8 Jan 2026 16:39:17 -0800 Subject: [PATCH 2/7] fill in some more sections --- .../plugin-ep-packaging.md | 27 ++++++++++++++++++- 1 file changed, 26 insertions(+), 1 deletion(-) diff --git a/docs/execution-providers/plugin-ep-packaging.md b/docs/execution-providers/plugin-ep-packaging.md index 54911e48f19f7..1ff0df0e044f0 100644 --- a/docs/execution-providers/plugin-ep-packaging.md +++ b/docs/execution-providers/plugin-ep-packaging.md @@ -8,6 +8,8 @@ This document aims to provide guidance for ONNX Runtime (ORT) plugin Execution P ### Usage +Here, we will refer to the ORT C API functions. Their equivalents should exist for other language bindings that support plugin EP usage. + Users are expected to call the ORT API `RegisterExecutionProviderLibrary()` to register the plugin EP library. Then, they may either choose to use the auto EP selection mechanism or manually call ORT API `SessionOptionsAppendExecutionProvider_V2()` to explicitly use the plugin EP. ### Structure @@ -28,4 +30,27 @@ There should be a way to get the package's plugin EP name. The user may require #### Naming -The name of the package should indicate that the package contains a plugin EP. A special prefix like "onnxruntime-ep" may be used to identify a plugin EP. As an example, for the fictional ContosoAI plugin EP, the Python package might be named "onnxruntime-ep-contosoai". +The name of the package should indicate that the package contains a plugin EP and be distinguishable from other ORT packages. For example, this may be done by using a special prefix or suffix. + +## Package-specific Guidance + +### PyPI + +#### Naming + +The prefix "onnxruntime-ep" can be used to identify a plugin EP. + +The suggested package naming convention is: "onnxruntime-ep-\". For example, "onnxruntime-ep-contoso-ai". + +#### TODO Other PyPI stuff + +### NuGet +TODO + +### Maven +TODO + +### Binary Archives +TODO + +### TODO other package types? From 253ebf7dffd82aca1ee3dad62c9116c284ab5053 Mon Sep 17 00:00:00 2001 From: edgchen1 <18449977+edgchen1@users.noreply.github.com> Date: Thu, 8 Jan 2026 17:18:17 -0800 Subject: [PATCH 3/7] add naming convention for plugin EP name --- docs/execution-providers/plugin-ep-libraries.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/execution-providers/plugin-ep-libraries.md b/docs/execution-providers/plugin-ep-libraries.md index 72bbea88d669d..068aa4bc82c92 100644 --- a/docs/execution-providers/plugin-ep-libraries.md +++ b/docs/execution-providers/plugin-ep-libraries.md @@ -45,7 +45,7 @@ The following table lists the **required** variables and functions that an imple GetName -Get the execution provider name. +Get the execution provider name.

The recommended convention for a plugin execution provider name is to have the name end with the suffix "PluginExecutionProvider". E.g., "ContosoAiPluginExecutionProvider". ExampleEp::GetNameImpl() From 89123be5964fef59e2fa0f726c1f0f0d5c35de81 Mon Sep 17 00:00:00 2001 From: edgchen1 <18449977+edgchen1@users.noreply.github.com> Date: Thu, 22 Jan 2026 16:44:37 -0800 Subject: [PATCH 4/7] update packaging doc --- .../plugin-ep-packaging.md | 41 ++++++++++++++----- 1 file changed, 31 insertions(+), 10 deletions(-) diff --git a/docs/execution-providers/plugin-ep-packaging.md b/docs/execution-providers/plugin-ep-packaging.md index 1ff0df0e044f0..5a62d21df03b3 100644 --- a/docs/execution-providers/plugin-ep-packaging.md +++ b/docs/execution-providers/plugin-ep-packaging.md @@ -8,9 +8,11 @@ This document aims to provide guidance for ONNX Runtime (ORT) plugin Execution P ### Usage -Here, we will refer to the ORT C API functions. Their equivalents should exist for other language bindings that support plugin EP usage. +Note: Generally, when referring to the ORT API, we will refer to the C API functions. Equivalents should exist for other language bindings that support plugin EP usage. -Users are expected to call the ORT API `RegisterExecutionProviderLibrary()` to register the plugin EP library. Then, they may either choose to use the auto EP selection mechanism or manually call ORT API `SessionOptionsAppendExecutionProvider_V2()` to explicitly use the plugin EP. +#### Manual EP Library Registration + +Users are expected to call [`OrtApi::RegisterExecutionProviderLibrary()`](https://onnxruntime.ai/docs/api/c/struct_ort_api.html#a7c8ea74a2ee54d03052f3d7cd1e1335d) to register the plugin EP library. Then, they may either choose to use the auto EP selection mechanism or manually call [`OrtApi::SessionOptionsAppendExecutionProvider_V2()`](https://onnxruntime.ai/docs/api/c/struct_ort_api.html#a285a5da8c9a63eff55dc48e4cf3b56f6) to explicitly use the plugin EP. ### Structure @@ -22,30 +24,49 @@ A plugin EP package should NOT contain the ORT shared library or other core ORT TODO: Should a plugin EP package have a dependency on the ORT package or be independent? +##### Shared Library File Naming + +The suggested plugin EP shared library file naming convention is "onnxruntime_ep_\" for the base name with the appropriate platform-specific prefixes or suffixes. + +For example, "onnxruntime_ep_contoso_ai.dll", "libonnxruntime_ep_contoso_ai.so", or "libonnxruntime_ep_contoso_ai.dylib". + #### Additional Information to Provide -There should be a way to get the package's plugin EP library path. The user will need the plugin EP library path to call the ORT API `RegisterExecutionProviderLibrary()`. For example, the package may provide a helper function that returns the path to the plugin EP library. +There should be a way to get the package's plugin EP library path. The user will need the plugin EP library path to call `OrtApi::RegisterExecutionProviderLibrary()`. For example, the package may provide a helper function that returns the path to the plugin EP library. + +There should be a way to get the package's plugin EP name. The user may require the plugin EP name to select the appropriate `OrtEpDevice` instances to provide to `OrtApi::SessionOptionsAppendExecutionProvider_V2()`. For example, the plugin EP name may be well-documented or made available with a helper function provided by the package. -There should be a way to get the package's plugin EP name. The user may require the plugin EP name to select the appropriate `OrtEpDevice` instances to provide to the ORT API `SessionOptionsAppendExecutionProvider_V2()`. For example, the plugin EP name may be well-documented or made available with a helper function provided by the package. +#### Package Naming -#### Naming +The name of the package should indicate that the package contains a plugin EP and be distinguishable from other ORT packages. -The name of the package should indicate that the package contains a plugin EP and be distinguishable from other ORT packages. For example, this may be done by using a special prefix or suffix. +For example, this may be done by using a special prefix or suffix. ## Package-specific Guidance ### PyPI -#### Naming +#### Package Naming The prefix "onnxruntime-ep" can be used to identify a plugin EP. -The suggested package naming convention is: "onnxruntime-ep-\". For example, "onnxruntime-ep-contoso-ai". +The suggested package naming convention is "onnxruntime-ep-\". + +For example, "onnxruntime-ep-contoso-ai". -#### TODO Other PyPI stuff +#### TODO other PyPI info ### NuGet -TODO + +#### Package Naming + +NuGet packages may use a reserved ID prefix. + +The suggested package naming convention is "\.ML.OnnxRuntime.\.EP". + +For example, "Contoso.ML.OnnxRuntime.ContosoAI.EP". + +#### TODO other NuGet info ### Maven TODO From 2505a1b1ff4ce9b45e82a779cb3d3c2dbaa2b1f0 Mon Sep 17 00:00:00 2001 From: edgchen1 <18449977+edgchen1@users.noreply.github.com> Date: Thu, 22 Jan 2026 17:30:59 -0800 Subject: [PATCH 5/7] update packaging docs --- .../plugin-ep-packaging.md | 81 +++++++++++++++---- 1 file changed, 65 insertions(+), 16 deletions(-) diff --git a/docs/execution-providers/plugin-ep-packaging.md b/docs/execution-providers/plugin-ep-packaging.md index 5a62d21df03b3..92ad51b54d5f4 100644 --- a/docs/execution-providers/plugin-ep-packaging.md +++ b/docs/execution-providers/plugin-ep-packaging.md @@ -22,19 +22,20 @@ A plugin EP package should contain the plugin EP shared library file and any oth A plugin EP package should NOT contain the ORT shared library or other core ORT libraries (e.g., onnxruntime.dll or libonnxruntime.so). Users should obtain the ORT library separately, most likely via installing the separate ONNX Runtime package. -TODO: Should a plugin EP package have a dependency on the ORT package or be independent? +A plugin EP package should have no need to depend on the separate ONNX Runtime package, so it is NOT recommended to do so. -##### Shared Library File Naming +#### Additional Information to Provide -The suggested plugin EP shared library file naming convention is "onnxruntime_ep_\" for the base name with the appropriate platform-specific prefixes or suffixes. +##### Library Path -For example, "onnxruntime_ep_contoso_ai.dll", "libonnxruntime_ep_contoso_ai.so", or "libonnxruntime_ep_contoso_ai.dylib". +There should be a way to get the package's plugin EP library path. The user will need the plugin EP library path to call `OrtApi::RegisterExecutionProviderLibrary()`. -#### Additional Information to Provide +For example, the package may provide a helper function that returns the path to the plugin EP library. The recommended name for this helper function is "get library path". -There should be a way to get the package's plugin EP library path. The user will need the plugin EP library path to call `OrtApi::RegisterExecutionProviderLibrary()`. For example, the package may provide a helper function that returns the path to the plugin EP library. +##### EP name(s) +There should be a way to get the plugin EP name(s) provided by the package. The user may require the plugin EP name(s) to select the appropriate `OrtEpDevice` instances to provide to `OrtApi::SessionOptionsAppendExecutionProvider_V2()`. -There should be a way to get the package's plugin EP name. The user may require the plugin EP name to select the appropriate `OrtEpDevice` instances to provide to `OrtApi::SessionOptionsAppendExecutionProvider_V2()`. For example, the plugin EP name may be well-documented or made available with a helper function provided by the package. +For example, the plugin EP name(s) may be well-documented or made available with a helper function provided by the package. The recommended name for this helper function is "get EP names". #### Package Naming @@ -54,7 +55,61 @@ The suggested package naming convention is "onnxruntime-ep-\". For example, "onnxruntime-ep-contoso-ai". -#### TODO other PyPI info +#### Helper Functions + +As mentioned in the general guidance section, the package should provide helper function `get_library_path()` to get the EP library path. The package may provide helper function `get_ep_names()` to get a list of EP names. + +#### Usage example + +```python +import onnxruntime as ort +import onnxruntime_ep_contoso_ai as contoso_ep + +# Path to the plugin EP library +ep_lib_path = contoso_ep.get_library_path() +# Registration name can be anything the application chooses +ep_registration_name = "contoso_ep_registration" + +# Register plugin EP library with ONNX Runtime +ort.register_execution_provider_library(ep_registration_name, ep_lib_path) + +# Create ORT session with explicit OrtEpDevice(s) + +# Get EP name(s) from the plugin EP library +ep_names = contoso_ep.get_ep_names() +# For this example we'll use the first one +ep_name = ep_names[0] + +# Select an OrtEpDevice +# For this example, we'll use any OrtEpDevices matching our EP name +all_ep_devices = ort.get_ep_devices() +selected_ep_devices = [ep_device for ep_device in all_ep_devices if ep_device.ep_name == ep_name] + +assert trt_ep_device != None + +sess_options = ort.SessionOptions() + +# EP-specific options +ep_options = {} + +# Equivalent to the C API's SessionOptionsAppendExecutionProvider_V2 that appends the plugin EP to the session options +sess_options.add_provider_for_devices(selected_ep_devices, ep_options) + +assert sess_options.has_providers() == True + +# Create ORT session with the plugin EP +model_path = "/path/to/model.onnx" +sess = ort.InferenceSession(model_path, sess_options=sess_options) + +# Use `sess` +# ... + +del sess + +# Unregister the library using the same registration name specified earlier +# Must only unregister a library after all sessions that use the library have been released +ort.unregister_execution_provider_library(ep_registration_name) +``` ### NuGet @@ -66,12 +121,6 @@ The suggested package naming convention is "\.ML.OnnxRuntime.\ Date: Thu, 22 Jan 2026 18:22:36 -0800 Subject: [PATCH 6/7] allow for single name with 'get ep name' --- docs/execution-providers/plugin-ep-packaging.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/docs/execution-providers/plugin-ep-packaging.md b/docs/execution-providers/plugin-ep-packaging.md index 92ad51b54d5f4..d7279546f8cbd 100644 --- a/docs/execution-providers/plugin-ep-packaging.md +++ b/docs/execution-providers/plugin-ep-packaging.md @@ -35,7 +35,7 @@ For example, the package may provide a helper function that returns the path to ##### EP name(s) There should be a way to get the plugin EP name(s) provided by the package. The user may require the plugin EP name(s) to select the appropriate `OrtEpDevice` instances to provide to `OrtApi::SessionOptionsAppendExecutionProvider_V2()`. -For example, the plugin EP name(s) may be well-documented or made available with a helper function provided by the package. The recommended name for this helper function is "get EP names". +For example, the plugin EP name(s) may be well-documented or made available with a helper function provided by the package. The recommended name for this helper function is "get EP name", or "get EP names" if there are multiple names. #### Package Naming @@ -57,7 +57,7 @@ For example, "onnxruntime-ep-contoso-ai". #### Helper Functions -As mentioned in the general guidance section, the package should provide helper function `get_library_path()` to get the EP library path. The package may provide helper function `get_ep_names()` to get a list of EP names. +As mentioned in the general guidance section, the package should provide helper function `get_library_path()` to get the EP library path. The package may provide helper function `get_ep_name()` or `get_ep_names()` to get the EP name(s). #### Usage example @@ -123,4 +123,4 @@ For example, "Contoso.ML.OnnxRuntime.ContosoAI.EP". #### Helper Functions -As mentioned in the general guidance section, the package should provide helper function `GetLibraryPath()` to get the EP library path. The package may provide helper function `GetEpNames()` to get a list of EP names. +As mentioned in the general guidance section, the package should provide helper function `GetLibraryPath()` to get the EP library path. The package may provide helper function `GetEpName()` or `GetEpNames()` to get the EP name(s). From 7f3670389dc965309aa10fb201aa8790da260442 Mon Sep 17 00:00:00 2001 From: edgchen1 <18449977+edgchen1@users.noreply.github.com> Date: Thu, 22 Jan 2026 20:50:42 -0800 Subject: [PATCH 7/7] fix example code --- docs/execution-providers/plugin-ep-packaging.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/execution-providers/plugin-ep-packaging.md b/docs/execution-providers/plugin-ep-packaging.md index d7279546f8cbd..ddd286d50fb67 100644 --- a/docs/execution-providers/plugin-ep-packaging.md +++ b/docs/execution-providers/plugin-ep-packaging.md @@ -85,7 +85,7 @@ ep_name = ep_names[0] all_ep_devices = ort.get_ep_devices() selected_ep_devices = [ep_device for ep_device in all_ep_devices if ep_device.ep_name == ep_name] -assert trt_ep_device != None +assert len(selected_ep_devices) > 0 sess_options = ort.SessionOptions()