Skip to content
Merged
Show file tree
Hide file tree
Changes from 25 commits
Commits
Show all changes
55 commits
Select commit Hold shift + click to select a range
859bf60
update proposed APIs
chilo-ms Apr 21, 2026
3fa8fb1
update API doc
chilo-ms Apr 22, 2026
2b780b9
update
chilo-ms Apr 22, 2026
69b814f
add model package apis
chilo-ms Apr 23, 2026
a2271cc
lintrunner -a
chilo-ms Apr 23, 2026
308757d
update
chilo-ms Apr 23, 2026
9a60431
update
chilo-ms Apr 24, 2026
58e3677
clean up
chilo-ms Apr 24, 2026
fadedbd
update
chilo-ms Apr 25, 2026
c8fc7e8
move varaint selector to another speparate files
chilo-ms Apr 27, 2026
ab3a7f9
Merge branch 'main' into chi/model_package_2
chilo-ms Apr 27, 2026
0a1b9fb
update old tests
chilo-ms Apr 27, 2026
e1b9f53
update
chilo-ms Apr 29, 2026
df89b2e
update model package apis
chilo-ms May 1, 2026
b06c240
update schemas
chilo-ms May 4, 2026
5113621
Add APIs for component model
chilo-ms May 5, 2026
bc65339
update apis and tests
chilo-ms May 5, 2026
fb49e36
add more tests
chilo-ms May 6, 2026
d2ff0cb
Merge branch 'main' into chi/model_package_2
chilo-ms May 6, 2026
62406ef
lintrunner -a
chilo-ms May 6, 2026
95b3a5f
update test
chilo-ms May 6, 2026
4c947ce
fix compile warnings
chilo-ms May 6, 2026
6f8ac08
fix compile warning
chilo-ms May 6, 2026
c692fd7
Remove README.md
chilo-ms May 6, 2026
6eb4fee
Merge branch 'main' into chi/model_package_2
chilo-ms May 11, 2026
c24c991
To accommodate validation for mutiple model/ep_compatability in a var…
chilo-ms May 12, 2026
fb9a391
remove unused function
chilo-ms May 12, 2026
19cd170
update API comments
chilo-ms May 13, 2026
40c703f
Model package API refinements: naming, traversal, session options, an…
jambayk May 19, 2026
d90f980
Model package: security, correctness, and session creation simplifica…
jambayk May 20, 2026
19730f3
Fix provider rebuild: always rebuild from effective options, use ep_d…
jambayk May 20, 2026
2670705
Enhance AddSessionConfigEntry and remove AddSessionOption
chilo-ms May 21, 2026
210cc38
Add model package as a standalone library
chilo-ms May 27, 2026
caf9ca2
update API to meet one EP per variant
chilo-ms May 27, 2026
d4f438e
Remove and add apis
chilo-ms May 27, 2026
2cd4355
variant.json is optional
chilo-ms May 27, 2026
4ec00dc
fix compile error
chilo-ms May 27, 2026
97412b5
fix mem leak in test
chilo-ms May 27, 2026
9f46fe1
refactor APIs
chilo-ms May 28, 2026
31311f7
Merge branch 'main' into chi/model_package_2
chilo-ms May 28, 2026
994ad10
single single ep compatibility info. per variant
chilo-ms May 28, 2026
de90468
update OrtModelPackageApi's summary
chilo-ms May 28, 2026
d979c60
refactor the implementation for single ep compatability per variant
chilo-ms May 29, 2026
b4f98fd
Merge branch 'main' into chi/model_package_2
chilo-ms Jun 1, 2026
dccdadd
Merge branch 'main' into chi/model_package_2
chilo-ms Jun 1, 2026
aa58e96
Revert ONNX submodule to previous commit
chilo-ms Jun 1, 2026
429ba69
Merge branch 'chi/model_package_2' of https://github.com/microsoft/on…
chilo-ms Jun 1, 2026
2f91bfb
single file in variant.json
chilo-ms Jun 1, 2026
0caf99e
refactor
chilo-ms Jun 1, 2026
4f889e6
Merge branch 'chi/model_package_2' of https://github.com/microsoft/on…
chilo-ms Jun 1, 2026
946520a
lintrunner -a
chilo-ms Jun 1, 2026
b3165f7
refactor
chilo-ms Jun 1, 2026
847f8e7
address issue when variant.json doesn't exist
chilo-ms Jun 1, 2026
6c4ab4d
fix for ISO build/pipeline
chilo-ms Jun 2, 2026
c4580fa
refactor and fix wasm pipeline issue
chilo-ms Jun 3, 2026
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
171 changes: 171 additions & 0 deletions include/onnxruntime/core/session/onnxruntime_c_api.h
Original file line number Diff line number Diff line change
Expand Up @@ -341,6 +341,9 @@ ORT_RUNTIME_CLASS(ExternalSemaphoreHandle); // EP-imported view of shared exte
ORT_RUNTIME_CLASS(DeviceEpIncompatibilityDetails);
ORT_RUNTIME_CLASS(EpAssignedSubgraph);
ORT_RUNTIME_CLASS(EpAssignedNode);
ORT_RUNTIME_CLASS(ModelPackageOptions);
ORT_RUNTIME_CLASS(ModelPackageContext);
ORT_RUNTIME_CLASS(ModelPackageComponentContext);

#ifdef _MSC_VER
typedef _Return_type_success_(return == 0) OrtStatus* OrtStatusPtr;
Expand Down Expand Up @@ -918,6 +921,9 @@ typedef struct OrtCompileApi OrtCompileApi;
struct OrtInteropApi;
typedef struct OrtInteropApi OrtInteropApi;

struct OrtModelPackageApi;
typedef struct OrtModelPackageApi OrtModelPackageApi;

struct OrtEpApi;
typedef struct OrtEpApi OrtEpApi;

Expand Down Expand Up @@ -7464,6 +7470,19 @@ struct OrtApi {
* \see OrtApi::SetSessionExecutionMode
*/
ORT_API2_STATUS(GetSessionExecutionMode, _In_ const OrtSessionOptions* options, _Out_ ExecutionMode* out);

Comment thread
chilo-ms marked this conversation as resolved.
Outdated
/** \brief Get the OrtModelPackageApi instance, providing APIs for loading, inspecting,
* and creating sessions from ORT model packages.
*
* ONNX Runtime releases guarantee that ::OrtApi is forward- and backward-compatible;
* ::OrtModelPackageApi carries its own independent versioning.
*
* \return Pointer to an ::OrtModelPackageApi, or NULL if model package support is not
* available in this build (e.g., minimal builds).
*
* \since Version 1.27.
*/
const OrtModelPackageApi*(ORT_API_CALL* GetModelPackageApi)(void);
};

/*
Expand Down Expand Up @@ -8551,6 +8570,158 @@ struct OrtInteropApi {
/// @}
};

/** \brief APIs for loading, inspecting, and creating sessions from ONNX model packages.
Comment thread
jambayk marked this conversation as resolved.
Outdated
*
* Obtain via OrtApi::GetModelPackageApi. Mirrors the shape of OrtCompileApi.
*
* Typical flow:
*
* const OrtModelPackageApi* pkg = g_ort->GetModelPackageApi();
*
* OrtModelPackageOptions* options = nullptr;
* pkg->CreateModelPackageOptionsFromSessionOptions(env, session_options, &options);
*
* OrtModelPackageContext* ctx = nullptr;
* pkg->CreateModelPackageContext(env, package_root, options, &ctx);
*
* size_t component_count = 0;
* pkg->ModelPackageContext_GetComponentModelCount(ctx, &component_count);
*
* const char* component_name = nullptr;
* pkg->ModelPackageContext_GetComponentModelName(ctx, 0, &component_name);
*
* OrtSession* session = nullptr;
* pkg->CreateSession(env, ctx, component_name, nullptr, nullptr, &session);
* g_ort->ReleaseSession(session);
* pkg->ReleaseModelPackageContext(ctx);
* pkg->ReleaseModelPackageOptions(options);
*
* \since Version 1.27.
*/
struct OrtModelPackageApi {
/// \name OrtModelPackageOptions
/// @{
ORT_API2_STATUS(CreateModelPackageOptionsFromSessionOptions,
_In_ const OrtEnv* env,
_In_ const OrtSessionOptions* session_options,
_Outptr_ OrtModelPackageOptions** out);

ORT_CLASS_RELEASE(ModelPackageOptions);
/// @}
/// \name OrtModelPackageContext
/// @{
ORT_API2_STATUS(CreateModelPackageContext,
_In_ const ORTCHAR_T* package_root,
_Outptr_ OrtModelPackageContext** out);

ORT_CLASS_RELEASE(ModelPackageContext);

ORT_API2_STATUS(ModelPackage_GetComponentModelCount,
_In_ const OrtModelPackageContext* ctx,
_Out_ size_t* out_count);

ORT_API2_STATUS(ModelPackage_GetComponentModelNames,
_In_ const OrtModelPackageContext* ctx,
_Outptr_result_buffer_maybenull_(*out_count) const char* const** out_names,
_Out_ size_t* out_count);

ORT_API2_STATUS(ModelPackage_GetModelVariantCount,
_In_ const OrtModelPackageContext* ctx,
_In_ const char* component_name,
_Out_ size_t* out_count);

ORT_API2_STATUS(ModelPackage_GetModelVariantNames,
_In_ const OrtModelPackageContext* ctx,
_In_ const char* component_name,
_Outptr_result_buffer_maybenull_(*out_count) const char* const** out_variant_names,
_Out_ size_t* out_count);

/** \brief Select a component model and return an opaque component instance.
*
* The variant selection is also performed during this call based on the component metadata and the provided options.
* The returned `OrtModelPackgeComponentContext*` is independent of `context` lifetime and must be released via
* `ReleaseComponentInstance`.
*
* \since Version 1.27.
*/
ORT_API2_STATUS(SelectComponent,
_In_ const OrtModelPackageContext* context,
_In_ const char* component_name,
_In_ const OrtModelPackageOptions* options,
_Outptr_ OrtModelPackageComponentContext** out);

ORT_CLASS_RELEASE(ModelPackageComponentContext);

ORT_API2_STATUS(ModelPackageComponent_GetSelectedVariantFolderPath,
_In_ const OrtModelPackageComponentContext* ctx,
_Outptr_ const ORTCHAR_T** folder_path);

ORT_API2_STATUS(ModelPackageComponent_GetSelectedVariantFileCount,
_In_ const OrtModelPackageComponentContext* ctx,
_Out_ size_t* num_files);

ORT_API2_STATUS(ModelPackageComponent_GetSelectedVariantFilePath,
_In_ const OrtModelPackageComponentContext* ctx,
_In_ size_t file_idx,
_Outptr_ const ORTCHAR_T** out_path);

/** \brief Get session options for the specific model file within the selected variant as flat key/value entries.
*
* Returns NULL/empty arrays when none specified.
* Memory is owned by `ctx` and valid until the next model-package query call on that context.
*
* \since Version 1.27.
*/
ORT_API2_STATUS(ModelPackageComponent_GetSelectedVariantFileSessionOptions,
_In_ const OrtModelPackageComponentContext* ctx,
_In_ size_t file_idx,
_Outptr_result_buffer_maybenull_(*num_entries) const char* const** option_keys,
_Outptr_result_buffer_maybenull_(*num_entries) const char* const** option_values,
_Out_ size_t* num_entries);

/** \brief Get provider options for the specific model file within the selected variant as flat key/value entries.
*
* Returns NULL/empty arrays when none specified.
* Memory is owned by `ctx` and valid until the next model-package query call on that context.
*
* \since Version 1.27.
*/
ORT_API2_STATUS(ModelPackageComponent_GetSelectedVariantFileProviderOptions,
_In_ const OrtModelPackageComponentContext* ctx,
_In_ size_t file_idx,
_Outptr_result_buffer_maybenull_(*num_entries) const char* const** option_keys,
_Outptr_result_buffer_maybenull_(*num_entries) const char* const** option_values,
_Out_ size_t* num_entries);

/// @}
/** \brief Create an OrtSession for a selected file within a component model variant.
*
* The chosen variant (and thus its EP selection) is determined by `context`, which
* was built from an OrtSessionOptions via CreateModelPackageOptionsFromSessionOptions.
*
* Session options precedence:
* 1. session_options == NULL (default path):
* ORT uses the OrtSessionOptions that was captured when `context` was created.
* Any variant-specific session and provider options declared in the package
* metadata are merged on top.
*
* 2. session_options != NULL (advanced path):
* ORT uses the caller-provided OrtSessionOptions as-is. Variant-specific
* session and provider options from the package metadata are NOT applied.
* Use this when custom EP setup is required (e.g., shared CUDA streams,
* shared QNN EP contexts, custom allocators).
*
* \since Version 1.27.
*/
ORT_API2_STATUS(CreateSession,
_In_ const OrtEnv* env,
_In_ OrtModelPackageComponentContext* context,
_In_opt_ const OrtSessionOptions* session_options,
_Outptr_ OrtSession** session);

// End of Version 1.27 - DO NOT MODIFY ABOVE
};

/*
* This is the old way to add the CUDA provider to the session, please use SessionOptionsAppendExecutionProvider_CUDA above to access the latest functionality
* This function always exists, but will only succeed if Onnxruntime was built with CUDA support and the CUDA provider shared library exists
Expand Down
134 changes: 0 additions & 134 deletions onnxruntime/core/session/model_package/README.md

This file was deleted.

Loading
Loading