diff --git a/docs/model_package.md b/docs/model_package.md index c684be3b58..2b5676a4ce 100644 --- a/docs/model_package.md +++ b/docs/model_package.md @@ -12,60 +12,60 @@ referenced from each variant's configuration with a path scheme. A model package is a directory containing a top-level `manifest.json`. The package owns a single component that holds the variants onnxruntime-genai loads. By convention the -component is named `model`. The component directory under `models/` carries a -`metadata.json` enumerating the variants. Each variant has its own directory containing a -complete `genai_config.json` along with the ONNX graph files and any other per-variant -assets. +component is named `model`. The component is declared inline in the manifest, so its +variant directories sit directly at the package root. Each variant has its own directory +containing a complete `genai_config.json` along with the ONNX graph files and any other +per-variant assets. Files shared across variants (such as tokenizer assets) live in a +content-addressed `shared_assets/sha256-/` directory and are referenced with the +`sha256:` scheme. ``` my-model.ortpackage/ ├── manifest.json -├── models/ -│ └── model/ -│ ├── metadata.json -│ ├── openvino-1/ -│ │ ├── genai_config.json -│ │ ├── model.onnx -│ │ └── model.onnx.data -│ └── openvino-2/ -│ ├── genai_config.json -│ ├── model.onnx -│ └── model.onnx.data -└── shared/ - ├── tokenizer.json - └── processor_config.json +├── openvino-1/ +│ ├── genai_config.json +│ ├── model.onnx +│ └── model.onnx.data +├── openvino-2/ +│ ├── genai_config.json +│ ├── model.onnx +│ └── model.onnx.data +└── shared_assets/ + └── sha256-/ + ├── tokenizer.json + └── processor_config.json ``` -The shape of `manifest.json`, `metadata.json`, and the variant declarations follows the -ONNX Runtime model package specification. The minimum needed for onnxruntime-genai is: +The shape of `manifest.json` and the variant declarations follows the ONNX Runtime model +package specification. The minimum needed for onnxruntime-genai is a single inline +component named `model`: ```jsonc // manifest.json { - "schema_version": 1, - "components": ["model"] -} -``` - -```jsonc -// models/model/metadata.json -{ - "component_name": "model", - "variants": { - "openvino-1": { - "ep": "OpenVINOExecutionProvider", - "device": "npu", - "compatibility_string": "" - }, - "openvino-2": { - "ep": "OpenVINOExecutionProvider", - "device": "npu", - "compatibility_string": "" + "schema_version": "1.0", + "components": { + "model": { + "variants": { + "openvino-1": { + "ep": "OpenVINOExecutionProvider", + "device": "npu", + "compatibility_string": "" + }, + "openvino-2": { + "ep": "OpenVINOExecutionProvider", + "device": "npu", + "compatibility_string": "" + } + } } } } ``` +Each variant's directory defaults to its name (`openvino-1`, `openvino-2`) relative to the +package root; declare `variant_directory` on a variant to override that. + In this example both variants target the same execution provider but compile for different hardware or software versions. The OpenVINO EP scores each variant's `compatibility_string` at load time and ORT selects the highest-scoring match — callers do not need to know which build is best for their machine. @@ -73,30 +73,33 @@ not need to know which build is best for their machine. Each variant directory must contain a `genai_config.json` describing the model to onnxruntime-genai. Variant directories are otherwise self-contained: they may hold the ONNX graph, external weights, custom op libraries, LoRA adapters, and any other files -specific to that build. - -> **Heads-up.** External weight files (`model.onnx.data`) currently sit inside each -> variant directory. An upcoming ONNX Runtime change adds content-addressed shared -> assets, which will let multiple variants of the same model reference a single copy of -> the weights from the package's `shared_assets/` area. Packages authored under the -> conventions in this document migrate without genai-side changes: only the weight -> references inside the variant's `genai_config.json` change form. +specific to that build. Files that are identical across variants can instead live in a +content-addressed shared asset and be referenced with the `sha256:` scheme (see below), +which lets multiple variants share a single on-disk copy. ## Sharing tokenizer and processor files across variants Tokenizer assets and processor configuration are typically identical across variants of a -model. They can live in any sibling directory under the package root and be referenced -from each variant's `genai_config.json` using the `package:` path scheme. +model. Rather than duplicating them in every variant directory, store them once as a +content-addressed *shared asset* and reference them from each variant's `genai_config.json` +using the `sha256:` path scheme. + +A shared asset is a directory under the package's `shared_assets/` area named +`sha256-`, where `` is the asset's content digest. ONNX Runtime owns the +shared-asset model: it discovers `shared_assets/sha256-/` directories at load time and +can remap a digest to a custom or external directory through a `shared_assets` override in +the manifest. Referencing an asset by digest therefore resolves to the right location even +when the manifest overrides it. -Set `model.tokenizer_dir` to the directory holding `tokenizer.json` and friends. The -value uses the path-scheme syntax described below. +Set `model.tokenizer_dir` to the asset's `sha256:` URI. An optional `/`-separated tail +selects a subdirectory within the asset. ```jsonc -// models/model/openvino-1/genai_config.json +// openvino-1/genai_config.json { "model": { "type": "phi", - "tokenizer_dir": "package:shared", + "tokenizer_dir": "sha256:", ... }, "search": {} @@ -108,17 +111,18 @@ If `tokenizer_dir` is left unset the tokenizer files are looked up alongside Processor-specific configuration (`model.vision.config_filename`, `model.speech.config_filename`) is intentionally left per-variant: those files are small -and keeping them next to the ONNX graphs avoids needing the `package:` resolver for them. +and keeping them next to the ONNX graphs avoids needing a shared asset for them. ## Path scheme -`model.tokenizer_dir` accepts either a plain path or a scheme-prefixed value: +`model.tokenizer_dir` accepts either a plain path or a scheme-prefixed value. Resolution is +performed by ONNX Runtime's model package resolver: | Form | Resolution | | --- | --- | | *(empty)* | Treated as the variant directory (the directory containing `genai_config.json`). | -| `package:` | Joined with the package root. Only valid when loading from a model package. | -| Any other value | Joined with the variant directory, matching how every other path in `genai_config.json` is resolved. | +| `sha256:` or `sha256:/` | A content-addressed shared asset. Resolves to the asset's directory (honoring manifest `shared_assets` overrides), optionally joined with the confined tail. Only valid when loading from a model package. | +| Any other value | A relative path resolved against the variant directory, with portable-layout confinement (no absolute paths, no `..`). | ## Loading a package @@ -182,17 +186,17 @@ auto model_ov = OgaModel::Create(*config); ## Authoring notes -- Every variant declared in `metadata.json` must have its own directory containing a - `genai_config.json`. Variant directories are completely independent — they may differ - in any genai_config field, including `context_length`, tokenizer-related defaults, or - the set of ONNX graphs they load. +- Every variant declared in the component's `variants` map must have its own directory + containing a `genai_config.json`. Variant directories are completely independent — they + may differ in any genai_config field, including `context_length`, tokenizer-related + defaults, or the set of ONNX graphs they load. - Variants targeting the same execution provider must each declare a distinct `compatibility_string` (and typically a `device`) so the EP can score them against the local hardware. ONNX Runtime treats the string as opaque and forwards it to the EP's validator; the EP defines the syntax. -- Files outside variant directories (such as the `shared/` directory in the example - above) are not interpreted by the runtime. Reference them from - `genai_config.json` with the `package:` scheme to keep them shared across variants. +- Files shared across variants live in a content-addressed shared asset under + `shared_assets/sha256-/`. Reference them from `genai_config.json` with the `sha256:` + scheme so multiple variants share a single on-disk copy. - The model package's execution provider names must match what ONNX Runtime expects (`CPUExecutionProvider`, `CUDAExecutionProvider`, `DmlExecutionProvider`, `OpenVINOExecutionProvider`, ...). The short forms used by genai_config (`cuda`, `dml`, diff --git a/src/config.cpp b/src/config.cpp index bef34b9e74..388e64a0b9 100644 --- a/src/config.cpp +++ b/src/config.cpp @@ -1623,24 +1623,22 @@ void OverlayConfig(Config& config, std::string_view json) { JSON::Parse(element, json); } -namespace { - -constexpr std::string_view kPackageScheme = "package:"; - -} // namespace - fs::path Config::ResolvePath(std::string_view value) const { if (value.empty()) { return config_path; } - if (value.size() >= kPackageScheme.size() && - value.compare(0, kPackageScheme.size(), kPackageScheme) == 0) { - if (package_root.string().empty()) { - throw std::runtime_error("Cannot resolve \"" + std::string{value} + - "\": this model was not loaded from a model package."); - } - const std::string remainder{value.substr(kPackageScheme.size())}; - return remainder.empty() ? package_root : package_root / remainder; + if (package_resolver) { + // Loaded from a model package: ORT owns all path-reference resolution (sha256: shared + // assets with manifest overrides, relative paths, confinement). + return package_resolver(config_path, value); + } + // Flat directory: sha256: shared-asset references are only meaningful inside a model package. + constexpr std::string_view kSharedAssetPrefix = "sha256:"; + if (value.substr(0, kSharedAssetPrefix.size()) == kSharedAssetPrefix) { + throw std::runtime_error( + "\"" + std::string{value} + + "\" is a sha256: shared-asset reference, which is only valid when loading from a model " + "package; this model was loaded from a plain directory."); } return config_path / std::string{value}; } diff --git a/src/config.h b/src/config.h index e7c1628b8d..c55d4e6096 100644 --- a/src/config.h +++ b/src/config.h @@ -5,6 +5,8 @@ #pragma once #include "provider_options.h" +#include + namespace Generators { struct RuntimeSettings; @@ -88,9 +90,15 @@ struct Config { fs::path config_path; // Path of the config directory fs::path package_root; // Package root if loaded from a model package, otherwise empty. - // Resolves a path-like string from genai_config.json. Empty -> config_path. - // "package:" -> package_root/ (errors when package_root is empty). Anything - // else is joined with config_path. + // When loaded from a model package, resolves path-shaped genai_config.json values through + // ORT's package resolver: a "sha256:[/tail]" content-addressed shared-asset reference + // (honoring manifest overrides) or a plain relative path against base_dir. Empty for flat + // model directories. Captures the OrtModelPackageContext to keep it alive for resolution. + std::function package_resolver; + + // Resolves a path-like string from genai_config.json. Empty -> config_path. When loaded + // from a package, delegates to package_resolver (sha256: shared assets, relative paths); + // otherwise the value is joined with config_path. fs::path ResolvePath(std::string_view value) const; using NamedString = Generators::NamedString; diff --git a/src/models/model.cpp b/src/models/model.cpp index 06e7729d51..c33556a01a 100644 --- a/src/models/model.cpp +++ b/src/models/model.cpp @@ -52,6 +52,15 @@ namespace { constexpr const char* kOrtSessionOptionsModelExternalInitializersFileFolderPath = "session.model_external_initializers_file_folder_path"; +constexpr const char* kOrtSessionOptionEpContextFilePath = "ep.context_file_path"; + +// Session-option config keys whose values are file/folder path references. When a model is loaded +// from a package these may be sha256: shared-asset URIs or relative paths, so their values are +// resolved against the model root (Config::ResolvePath) before reaching ORT. +bool IsPathValuedSessionOption(std::string_view key) { + return key == kOrtSessionOptionsModelExternalInitializersFileFolderPath || + key == kOrtSessionOptionEpContextFilePath; +} } // namespace @@ -304,7 +313,7 @@ Tokenizer::Tokenizer(Config& config) : bos_token_id_{config.model.bos_token_id}, const char* keys[] = {"add_special_tokens", "skip_special_tokens"}; const char* values[] = {"false", "true"}; - // Resolve tokenizer_dir (may be empty, relative, absolute, or "package:"-scheme). + // Resolve tokenizer_dir (may be empty, relative, absolute, or a "sha256:" shared-asset reference). const fs::path tokenizer_dir = config.ResolvePath(config.model.tokenizer_dir); CheckResult(OrtxCreateTokenizerWithOptions(tokenizer_.Address(), tokenizer_dir.string().c_str(), keys, values, 2)); } @@ -664,7 +673,12 @@ void Model::CreateSessionOptionsFromConfig(const Config::SessionOptions& config_ * Reference: https://github.com/microsoft/onnxruntime/blob/main/include/onnxruntime/core/session/onnxruntime_session_options_config_keys.h */ for (auto& config_entry : config_session_options.config_entries) { - session_options.AddConfigEntry(config_entry.first.c_str(), config_entry.second.c_str()); + if (!config_entry.second.empty() && IsPathValuedSessionOption(config_entry.first)) { + const std::string resolved = config_->ResolvePath(config_entry.second).string(); + session_options.AddConfigEntry(config_entry.first.c_str(), resolved.c_str()); + } else { + session_options.AddConfigEntry(config_entry.first.c_str(), config_entry.second.c_str()); + } } // Register custom ops libraries only if explicitly configured @@ -783,11 +797,15 @@ std::unique_ptr Model::CreateSession(OrtEnv& ort_env, const std::str if (model_data_it->second.empty()) { throw std::runtime_error("Failed to load model data from memory for " + model_filename); } - // For models loaded from memory that reference external data files, tell ORT where to find them - // via the kOrtSessionOptionsModelExternalInitializersFileFolderPath session config entry. - const fs::path external_initializers_path = fs::absolute(config_->config_path); - session_options->AddConfigEntry(kOrtSessionOptionsModelExternalInitializersFileFolderPath, - external_initializers_path.string().c_str()); + // For models loaded from memory that reference external data files, ORT has no model + // directory to resolve them against. Default the external-initializers folder to the config + // directory, but only when the config did not already set it (a genai_config session option, + // resolved in CreateSessionOptionsFromConfig, takes precedence). + if (!session_options->HasConfigEntry(kOrtSessionOptionsModelExternalInitializersFileFolderPath)) { + const fs::path external_initializers_path = fs::absolute(config_->config_path); + session_options->AddConfigEntry(kOrtSessionOptionsModelExternalInitializersFileFolderPath, + external_initializers_path.string().c_str()); + } return OrtSession::Create(ort_env, model_data_it->second.data(), model_data_it->second.size(), session_options); } @@ -834,6 +852,14 @@ std::unique_ptr CreateConfig(OrtEnv& ort_env, const char* config_path, c auto load = OpenAndSelectVariant(ort_env, path, ep_str); auto config = std::make_unique(load.variant_dir, std::string_view{}); config->package_root = load.package_root; + // Delegate genai_config path resolution to ORT's package resolver, capturing the package + // context to keep it alive for the lifetime of the config. + auto package_context = load.context; + config->package_resolver = [package_context](const fs::path& base_dir, + std::string_view value) -> fs::path { + return fs::path{package_context->ResolveStringRef(base_dir.string(), std::string{value}, + /*must_exist=*/false)}; + }; return config; #else throw std::runtime_error( diff --git a/src/models/model_package.cpp b/src/models/model_package.cpp index df37f79ded..4e479642c4 100644 --- a/src/models/model_package.cpp +++ b/src/models/model_package.cpp @@ -72,7 +72,7 @@ bool IsModelPackage(const fs::path& path) { PackageLoadResult OpenAndSelectVariant(OrtEnv& env, const fs::path& package_root, const std::string& explicit_ep) { - auto pkg_ctx = OrtModelPackageContext::Create(package_root.c_str()); + std::shared_ptr pkg_ctx = OrtModelPackageContext::Create(package_root.c_str()); // Single-component restriction: the package author designates "the" genai component by // making it the only one. @@ -123,6 +123,8 @@ PackageLoadResult OpenAndSelectVariant(OrtEnv& env, PackageLoadResult result; result.package_root = package_root; result.variant_dir = fs::path{component_ctx->GetSelectedVariantFolderPath()}; + // Keep the package context alive so genai_config path references can be resolved later. + result.context = std::move(pkg_ctx); return result; } diff --git a/src/models/model_package.h b/src/models/model_package.h index b5f4cf7e95..a5a0c919cf 100644 --- a/src/models/model_package.h +++ b/src/models/model_package.h @@ -2,6 +2,7 @@ // Licensed under the MIT License. #pragma once +#include #include #include "../filesystem.h" @@ -16,6 +17,9 @@ bool IsModelPackage(const fs::path& path); struct PackageLoadResult { fs::path package_root; fs::path variant_dir; + // The opened package context, kept alive so genai_config path references (e.g. "sha256:" + // shared-asset refs) can be resolved on demand through ORT's package resolver. + std::shared_ptr context; }; // Opens a package and selects the variant for its single component. When `explicit_ep` is diff --git a/src/models/onnxruntime_api.h b/src/models/onnxruntime_api.h index 66990778f4..17dd637c41 100644 --- a/src/models/onnxruntime_api.h +++ b/src/models/onnxruntime_api.h @@ -661,6 +661,7 @@ struct OrtSessionOptions { OrtSessionOptions& DisablePerSessionThreads(); ///< Wraps OrtApi::DisablePerSessionThreads OrtSessionOptions& AddConfigEntry(const char* config_key, const char* config_value); ///< Wraps OrtApi::AddSessionConfigEntry + bool HasConfigEntry(const char* config_key) const; ///< Wraps OrtApi::HasSessionConfigEntry OrtSessionOptions& AddInitializer(const char* name, const OrtValue& ort_val); ///< Wraps OrtApi::AddInitializer OrtSessionOptions& AddExternalInitializers(const std::vector& names, const std::vector>& ort_values); ///< Wraps OrtApi::AddExternalInitializers @@ -1514,6 +1515,8 @@ struct ModelPackageApi { ModelPackage_GetVariantNames{nullptr}; OrtExperimental_OrtModelPackageApi_ModelPackage_GetVariantEpName_SinceV28_Fn ModelPackage_GetVariantEpName{nullptr}; + OrtExperimental_OrtModelPackageApi_ModelPackage_ResolveStringRef_SinceV28_Fn + ModelPackage_ResolveStringRef{nullptr}; OrtExperimental_OrtModelPackageApi_SelectComponent_SinceV28_Fn SelectComponent{nullptr}; OrtExperimental_OrtModelPackageApi_ReleaseModelPackageComponentContext_SinceV28_Fn @@ -1556,6 +1559,14 @@ struct OrtModelPackageContext { /// the variant does not declare an EP. std::string GetVariantEpName(const char* component_name, const char* variant_name) const; + /// Resolves a path reference from the package to an on-disk path using the model_package + /// rules: a "sha256:[/tail]" content-addressed shared-asset reference (honoring + /// manifest overrides), or a plain relative path resolved against `base_dir` (empty + /// base_dir falls back to the package root). When `must_exist` is true the resolved path + /// must exist on disk. + std::string ResolveStringRef(const std::string& base_dir, const std::string& input, + bool must_exist) const; + std::unique_ptr SelectComponent( const char* component_name, const OrtModelPackageOptions& options) const; diff --git a/src/models/onnxruntime_inline.h b/src/models/onnxruntime_inline.h index a25e618a5d..eeb4ccff54 100644 --- a/src/models/onnxruntime_inline.h +++ b/src/models/onnxruntime_inline.h @@ -680,6 +680,12 @@ inline OrtSessionOptions& OrtSessionOptions::AddConfigEntry(const char* config_k return *this; } +inline bool OrtSessionOptions::HasConfigEntry(const char* config_key) const { + int out = 0; + Ort::ThrowOnError(Ort::api->HasSessionConfigEntry(this, config_key, &out)); + return out != 0; +} + inline OrtSessionOptions& OrtSessionOptions::AddInitializer(const char* name, const OrtValue& ort_val) { Ort::ThrowOnError(Ort::api->AddInitializer(this, name, &ort_val)); return *this; @@ -1560,6 +1566,7 @@ inline const ModelPackageApi& GetModelPackageApi() { f.ModelPackage_GetVariantCount = GENAI_MP_V28_FN(ModelPackage_GetVariantCount); f.ModelPackage_GetVariantNames = GENAI_MP_V28_FN(ModelPackage_GetVariantNames); f.ModelPackage_GetVariantEpName = GENAI_MP_V28_FN(ModelPackage_GetVariantEpName); + f.ModelPackage_ResolveStringRef = GENAI_MP_V28_FN(ModelPackage_ResolveStringRef); f.SelectComponent = GENAI_MP_V28_FN(SelectComponent); f.ReleaseModelPackageComponentContext = GENAI_MP_V28_FN(ReleaseModelPackageComponentContext); f.ModelPackageComponent_GetSelectedVariantFolderPath = @@ -1622,6 +1629,15 @@ inline std::string OrtModelPackageContext::GetVariantEpName(const char* componen return (ep == nullptr) ? std::string{} : std::string{ep}; } +inline std::string OrtModelPackageContext::ResolveStringRef(const std::string& base_dir, + const std::string& input, + bool must_exist) const { + const char* resolved = nullptr; + Ort::ThrowOnError(Ort::GetModelPackageApi().ModelPackage_ResolveStringRef( + this, base_dir.empty() ? nullptr : base_dir.c_str(), input.c_str(), must_exist ? 1 : 0, &resolved)); + return (resolved == nullptr) ? std::string{} : std::string{resolved}; +} + inline std::unique_ptr OrtModelPackageContext::SelectComponent( const char* component_name, const OrtModelPackageOptions& options) const { OrtModelPackageComponentContext* p = nullptr; diff --git a/test/model_package_test.cpp b/test/model_package_test.cpp index 12a78a7903..33c0e3b9c7 100644 --- a/test/model_package_test.cpp +++ b/test/model_package_test.cpp @@ -55,23 +55,27 @@ struct VariantSpec { }; // Builds a minimal model package the ORT model_package API can open: a single "model" -// component whose variants each get a placeholder model.onnx and, optionally, a config. +// component declared inline in the manifest, whose variants each get a placeholder +// model.onnx and, optionally, a config. The component is inline so the variant directories +// live at the package root with no models/ nesting. fs_std::path WritePackage(const std::string& suffix, const std::vector& variants) { const auto root = MakeTempDir(suffix); - WriteFile(root / "manifest.json", - "{ \"schema_version\": 1, \"components\": [\"model\"] }"); - - std::string metadata = "{\n \"component_name\": \"model\",\n \"variants\": {\n"; + // The single "model" component is declared inline in the manifest, so its variant + // directories sit at the package root: variant_directory defaults to the variant name + // relative to the component directory, which is the package root for an inline component. + // No models/ nesting and no separate component.json — everything stays at the top level. + std::string manifest = + "{\n \"schema_version\": \"1.0\",\n \"components\": {\n \"model\": {\n \"variants\": {\n"; for (size_t i = 0; i < variants.size(); ++i) { - metadata += " \"" + variants[i].name + "\": { \"ep\": \"" + variants[i].ep + "\" }"; - metadata += (i + 1 == variants.size()) ? "\n" : ",\n"; + manifest += " \"" + variants[i].name + "\": { \"ep\": \"" + variants[i].ep + "\" }"; + manifest += (i + 1 == variants.size()) ? "\n" : ",\n"; } - metadata += " }\n}\n"; - WriteFile(root / "models" / "model" / "metadata.json", metadata); + manifest += " }\n }\n }\n}\n"; + WriteFile(root / "manifest.json", manifest); for (const auto& variant : variants) { - const auto variant_dir = root / "models" / "model" / variant.name; + const auto variant_dir = root / variant.name; WriteFile(variant_dir / "model.onnx", "placeholder"); if (variant.valid_config) { WriteFile(variant_dir / "genai_config.json", @@ -94,6 +98,29 @@ std::string CaptureThrowMessage(Fn&& fn) { return {}; } +// Produces an ONNX model whose initializers live in a sidecar external-data file, by loading +// `src_onnx` with optimizations disabled and exporting an optimized copy with external +// initializers. Writes /model.onnx plus /model.onnx.data and returns the +// model path. Used to exercise the external-initializers folder session option. +fs_std::path ExportModelWithExternalData(const fs_std::path& src_onnx, const fs_std::path& out_dir) { + // The low-level ORT wrappers below run in this test binary, which has its own copy of the + // Ort::api pointer (the genai .so initializes its own). Initialize ours before using them. + Ort::InitApi(); + fs_std::create_directories(out_dir); + const auto model_path = out_dir / "model.onnx"; + auto session_options = OrtSessionOptions::Create(); + session_options->SetGraphOptimizationLevel(ORT_DISABLE_ALL); + session_options->SetOptimizedModelFilePath(model_path.c_str()); + session_options->AddConfigEntry("session.optimized_model_external_initializers_file_name", + "model.onnx.data"); + session_options->AddConfigEntry("session.optimized_model_external_initializers_min_size_in_bytes", + "0"); + // Creating the session writes the optimized model + external data as a side effect. + auto env = OrtEnv::Create(); + OrtSession::Create(*env, src_onnx.c_str(), session_options.get()); + return model_path; +} + } // namespace TEST(ModelPackage, RejectsFlatDirectory) { @@ -150,23 +177,28 @@ TEST(ModelPackage, AcceptsFullEpName) { EXPECT_NO_THROW(OgaConfig::CreateFromPackageEp(root.string().c_str(), "CPUExecutionProvider")); } -TEST(ModelPackage, TokenizerResolvesThroughPackageRoot) { - // End-to-end: a real model lives in the cpu variant while its tokenizer lives in the - // package's shared/ directory, referenced via a "package:" tokenizer_dir. Loading the - // model and tokenizing exercises package_root resolution through the public API. +TEST(ModelPackage, TokenizerResolvesThroughSharedAsset) { + // End-to-end: a real model lives in the cpu variant while its tokenizer lives in a + // content-addressed shared asset, referenced via a "sha256:" tokenizer_dir. ORT discovers + // the shared_assets/sha256-/ directory at load time; loading the model and tokenizing + // exercises sha256: resolution through the public API. const fs_std::path src_model = fs_std::path(MODEL_PATH) / "hf-internal-testing" / "tiny-random-gpt2-fp32"; const auto root = MakeTempDir("e2e_pkg"); - const auto variant_dir = root / "models" / "model" / "cpu"; + const auto variant_dir = root / "cpu"; + // A valid sha256 URI is "sha256:" + 64 lowercase hex chars; the on-disk asset directory is + // shared_assets/sha256-/. The bytes need not match the digest: ORT discovers the + // directory by name, it does not re-hash the contents at load time. + const std::string digest(64, 'a'); + const auto asset_dir = root / "shared_assets" / ("sha256-" + digest); fs_std::create_directories(variant_dir); - fs_std::create_directories(root / "shared"); + fs_std::create_directories(asset_dir); + // Inline single "model" component: the cpu variant directory sits at the package root. WriteFile(root / "manifest.json", - "{ \"schema_version\": 1, \"components\": [\"model\"] }"); - WriteFile(root / "models" / "model" / "metadata.json", - "{ \"component_name\": \"model\"," - " \"variants\": { \"cpu\": { \"ep\": \"CPUExecutionProvider\" } } }"); + "{ \"schema_version\": \"1.0\", \"components\": { \"model\": { \"variants\":" + " { \"cpu\": { \"ep\": \"CPUExecutionProvider\" } } } } }"); // The model file stays beside genai_config.json in the variant directory. std::error_code ec; @@ -174,21 +206,21 @@ TEST(ModelPackage, TokenizerResolvesThroughPackageRoot) { fs_std::copy_options::overwrite_existing, ec); ASSERT_FALSE(ec) << ec.message(); - // The tokenizer files move to the package-level shared/ directory. + // The tokenizer files move to the content-addressed shared asset directory. for (const auto& entry : fs_std::directory_iterator(src_model)) { const auto name = entry.path().filename().string(); if (name == "past.onnx" || name == "genai_config.json") continue; - fs_std::copy_file(entry.path(), root / "shared" / name, + fs_std::copy_file(entry.path(), asset_dir / name, fs_std::copy_options::overwrite_existing, ec); ASSERT_FALSE(ec) << ec.message(); } - // Point tokenizer_dir at the shared/ directory using the "package:" scheme. + // Point tokenizer_dir at the shared asset using the "sha256:" scheme. std::string config = ReadFile(src_model / "genai_config.json"); const std::string anchor = "\"type\": \"gpt2\","; const auto pos = config.find(anchor); ASSERT_NE(pos, std::string::npos); - config.insert(pos + anchor.size(), "\n \"tokenizer_dir\": \"package:shared\","); + config.insert(pos + anchor.size(), "\n \"tokenizer_dir\": \"sha256:" + digest + "\","); WriteFile(variant_dir / "genai_config.json", config); auto oga_config = OgaConfig::CreateFromPackageEp(root.string().c_str(), "cpu"); @@ -201,4 +233,102 @@ TEST(ModelPackage, TokenizerResolvesThroughPackageRoot) { EXPECT_GT(sequences->SequenceCount(0), 0u); } +TEST(ModelPackage, ExternalInitializersFolderResolvesThroughSharedAsset) { + // A variant's model keeps its weights in a sidecar external-data file that lives in a + // content-addressed shared asset, referenced from the genai_config session options via + // "session.model_external_initializers_file_folder_path": "sha256:". The model loads only + // if genai resolves that folder through the package (relative/sha256: -> absolute) and hands it + // to ORT; otherwise ORT looks beside the model file and the weights are missing. + const fs_std::path src_model = fs_std::path(MODEL_PATH) / "hf-internal-testing" / + "tiny-random-gpt2-fp32"; + + const auto root = MakeTempDir("e2e_extdata"); + const auto variant_dir = root / "cpu"; + const std::string digest(64, 'b'); + const auto asset_dir = root / "shared_assets" / ("sha256-" + digest); + fs_std::create_directories(variant_dir); + fs_std::create_directories(asset_dir); + + WriteFile(root / "manifest.json", + "{ \"schema_version\": \"1.0\", \"components\": { \"model\": { \"variants\":" + " { \"cpu\": { \"ep\": \"CPUExecutionProvider\" } } } } }"); + + // Build an external-data model in the variant dir, then move only its weights blob into the + // shared asset so it can be found solely through the resolved folder option. + ExportModelWithExternalData(src_model / "past.onnx", variant_dir); + std::error_code ec; + fs_std::rename(variant_dir / "model.onnx.data", asset_dir / "model.onnx.data", ec); + ASSERT_FALSE(ec) << ec.message(); + + // Tokenizer files sit beside the model so the genai model directory is complete. + for (const auto& entry : fs_std::directory_iterator(src_model)) { + const auto name = entry.path().filename().string(); + if (name == "past.onnx" || name == "genai_config.json") continue; + fs_std::copy_file(entry.path(), variant_dir / name, + fs_std::copy_options::overwrite_existing, ec); + ASSERT_FALSE(ec) << ec.message(); + } + + // Author a config pointing the decoder at the external-data model, optionally referencing the + // shared-asset weights folder by sha256: URI. + auto write_config = [&](bool with_folder_option) { + std::string config = ReadFile(src_model / "genai_config.json"); + const auto file_pos = config.find("past.onnx"); + ASSERT_NE(file_pos, std::string::npos); + config.replace(file_pos, std::string("past.onnx").size(), "model.onnx"); + if (with_folder_option) { + const std::string anchor = "\"session_options\": {"; + const auto pos = config.find(anchor); + ASSERT_NE(pos, std::string::npos); + config.insert(pos + anchor.size(), + "\n \"session.model_external_initializers_file_folder_path\": \"sha256:" + + digest + "\","); + } + WriteFile(variant_dir / "genai_config.json", config); + }; + + // With the folder option, the weights resolve through the shared asset and the model loads. + write_config(/*with_folder_option=*/true); + EXPECT_NO_THROW({ + auto oga_config = OgaConfig::CreateFromPackageEp(root.string().c_str(), "cpu"); + auto model = OgaModel::Create(*oga_config); + }); + + // Without it, ORT looks beside model.onnx, the weights blob is absent, and loading fails. + write_config(/*with_folder_option=*/false); + const std::string message = CaptureThrowMessage([&] { + auto oga_config = OgaConfig::CreateFromPackageEp(root.string().c_str(), "cpu"); + auto model = OgaModel::Create(*oga_config); + }); + EXPECT_FALSE(message.empty()); +} + +TEST(ModelPackage, FlatDirectoryRejectsSha256Reference) { + // sha256: references only resolve inside a package. In a plain directory, resolving one must + // fail with a clear error instead of silently producing a bogus "/sha256:" path. + const fs_std::path src_model = fs_std::path(MODEL_PATH) / "hf-internal-testing" / + "tiny-random-gpt2-fp32"; + const auto dir = MakeTempDir("flat_sha256"); + std::error_code ec; + for (const auto& entry : fs_std::directory_iterator(src_model)) { + fs_std::copy_file(entry.path(), dir / entry.path().filename().string(), + fs_std::copy_options::overwrite_existing, ec); + ASSERT_FALSE(ec) << ec.message(); + } + + std::string config = ReadFile(src_model / "genai_config.json"); + const std::string anchor = "\"type\": \"gpt2\","; + const auto pos = config.find(anchor); + ASSERT_NE(pos, std::string::npos); + config.insert(pos + anchor.size(), + "\n \"tokenizer_dir\": \"sha256:" + std::string(64, 'a') + "\","); + WriteFile(dir / "genai_config.json", config); + + const std::string message = CaptureThrowMessage([&] { + auto model = OgaModel::Create(dir.string().c_str()); + auto tokenizer = OgaTokenizer::Create(*model); + }); + EXPECT_NE(message.find("sha256:"), std::string::npos) << message; +} + #endif // ORT_GENAI_HAS_MODEL_PACKAGE