Update ort and ort-genai package

.pipelines/foundry-local-packaging.yml

@@ -56,9 +56,9 @@ variables:
 # C++ SDK (sdk_v2/cpp) native dependency versions. Must match cmake defaults
 # in sdk_v2/deps_versions.json.
 - name: cppOrtVersion
-  value: '1.26.0'
+  value: '1.28.0'
 - name: cppGenaiVersion
-  value: '0.14.1'
+  value: '0.15.1'
 - name: cppWinmlVersion
   value: '2.1.70'
 - name: cppBuildConfig

.pipelines/v2/sdk_v2-js-pipeline-plan.md

@@ -114,7 +114,7 @@ matrix.
 Build and test on the `onnxruntime-linux-ARM64-CPU-2019` pool
 (`hostArchitecture: arm64`). Same `node-gyp rebuild` path as Linux x64.
 `install-native.cjs` selects the CPU-only ORT NuGet package
-(`Microsoft.ML.OnnxRuntime.Foundry`) for `linux-arm64` — matching the
+(`Microsoft.ML.OnnxRuntime`) for `linux-arm64` — matching the
 C++ native pipeline which also uses CPU-only ORT for ARM64.
 
 ## Test stage details
@@ -220,7 +220,7 @@ shape.
 - **Single combined tarball:** `js_pack` assembles all five prebuilds
   into one `foundry-local-sdk-<version>.tgz`.
 - **JS scoped out of WinML.**
-- **Linux ARM64 ORT package:** `Microsoft.ML.OnnxRuntime.Foundry` (CPU-only),
+- **Linux ARM64 ORT package:** `Microsoft.ML.OnnxRuntime` (CPU-only),
   matching the C++ native ARM64 pipeline.
 
 ## Open items

.pipelines/v2/sdk_v2-pipeline-plan.md

@@ -31,7 +31,7 @@ are gated separately via `.pipelines/v1/templates/stages-sdk-v1.yml`.
 |-----------------|-----------------------------------|-------|------|-----------------------------------------|
 | Windows x64     | `onnxruntime-Win-CPU-2022`        | ✅    | ✅   | Also stages public headers              |
 | Windows ARM64   | `onnxruntime-Win-CPU-2022`        | ✅    | ❌   | Cross-compiled from x64 host            |
-| Linux x64       | `onnxruntime-Ubuntu2404-AMD-CPU`  | ✅    | ✅   | Pulls extra `OnnxRuntime.Gpu.Linux` pkg |
+| Linux x64       | `onnxruntime-Ubuntu2404-AMD-CPU`  | ✅    | ✅   | Native CPU-only build                   |
 | Linux ARM64     | `onnxruntime-linux-ARM64-CPU-2019`| ✅    | ✅   | Native CPU-only build                   |
 | macOS ARM64     | `AcesShared` (Sequoia)            | ✅    | ✅   | Native                                  |
 
@@ -85,15 +85,8 @@ are gated separately via `.pipelines/v1/templates/stages-sdk-v1.yml`.
 
    Bumping ORT/GenAI is a one-file edit.
 9. **ORT/GenAI come from public PyPI.** No private feed plumbing required
-   for the wheel install path:
-   - `onnxruntime-core` (Windows/macOS)
-   - `onnxruntime-genai-core` (Windows/macOS)
-   - `onnxruntime-gpu` / `onnxruntime-genai-cuda` (Linux x64)
-   - `onnxruntime` / `onnxruntime-genai` (Linux ARM64, CPU-only)
-
-   Import-name mapping is platform-dependent: Linux uses `onnxruntime` /
-   `onnxruntime_genai`; Windows/macOS use `onnxruntime_core` /
-   `onnxruntime_genai_core`.
+   for the wheel install path. Every platform uses `onnxruntime` and
+   `onnxruntime-genai-core` (`onnxruntime` and `onnxruntime_genai_core` imports).
 10. **C++ staging step is the policy authority for native payload contents.**
     `steps-build-{windows,linux,macos}.yml` stage the **full runtime closure**
     of `foundry_local` into the `cpp-native-<rid>` artifact, with explicit
@@ -117,13 +110,9 @@ are gated separately via `.pipelines/v1/templates/stages-sdk-v1.yml`.
 
     Each step fails loudly if its primary library is missing.
 11. **Python runtime ORT discovery.** `lib_loader.py::prepare_native_dependencies()`
-    bridges between the in-wheel `foundry_local` and the pip-installed ORT
-    packages:
-    - **Windows:** `os.add_dll_directory(...)` for each ORT package directory.
-    - **Linux/macOS:** create symlinks
-      `_native/<rid>/{onnxruntime,onnxruntime-genai}.{so,dylib}` pointing at
-      the package-installed `lib*` files (workaround for
-      [onnxruntime#27263](https://github.com/microsoft/onnxruntime/issues/27263)).
+    preloads ORT and GenAI by absolute path. Windows also registers their DLL
+    directories; macOS creates the unversioned ORT symlink required by GenAI
+    ([onnxruntime#27263](https://github.com/microsoft/onnxruntime/issues/27263)).
 
     Wired into `_native/api.py` between `find_library()` and the cffi
     extension import. Idempotent and silent on failure.
@@ -247,16 +236,16 @@ purposes:
 
 1. **Version pinning** — the `KEY=PATH` pairs are passed via
    `--cmake_extra_defines` (`ORT_FETCH_URL`, `GENAI_FETCH_URL`,
-   `WINML_EP_CATALOG_FETCH_URL`, `ORT_GPU_LINUX_FETCH_URL`) so the cmake
-   defaults in `FindOnnxRuntime.cmake` / `FindOnnxRuntimeGenAI.cmake` are
-   never silently substituted.
+   `WINML_EP_CATALOG_FETCH_URL`) so the cmake defaults in
+   `FindOnnxRuntime.cmake` / `FindOnnxRuntimeGenAI.cmake` are never
+   silently substituted.
 2. **Stage isolation** — the build step no longer needs network access to
    the package feed once prefetch has completed.
 
 Versions are pipeline-level variables, currently:
 
-* `ortVersion`        `1.26.0`   (`Microsoft.ML.OnnxRuntime.Foundry`)
-* `genaiVersion`      `0.14.1`   (`Microsoft.ML.OnnxRuntimeGenAI.Foundry`)
+* `ortVersion`        `1.28.0`   (`Microsoft.ML.OnnxRuntime`)
+* `genaiVersion`      `0.15.1`   (`Microsoft.ML.OnnxRuntimeGenAI.Foundry`)
 * `winmlVersion`      `2.1.70`    (`Microsoft.Windows.AI.MachineLearning`, WinML 2.x reg-free)
 
 These must be kept in sync with the cmake defaults and with

.pipelines/v2/templates/steps-build-linux.yml

@@ -4,8 +4,7 @@
 # linux-arm64 (pool: onnxruntime-linux-ARM64-CPU-2019) via the `arch`
 # parameter. Each pool has the toolchain pre-installed; we still bootstrap
 # vcpkg and pre-download NuGet packages from the aiinfra feed for version
-# pinning. x64 also pre-downloads Microsoft.ML.OnnxRuntime.Gpu.Linux for
-# CUDA support; arm64 is CPU-only and skips that package.
+# pinning.
 
 parameters:
 - name: arch
@@ -36,7 +35,6 @@ steps:
     genaiVersion: ${{ parameters.genaiVersion }}
     winmlVersion: ''
     includeWinml: false
-    includeOrtGpuLinux: ${{ eq(parameters.arch, 'x64') }}
     shell: bash
 
 # Bake the pipeline-computed version into the binary so FoundryLocalGetVersionString()

.pipelines/v2/templates/steps-build-python.yml

@@ -121,10 +121,8 @@ steps:
         #   - Windows:  foundry_local.{dll,pdb,lib}
         #   - Linux:    libfoundry_local.so
         #   - macOS:    libfoundry_local.dylib
-        # ORT/GenAI native libs are intentionally absent — those come from pip
-        # deps (onnxruntime-{core,gpu} / onnxruntime-genai-{core,cuda}). Test
-        # and example binaries are also filtered out upstream. We just copy
-        # the curated artifact contents straight in.
+        # ORT/GenAI come from pip dependencies; this artifact contains only the
+        # curated Foundry Local native files.
         $files = Get-ChildItem -Path "${{ parameters.nativeArtifactDir }}" -Recurse -File
         if ($files.Count -eq 0) {
             throw "No native artifacts found under ${{ parameters.nativeArtifactDir }}"
@@ -135,12 +133,8 @@ steps:
         Write-Host "Copied $($files.Count) native file(s) to $dest"
         Get-ChildItem $dest | ForEach-Object { Write-Host "  $($_.Name)" }
 
-# Note: libfoundry_local.{so,dylib} already has $ORIGIN / @loader_path baked
-# into its rpath at CMake build time (see sdk_v2/cpp/CMakeLists.txt's RPATH
-# block on the foundry_local target). No patchelf / install_name_tool fixup
-# is needed here — the binary will look for ORT/GenAI deps next to itself
-# in _native/<rid>/, which is where lib_loader._create_ort_symlinks drops
-# them at process start.
+# lib_loader.py preloads the pip-installed ORT and GenAI libraries, so no
+# patchelf or install_name_tool fixup is needed.
 
 - task: PowerShell@2
   displayName: 'Clean stale build artifacts'

.pipelines/v2/templates/steps-build-windows.yml

@@ -7,7 +7,7 @@
 # Parameters:
 #   arch           – 'x64' or 'arm64'
 #   buildConfig    – CMake config (Debug, Release, RelWithDebInfo, MinSizeRel)
-#   ortVersion     – Microsoft.ML.OnnxRuntime.Foundry version
+#   ortVersion     – Microsoft.ML.OnnxRuntime version
 #   genaiVersion   – Microsoft.ML.OnnxRuntimeGenAI.Foundry version
 #   winmlVersion   – Microsoft.Windows.AI.MachineLearning version
 #   runTests       – Whether to run tests
@@ -207,7 +207,7 @@ steps:
 # The WinML DLL is delay-loaded (see /DELAYLOAD in CMakeLists.txt) so it is
 # NOT needed at foundry_local.dll load time, but the cmake post-build copy
 # stages it next to foundry_local.dll for runtime EP discovery. ORT/GenAI
-# come from the onnxruntime-core / onnxruntime-genai-core pip deps.
+# come from the onnxruntime / onnxruntime-genai-core pip deps.
 - task: PowerShell@2
   displayName: 'Stage native artifacts'
   inputs:

.pipelines/v2/templates/steps-prefetch-nuget.yml

@@ -5,11 +5,10 @@
 # `KEY=PATH` pairs to pass to build.py --cmake_extra_defines.
 #
 # Parameters:
-#   ortVersion     – Microsoft.ML.OnnxRuntime.Foundry version
+#   ortVersion     – Microsoft.ML.OnnxRuntime version
 #   genaiVersion   – Microsoft.ML.OnnxRuntimeGenAI.Foundry version
 #   winmlVersion   – Microsoft.Windows.AI.MachineLearning version (Windows only)
 #   includeWinml   – Download WinML and emit WINML_EP_CATALOG_FETCH_URL
-#   includeOrtGpuLinux – Also download Microsoft.ML.OnnxRuntime.Gpu.Linux (Linux only)
 #   shell          – 'pwsh' (Windows/macOS) or 'bash' (Linux)
 #   usePwsh        – Whether PowerShell tasks should use PowerShell Core
 
@@ -24,9 +23,6 @@ parameters:
 - name: includeWinml
   type: boolean
   default: false
-- name: includeOrtGpuLinux
-  type: boolean
-  default: false
 - name: shell
   type: string
   values: ['pwsh', 'bash']
@@ -80,19 +76,16 @@ steps:
         $cacheDir = "$(Build.BinariesDirectory)/nuget_packages"
         New-Item -ItemType Directory -Force -Path $cacheDir | Out-Null
 
-        # All four packages are public on nuget.org. Foundry Local Core's
+        # These packages are public on nuget.org. Foundry Local Core's
         # nuget.config maps everything except Microsoft.Telemetry* to nuget.org
         # (see .pipelines/templates/build-core-steps.yml), so we follow the
         # same source of truth here.
         $feed = "https://www.nuget.org/api/v2/package"
 
         $packages = @(
           @{ key = 'genai'; id = 'Microsoft.ML.OnnxRuntimeGenAI.Foundry'; version = '${{ parameters.genaiVersion }}' },
-          @{ key = 'ort';   id = 'Microsoft.ML.OnnxRuntime.Foundry';     version = '${{ parameters.ortVersion }}'   }
+          @{ key = 'ort';   id = 'Microsoft.ML.OnnxRuntime';             version = '${{ parameters.ortVersion }}'   }
         )
-        if ($${{ parameters.includeOrtGpuLinux }}) {
-          $packages += @{ key = 'ort_gpu_linux'; id = 'Microsoft.ML.OnnxRuntime.Gpu.Linux'; version = '${{ parameters.ortVersion }}' }
-        }
 
         $defines = @()
         foreach ($pkg in $packages) {
@@ -104,9 +97,8 @@ steps:
           Write-Host "  -> $out ($((Get-Item $out).Length) bytes)"
 
           switch ($pkg.key) {
-            'genai'         { $defines += "GENAI_FETCH_URL=$out" }
-            'ort'           { $defines += "ORT_FETCH_URL=$out" }
-            'ort_gpu_linux' { $defines += "ORT_GPU_LINUX_FETCH_URL=$out" }
+            'genai' { $defines += "GENAI_FETCH_URL=$out" }
+            'ort'   { $defines += "ORT_FETCH_URL=$out" }
           }
         }
 
@@ -132,25 +124,21 @@ steps:
       cacheDir="$(Build.BinariesDirectory)/nuget_packages"
       mkdir -p "$cacheDir"
 
-      # All four packages are public on nuget.org. Foundry Local Core's
+      # These packages are public on nuget.org. Foundry Local Core's
       # nuget.config maps everything except Microsoft.Telemetry* to nuget.org
       # (see .pipelines/templates/build-core-steps.yml), so we follow the
       # same source of truth here.
       feed="https://www.nuget.org/api/v2/package"
 
       declare -a entries=(
         "genai:Microsoft.ML.OnnxRuntimeGenAI.Foundry:${{ parameters.genaiVersion }}"
-        "ort:Microsoft.ML.OnnxRuntime.Foundry:${{ parameters.ortVersion }}"
+        "ort:Microsoft.ML.OnnxRuntime:${{ parameters.ortVersion }}"
       )
       if [ "${{ parameters.includeWinml }}" = "True" ]; then
         # WinML is Windows-only; the bash branch should never receive includeWinml=true.
         echo "ERROR: includeWinml=true is not supported on the bash prefetch branch (WinML is Windows-only)." >&2
         exit 1
       fi
-      if [ "${{ parameters.includeOrtGpuLinux }}" = "True" ]; then
-        entries+=("ort_gpu_linux:Microsoft.ML.OnnxRuntime.Gpu.Linux:${{ parameters.ortVersion }}")
-      fi
-
       defines=()
       for entry in "${entries[@]}"; do
         IFS=: read -r key id version <<< "$entry"
@@ -161,9 +149,8 @@ steps:
         echo "  -> $out ($(stat -c%s "$out" 2>/dev/null || stat -f%z "$out") bytes)"
 
         case "$key" in
-          genai)         defines+=("\"GENAI_FETCH_URL=$out\"") ;;
-          ort)           defines+=("\"ORT_FETCH_URL=$out\"") ;;
-          ort_gpu_linux) defines+=("\"ORT_GPU_LINUX_FETCH_URL=$out\"") ;;
+          genai) defines+=("\"GENAI_FETCH_URL=$out\"") ;;
+          ort)   defines+=("\"ORT_FETCH_URL=$out\"") ;;
         esac
       done
 

.pipelines/v2/templates/steps-test-python.yml

@@ -135,6 +135,10 @@ steps:
         & "$(venvPy)" -m pip install --upgrade pip
         & "$(venvPy)" -m pip install $wheel
         if ($LASTEXITCODE -ne 0) { exit $LASTEXITCODE }
+        & "$(venvPy)" -c "import importlib.util; assert importlib.util.find_spec('onnxruntime'); assert importlib.util.find_spec('onnxruntime_genai_core')"
+        if ($LASTEXITCODE -ne 0) {
+            throw "Wheel install did not resolve universal onnxruntime / onnxruntime-genai-core dependencies"
+        }
         & "$(venvPy)" -m pip install pytest pytest-cov
         if ($LASTEXITCODE -ne 0) { exit $LASTEXITCODE }
 

sdk_v2/cpp/CMakeLists.txt

@@ -169,6 +169,7 @@ set(FOUNDRY_LOCAL_SOURCES
     src/inferencing/generative/genai_config.cc
     src/http/http_client.cc
     src/inferencing/generative/genai_model_instance.cc
+    src/inferencing/generative/tokenizer.cc
     src/logger.cc
     src/spdlog_logger.cc
     src/model_info.cc

sdk_v2/cpp/nuget/Microsoft.AI.Foundry.Local.Runtime.nuspec

@@ -18,7 +18,7 @@
 
     <dependencies>
       <group>
-        <dependency id="Microsoft.ML.OnnxRuntime.Foundry" version="$ort_version$" />
+        <dependency id="Microsoft.ML.OnnxRuntime" version="$ort_version$" />
         <dependency id="Microsoft.ML.OnnxRuntimeGenAI.Foundry" version="$genai_version$" />
       </group>
     </dependencies>

sdk_v2/cpp/nuget/pack.py

@@ -74,7 +74,7 @@ def _parse_args() -> argparse.Namespace:
     parser.add_argument("--version", required=True,
                         help="Package version (e.g. 0.1.0 or 0.1.0-dev.20260419).")
     parser.add_argument("--ort_version", required=True,
-                        help="Minimum Microsoft.ML.OnnxRuntime.Foundry version.")
+                        help="Minimum Microsoft.ML.OnnxRuntime version.")
     parser.add_argument("--genai_version", required=True,
                         help="Minimum Microsoft.ML.OnnxRuntimeGenAI.Foundry version.")
     parser.add_argument("--package_id", default="Microsoft.AI.Foundry.Local.Runtime",

sdk_v2/cpp/src/inferencing/generative/audio/audio_session.cc

@@ -284,7 +284,7 @@ void AudioSession::ProcessStreamingAudio(const AudioItem& format_item, ItemQueue
   }
 
   auto generator = OgaGenerator::Create(oga_model, *gen_params);
-  auto tokenizer_stream = OgaTokenizerStream::Create(Model().GetOgaTokenizer());
+  auto tokenizer_stream = OgaTokenizerStream::Create(Model().Tokenizer().Oga());
 
   auto streaming_callback = CreateCallbackHandler(request);
   std::vector<std::string> token_texts;

sdk_v2/cpp/src/inferencing/generative/audio/onnx_audio_generator.cc

@@ -176,7 +176,7 @@ std::unique_ptr<OnnxAudioGenerator> OnnxAudioGenerator::Create(const std::string
   int prompt_token_count = static_cast<int>(generator->GetSequenceCount(0));
 
   // 7. Create tokenizer stream for decoding (no special-token stream needed for audio)
-  auto stream = OgaTokenizerStream::Create(model.GetOgaTokenizer());
+  auto stream = OgaTokenizerStream::Create(model.Tokenizer().Oga());
 
   // `std::make_unique` cannot access the private constructor, so use `new` directly.
   return std::unique_ptr<OnnxAudioGenerator>(new OnnxAudioGenerator(std::move(audios),

sdk_v2/cpp/src/inferencing/generative/chat/chat_template.cc

@@ -2,6 +2,7 @@
 // Licensed under the MIT License.
 #include "inferencing/generative/chat/chat_template.h"
 #include "exception.h"
+#include "inferencing/generative/genai_model_instance.h"
 #include "items/message_item.h"
 #include "items/text_item.h"
 #include "utils.h"
@@ -41,13 +42,13 @@ std::string RenderMessageForPrompt(const MessageItem& msg) {
 }
 
 std::string BuildChatPrompt(const std::vector<MessageItem>& messages,
-                            OgaTokenizer& tokenizer,
+                            GenAIModelInstance& model,
                             const std::string& tools_json) {
   if (messages.empty()) {
     FL_THROW(FOUNDRY_LOCAL_ERROR_INTERNAL, "messages must not be empty");
   }
 
-  // Build messages JSON array matching the format expected by OgaTokenizer::ApplyChatTemplate.
+  // Build messages JSON array matching the format expected by the chat template.
   // Format: [{"role": "system", "content": "..."}, {"role": "user", "content": "..."}, ...]
   nlohmann::json messages_json = nlohmann::json::array();
   for (const auto& msg : messages) {
@@ -57,21 +58,15 @@ std::string BuildChatPrompt(const std::vector<MessageItem>& messages,
   std::string messages_str = messages_json.dump();
   const char* tools_ptr = tools_json.empty() ? nullptr : tools_json.c_str();
 
-  // ApplyChatTemplate: template_str=nullptr uses the model's built-in template,
-  // add_generation_prompt=true appends the assistant turn prefix
-  OgaString result = tokenizer.ApplyChatTemplate(nullptr,  // use model's template
-                                                 messages_str.c_str(),
-                                                 tools_ptr,
-                                                 true);  // add_generation_prompt
-
-  return std::string(static_cast<const char*>(result));
+  // ApplyChatTemplate uses the model's built-in template (template_str=nullptr) and appends the assistant
+  // turn prefix (add_generation_prompt=true). Routed through GenAIModelInstance so the shared, non-reentrant
+  // tokenizer is accessed under its mutex.
+  return model.Tokenizer().ApplyChatTemplate(messages_str.c_str(), tools_ptr, /*add_generation_prompt=*/true);
 }
 
 std::unique_ptr<OgaSequences> EncodePrompt(const std::string& prompt,
-                                           OgaTokenizer& tokenizer) {
-  auto sequences = OgaSequences::Create();
-  tokenizer.Encode(prompt.c_str(), *sequences);
-  return sequences;
+                                           GenAIModelInstance& model) {
+  return model.Tokenizer().Encode(prompt.c_str());
 }
 
 }  // namespace fl