Update ort and ort-genai package

.pipelines/foundry-local-packaging.yml

@@ -56,9 +56,9 @@ variables:
 # C++ SDK (sdk_v2/cpp) native dependency versions. Must match cmake defaults
 # in sdk_v2/deps_versions.json.
 - name: cppOrtVersion
-  value: '1.26.0'
+  value: '1.28.0'
 - name: cppGenaiVersion
-  value: '0.14.1'
+  value: '0.15.1'
 - name: cppWinmlVersion
   value: '2.1.70'
 - name: cppBuildConfig

.pipelines/v2/sdk_v2-js-pipeline-plan.md

@@ -113,9 +113,7 @@ matrix.
 
 Build and test on the `onnxruntime-linux-ARM64-CPU-2019` pool
 (`hostArchitecture: arm64`). Same `node-gyp rebuild` path as Linux x64.
-`install-native.cjs` selects the CPU-only ORT NuGet package
-(`Microsoft.ML.OnnxRuntime.Foundry`) for `linux-arm64` — matching the
-C++ native pipeline which also uses CPU-only ORT for ARM64.
+`install-native.cjs` uses `Microsoft.ML.OnnxRuntime`, matching the C++ native pipeline.
 
 ## Test stage details
 
@@ -220,8 +218,7 @@ shape.
 - **Single combined tarball:** `js_pack` assembles all five prebuilds
   into one `foundry-local-sdk-<version>.tgz`.
 - **JS scoped out of WinML.**
-- **Linux ARM64 ORT package:** `Microsoft.ML.OnnxRuntime.Foundry` (CPU-only),
-  matching the C++ native ARM64 pipeline.
+- **ORT package:** `Microsoft.ML.OnnxRuntime` on every RID.
 
 ## Open items
 

.pipelines/v2/sdk_v2-pipeline-plan.md

@@ -31,7 +31,7 @@ are gated separately via `.pipelines/v1/templates/stages-sdk-v1.yml`.
 |-----------------|-----------------------------------|-------|------|-----------------------------------------|
 | Windows x64     | `onnxruntime-Win-CPU-2022`        | ✅    | ✅   | Also stages public headers              |
 | Windows ARM64   | `onnxruntime-Win-CPU-2022`        | ✅    | ❌   | Cross-compiled from x64 host            |
-| Linux x64       | `onnxruntime-Ubuntu2404-AMD-CPU`  | ✅    | ✅   | Pulls extra `OnnxRuntime.Gpu.Linux` pkg |
+| Linux x64       | `onnxruntime-Ubuntu2404-AMD-CPU`  | ✅    | ✅   | Native CPU-only build                   |
 | Linux ARM64     | `onnxruntime-linux-ARM64-CPU-2019`| ✅    | ✅   | Native CPU-only build                   |
 | macOS ARM64     | `AcesShared` (Sequoia)            | ✅    | ✅   | Native                                  |
 
@@ -85,15 +85,8 @@ are gated separately via `.pipelines/v1/templates/stages-sdk-v1.yml`.
 
    Bumping ORT/GenAI is a one-file edit.
 9. **ORT/GenAI come from public PyPI.** No private feed plumbing required
-   for the wheel install path:
-   - `onnxruntime-core` (Windows/macOS)
-   - `onnxruntime-genai-core` (Windows/macOS)
-   - `onnxruntime-gpu` / `onnxruntime-genai-cuda` (Linux x64)
-   - `onnxruntime` / `onnxruntime-genai` (Linux ARM64, CPU-only)
-
-   Import-name mapping is platform-dependent: Linux uses `onnxruntime` /
-   `onnxruntime_genai`; Windows/macOS use `onnxruntime_core` /
-   `onnxruntime_genai_core`.
+   for the wheel install path. Every platform uses `onnxruntime` and
+   `onnxruntime-genai-core` (`onnxruntime` and `onnxruntime_genai_core` imports).
 10. **C++ staging step is the policy authority for native payload contents.**
     `steps-build-{windows,linux,macos}.yml` stage the **full runtime closure**
     of `foundry_local` into the `cpp-native-<rid>` artifact, with explicit
@@ -117,13 +110,9 @@ are gated separately via `.pipelines/v1/templates/stages-sdk-v1.yml`.
 
     Each step fails loudly if its primary library is missing.
 11. **Python runtime ORT discovery.** `lib_loader.py::prepare_native_dependencies()`
-    bridges between the in-wheel `foundry_local` and the pip-installed ORT
-    packages:
-    - **Windows:** `os.add_dll_directory(...)` for each ORT package directory.
-    - **Linux/macOS:** create symlinks
-      `_native/<rid>/{onnxruntime,onnxruntime-genai}.{so,dylib}` pointing at
-      the package-installed `lib*` files (workaround for
-      [onnxruntime#27263](https://github.com/microsoft/onnxruntime/issues/27263)).
+    preloads ORT and GenAI by absolute path. Windows also registers their DLL
+    directories; macOS creates the unversioned ORT symlink required by GenAI
+    ([onnxruntime#27263](https://github.com/microsoft/onnxruntime/issues/27263)).
 
     Wired into `_native/api.py` between `find_library()` and the cffi
     extension import. Idempotent and silent on failure.
@@ -247,16 +236,16 @@ purposes:
 
 1. **Version pinning** — the `KEY=PATH` pairs are passed via
    `--cmake_extra_defines` (`ORT_FETCH_URL`, `GENAI_FETCH_URL`,
-   `WINML_EP_CATALOG_FETCH_URL`, `ORT_GPU_LINUX_FETCH_URL`) so the cmake
-   defaults in `FindOnnxRuntime.cmake` / `FindOnnxRuntimeGenAI.cmake` are
-   never silently substituted.
+   `WINML_EP_CATALOG_FETCH_URL`) so the cmake defaults in
+   `FindOnnxRuntime.cmake` / `FindOnnxRuntimeGenAI.cmake` are never
+   silently substituted.
 2. **Stage isolation** — the build step no longer needs network access to
    the package feed once prefetch has completed.
 
 Versions are pipeline-level variables, currently:
 
-* `ortVersion`        `1.26.0`   (`Microsoft.ML.OnnxRuntime.Foundry`)
-* `genaiVersion`      `0.14.1`   (`Microsoft.ML.OnnxRuntimeGenAI.Foundry`)
+* `ortVersion`        `1.28.0`   (`Microsoft.ML.OnnxRuntime`)
+* `genaiVersion`      `0.15.1`   (`Microsoft.ML.OnnxRuntimeGenAI.Foundry`)
 * `winmlVersion`      `2.1.70`    (`Microsoft.Windows.AI.MachineLearning`, WinML 2.x reg-free)
 
 These must be kept in sync with the cmake defaults and with

.pipelines/v2/templates/steps-build-linux.yml

@@ -4,8 +4,7 @@
 # linux-arm64 (pool: onnxruntime-linux-ARM64-CPU-2019) via the `arch`
 # parameter. Each pool has the toolchain pre-installed; we still bootstrap
 # vcpkg and pre-download NuGet packages from the aiinfra feed for version
-# pinning. x64 also pre-downloads Microsoft.ML.OnnxRuntime.Gpu.Linux for
-# CUDA support; arm64 is CPU-only and skips that package.
+# pinning.
 
 parameters:
 - name: arch
@@ -36,7 +35,6 @@ steps:
     genaiVersion: ${{ parameters.genaiVersion }}
     winmlVersion: ''
     includeWinml: false
-    includeOrtGpuLinux: ${{ eq(parameters.arch, 'x64') }}
     shell: bash
 
 # Bake the pipeline-computed version into the binary so FoundryLocalGetVersionString()

.pipelines/v2/templates/steps-build-python.yml

@@ -121,10 +121,8 @@ steps:
         #   - Windows:  foundry_local.{dll,pdb,lib}
         #   - Linux:    libfoundry_local.so
         #   - macOS:    libfoundry_local.dylib
-        # ORT/GenAI native libs are intentionally absent — those come from pip
-        # deps (onnxruntime-{core,gpu} / onnxruntime-genai-{core,cuda}). Test
-        # and example binaries are also filtered out upstream. We just copy
-        # the curated artifact contents straight in.
+        # ORT/GenAI come from pip dependencies; this artifact contains only the
+        # curated Foundry Local native files.
         $files = Get-ChildItem -Path "${{ parameters.nativeArtifactDir }}" -Recurse -File
         if ($files.Count -eq 0) {
             throw "No native artifacts found under ${{ parameters.nativeArtifactDir }}"
@@ -135,12 +133,8 @@ steps:
         Write-Host "Copied $($files.Count) native file(s) to $dest"
         Get-ChildItem $dest | ForEach-Object { Write-Host "  $($_.Name)" }
 
-# Note: libfoundry_local.{so,dylib} already has $ORIGIN / @loader_path baked
-# into its rpath at CMake build time (see sdk_v2/cpp/CMakeLists.txt's RPATH
-# block on the foundry_local target). No patchelf / install_name_tool fixup
-# is needed here — the binary will look for ORT/GenAI deps next to itself
-# in _native/<rid>/, which is where lib_loader._create_ort_symlinks drops
-# them at process start.
+# lib_loader.py preloads the pip-installed ORT and GenAI libraries, so no
+# patchelf or install_name_tool fixup is needed.
 
 - task: PowerShell@2
   displayName: 'Clean stale build artifacts'

.pipelines/v2/templates/steps-build-windows.yml

@@ -7,7 +7,7 @@
 # Parameters:
 #   arch           – 'x64' or 'arm64'
 #   buildConfig    – CMake config (Debug, Release, RelWithDebInfo, MinSizeRel)
-#   ortVersion     – Microsoft.ML.OnnxRuntime.Foundry version
+#   ortVersion     – Microsoft.ML.OnnxRuntime version
 #   genaiVersion   – Microsoft.ML.OnnxRuntimeGenAI.Foundry version
 #   winmlVersion   – Microsoft.Windows.AI.MachineLearning version
 #   runTests       – Whether to run tests
@@ -207,7 +207,7 @@ steps:
 # The WinML DLL is delay-loaded (see /DELAYLOAD in CMakeLists.txt) so it is
 # NOT needed at foundry_local.dll load time, but the cmake post-build copy
 # stages it next to foundry_local.dll for runtime EP discovery. ORT/GenAI
-# come from the onnxruntime-core / onnxruntime-genai-core pip deps.
+# come from the onnxruntime / onnxruntime-genai-core pip deps.
 - task: PowerShell@2
   displayName: 'Stage native artifacts'
   inputs:

.pipelines/v2/templates/steps-prefetch-nuget.yml

@@ -5,11 +5,10 @@
 # `KEY=PATH` pairs to pass to build.py --cmake_extra_defines.
 #
 # Parameters:
-#   ortVersion     – Microsoft.ML.OnnxRuntime.Foundry version
+#   ortVersion     – Microsoft.ML.OnnxRuntime version
 #   genaiVersion   – Microsoft.ML.OnnxRuntimeGenAI.Foundry version
 #   winmlVersion   – Microsoft.Windows.AI.MachineLearning version (Windows only)
 #   includeWinml   – Download WinML and emit WINML_EP_CATALOG_FETCH_URL
-#   includeOrtGpuLinux – Also download Microsoft.ML.OnnxRuntime.Gpu.Linux (Linux only)
 #   shell          – 'pwsh' (Windows/macOS) or 'bash' (Linux)
 #   usePwsh        – Whether PowerShell tasks should use PowerShell Core
 
@@ -24,9 +23,6 @@ parameters:
 - name: includeWinml
   type: boolean
   default: false
-- name: includeOrtGpuLinux
-  type: boolean
-  default: false
 - name: shell
   type: string
   values: ['pwsh', 'bash']
@@ -80,19 +76,16 @@ steps:
         $cacheDir = "$(Build.BinariesDirectory)/nuget_packages"
         New-Item -ItemType Directory -Force -Path $cacheDir | Out-Null
 
-        # All four packages are public on nuget.org. Foundry Local Core's
+        # These packages are public on nuget.org. Foundry Local Core's
         # nuget.config maps everything except Microsoft.Telemetry* to nuget.org
         # (see .pipelines/templates/build-core-steps.yml), so we follow the
         # same source of truth here.
         $feed = "https://www.nuget.org/api/v2/package"
 
         $packages = @(
           @{ key = 'genai'; id = 'Microsoft.ML.OnnxRuntimeGenAI.Foundry'; version = '${{ parameters.genaiVersion }}' },
-          @{ key = 'ort';   id = 'Microsoft.ML.OnnxRuntime.Foundry';     version = '${{ parameters.ortVersion }}'   }
+          @{ key = 'ort';   id = 'Microsoft.ML.OnnxRuntime';             version = '${{ parameters.ortVersion }}'   }
         )
-        if ($${{ parameters.includeOrtGpuLinux }}) {
-          $packages += @{ key = 'ort_gpu_linux'; id = 'Microsoft.ML.OnnxRuntime.Gpu.Linux'; version = '${{ parameters.ortVersion }}' }
-        }
 
         $defines = @()
         foreach ($pkg in $packages) {
@@ -104,9 +97,8 @@ steps:
           Write-Host "  -> $out ($((Get-Item $out).Length) bytes)"
 
           switch ($pkg.key) {
-            'genai'         { $defines += "GENAI_FETCH_URL=$out" }
-            'ort'           { $defines += "ORT_FETCH_URL=$out" }
-            'ort_gpu_linux' { $defines += "ORT_GPU_LINUX_FETCH_URL=$out" }
+            'genai' { $defines += "GENAI_FETCH_URL=$out" }
+            'ort'   { $defines += "ORT_FETCH_URL=$out" }
           }
         }
 
@@ -132,25 +124,21 @@ steps:
       cacheDir="$(Build.BinariesDirectory)/nuget_packages"
       mkdir -p "$cacheDir"
 
-      # All four packages are public on nuget.org. Foundry Local Core's
+      # These packages are public on nuget.org. Foundry Local Core's
       # nuget.config maps everything except Microsoft.Telemetry* to nuget.org
       # (see .pipelines/templates/build-core-steps.yml), so we follow the
       # same source of truth here.
       feed="https://www.nuget.org/api/v2/package"
 
       declare -a entries=(
         "genai:Microsoft.ML.OnnxRuntimeGenAI.Foundry:${{ parameters.genaiVersion }}"
-        "ort:Microsoft.ML.OnnxRuntime.Foundry:${{ parameters.ortVersion }}"
+        "ort:Microsoft.ML.OnnxRuntime:${{ parameters.ortVersion }}"
       )
       if [ "${{ parameters.includeWinml }}" = "True" ]; then
         # WinML is Windows-only; the bash branch should never receive includeWinml=true.
         echo "ERROR: includeWinml=true is not supported on the bash prefetch branch (WinML is Windows-only)." >&2
         exit 1
       fi
-      if [ "${{ parameters.includeOrtGpuLinux }}" = "True" ]; then
-        entries+=("ort_gpu_linux:Microsoft.ML.OnnxRuntime.Gpu.Linux:${{ parameters.ortVersion }}")
-      fi
-
       defines=()
       for entry in "${entries[@]}"; do
         IFS=: read -r key id version <<< "$entry"
@@ -161,9 +149,8 @@ steps:
         echo "  -> $out ($(stat -c%s "$out" 2>/dev/null || stat -f%z "$out") bytes)"
 
         case "$key" in
-          genai)         defines+=("\"GENAI_FETCH_URL=$out\"") ;;
-          ort)           defines+=("\"ORT_FETCH_URL=$out\"") ;;
-          ort_gpu_linux) defines+=("\"ORT_GPU_LINUX_FETCH_URL=$out\"") ;;
+          genai) defines+=("\"GENAI_FETCH_URL=$out\"") ;;
+          ort)   defines+=("\"ORT_FETCH_URL=$out\"") ;;
         esac
       done
 

.pipelines/v2/templates/steps-test-python.yml

@@ -135,6 +135,10 @@ steps:
         & "$(venvPy)" -m pip install --upgrade pip
         & "$(venvPy)" -m pip install $wheel
         if ($LASTEXITCODE -ne 0) { exit $LASTEXITCODE }
+        & "$(venvPy)" -c "import importlib.util; assert importlib.util.find_spec('onnxruntime'); assert importlib.util.find_spec('onnxruntime_genai_core')"
+        if ($LASTEXITCODE -ne 0) {
+            throw "Wheel install did not resolve universal onnxruntime / onnxruntime-genai-core dependencies"
+        }
         & "$(venvPy)" -m pip install pytest pytest-cov
         if ($LASTEXITCODE -ne 0) { exit $LASTEXITCODE }
 

sdk_v2/cpp/CMakeLists.txt

@@ -169,6 +169,7 @@ set(FOUNDRY_LOCAL_SOURCES
     src/inferencing/generative/genai_config.cc
     src/http/http_client.cc
     src/inferencing/generative/genai_model_instance.cc
+    src/inferencing/generative/tokenizer.cc
     src/logger.cc
     src/spdlog_logger.cc
     src/model_info.cc

sdk_v2/cpp/cmake/FindOnnxRuntime.cmake

@@ -1,10 +1,9 @@
 # Copyright (c) Microsoft. All rights reserved.
 # Find/acquire ONNX Runtime.
 #
-# Sources ORT from Microsoft.ML.OnnxRuntime.Foundry (or Microsoft.ML.OnnxRuntime
-# on Android) via FetchContent — nuget.org for releases, the ORT-Nightly ADO
-# feed for -dev- versions. The version comes from sdk_v2/deps_versions.json and
-# is shared by all platforms.
+# Sources ORT from Microsoft.ML.OnnxRuntime via FetchContent — nuget.org for
+# releases and the ORT-Nightly ADO feed for -dev- versions. The version comes
+# from sdk_v2/deps_versions.json and is shared by all platforms.
 #
 # Creates an IMPORTED target: OnnxRuntime::OnnxRuntime
 
@@ -74,15 +73,7 @@ else()
         string(JSON ORT_VERSION GET "${_DEPS_JSON}" "onnxruntime" "version")
         message(STATUS "ORT_VERSION=${ORT_VERSION} (from ${_DEPS_FILE})")
     endif()
-    if(NOT ORT_PACKAGE_NAME)
-        if(ANDROID)
-            # The Foundry meta-package may not contain Android binaries;
-            # use the base ORT package which includes the AAR.
-            set(ORT_PACKAGE_NAME "Microsoft.ML.OnnxRuntime")
-        else()
-            set(ORT_PACKAGE_NAME "Microsoft.ML.OnnxRuntime.Foundry")
-        endif()
-    endif()
+    set(ORT_PACKAGE_NAME "Microsoft.ML.OnnxRuntime")
 
     # ORT_FETCH_URL can be set externally (e.g. for CI where nuget.org is blocked).
     set(ORT_FETCH_URL "" CACHE STRING "Override URL or local path for the OnnxRuntime NuGet package")
@@ -135,48 +126,6 @@ else()
              DESTINATION "${ortlib_SOURCE_DIR}/runtimes/android/native/")
         set(_ORT_LIB_DIR "${ortlib_SOURCE_DIR}/runtimes/android/native/jni/${ANDROID_ABI}")
         message(STATUS "Extracted ORT Android AAR: ${_ORT_AAR_PATH}")
-    elseif(CMAKE_SYSTEM_NAME STREQUAL "Linux")
-        if(_ORT_PLATFORM STREQUAL "linux-x64")
-            # On Linux x64 the Foundry meta-package doesn't contain libonnxruntime.so directly —
-            # it's in the Microsoft.ML.OnnxRuntime.Gpu.Linux dependency package.
-            set(_ORT_GPU_LINUX_PACKAGE "Microsoft.ML.OnnxRuntime.Gpu.Linux")
-
-            # ORT_GPU_LINUX_FETCH_URL can be set externally (e.g. for CI where nuget.org is blocked).
-            set(ORT_GPU_LINUX_FETCH_URL "" CACHE STRING "Override URL or local path for the ORT GPU Linux NuGet package")
-
-            if(NOT ORT_GPU_LINUX_FETCH_URL)
-                if(ORT_VERSION MATCHES "-dev-")
-                    set(ORT_GPU_LINUX_FETCH_URL "https://pkgs.dev.azure.com/${ORT_FEED_ORG}/${ORT_FEED_PROJECT}/_apis/packaging/feeds/${ORT_FEED_ID}/nuget/packages/${_ORT_GPU_LINUX_PACKAGE}/versions/${ORT_VERSION}/content?api-version=6.0-preview.1")
-                    message(STATUS "Downloading ${_ORT_GPU_LINUX_PACKAGE} ${ORT_VERSION} from ORT-Nightly feed")
-                else()
-                    string(TOLOWER "${_ORT_GPU_LINUX_PACKAGE}" _ORT_GPU_LINUX_LOWER)
-                    set(ORT_GPU_LINUX_FETCH_URL "https://api.nuget.org/v3-flatcontainer/${_ORT_GPU_LINUX_LOWER}/${ORT_VERSION}/${_ORT_GPU_LINUX_LOWER}.${ORT_VERSION}.nupkg")
-                    message(STATUS "Downloading ${_ORT_GPU_LINUX_PACKAGE} ${ORT_VERSION} from nuget.org")
-                endif()
-            else()
-                message(STATUS "Using pre-configured ORT_GPU_LINUX_FETCH_URL: ${ORT_GPU_LINUX_FETCH_URL}")
-            endif()
-
-            # Normalize backslashes and handle .nupkg extension
-            string(REPLACE "\\" "/" ORT_GPU_LINUX_FETCH_URL "${ORT_GPU_LINUX_FETCH_URL}")
-            if(ORT_GPU_LINUX_FETCH_URL MATCHES "\\.nupkg$" AND NOT ORT_GPU_LINUX_FETCH_URL MATCHES "^https?://")
-                set(_ORT_GPU_ZIP_PATH "${CMAKE_BINARY_DIR}/_deps/ort_gpu_linux-download/ort_gpu_linux.zip")
-                get_filename_component(_ORT_GPU_ZIP_DIR "${_ORT_GPU_ZIP_PATH}" DIRECTORY)
-                file(MAKE_DIRECTORY "${_ORT_GPU_ZIP_DIR}")
-                file(COPY_FILE "${ORT_GPU_LINUX_FETCH_URL}" "${_ORT_GPU_ZIP_PATH}")
-                set(ORT_GPU_LINUX_FETCH_URL "${_ORT_GPU_ZIP_PATH}")
-            endif()
-
-            FetchContent_Declare(ort_gpu_linux URL ${ORT_GPU_LINUX_FETCH_URL} DOWNLOAD_EXTRACT_TIMESTAMP TRUE DOWNLOAD_NAME ort_gpu_linux.zip)
-            FetchContent_MakeAvailable(ort_gpu_linux)
-
-            set(_ORT_LIB_DIR "${ort_gpu_linux_SOURCE_DIR}/runtimes/${_ORT_PLATFORM}/native")
-            message(STATUS "OnnxRuntime GPU Linux package: ${ort_gpu_linux_SOURCE_DIR}")
-        else()
-            # linux-arm64: libonnxruntime.so is included directly in the Microsoft.ML.OnnxRuntime.Foundry
-            # package under runtimes/linux-arm64/native — no separate GPU package needed.
-            message(STATUS "OnnxRuntime Linux ARM64: using Foundry package runtimes/${_ORT_PLATFORM}/native")
-        endif()
     endif()
 endif()
 

sdk_v2/cpp/nuget/Microsoft.AI.Foundry.Local.Runtime.nuspec

@@ -18,7 +18,7 @@
 
     <dependencies>
       <group>
-        <dependency id="Microsoft.ML.OnnxRuntime.Foundry" version="$ort_version$" />
+        <dependency id="Microsoft.ML.OnnxRuntime" version="$ort_version$" />
         <dependency id="Microsoft.ML.OnnxRuntimeGenAI.Foundry" version="$genai_version$" />
       </group>
     </dependencies>

sdk_v2/cpp/nuget/pack.py

@@ -74,7 +74,7 @@ def _parse_args() -> argparse.Namespace:
     parser.add_argument("--version", required=True,
                         help="Package version (e.g. 0.1.0 or 0.1.0-dev.20260419).")
     parser.add_argument("--ort_version", required=True,
-                        help="Minimum Microsoft.ML.OnnxRuntime.Foundry version.")
+                        help="Minimum Microsoft.ML.OnnxRuntime version.")
     parser.add_argument("--genai_version", required=True,
                         help="Minimum Microsoft.ML.OnnxRuntimeGenAI.Foundry version.")
     parser.add_argument("--package_id", default="Microsoft.AI.Foundry.Local.Runtime",

sdk_v2/cpp/src/inferencing/generative/audio/audio_session.cc

@@ -284,7 +284,7 @@ void AudioSession::ProcessStreamingAudio(const AudioItem& format_item, ItemQueue
   }
 
   auto generator = OgaGenerator::Create(oga_model, *gen_params);
-  auto tokenizer_stream = OgaTokenizerStream::Create(Model().GetOgaTokenizer());
+  auto tokenizer_stream = OgaTokenizerStream::Create(Model().Tokenizer().Oga());
 
   auto streaming_callback = CreateCallbackHandler(request);
   std::vector<std::string> token_texts;