Separate onnxruntime_providers compilation from its archive

[Ryan VanderMeulen (rvandermeulen)]

Supersedes #30511, which GitHub closed automatically when I force-pushed a commit that had lost its parent, and will not let me reopen. The review discussion there is still readable; both of Edward Chen (@edgchen1)'s comments on it are addressed here.

The change

On MSVC shared-library builds the provider sources go into an object library, onnxruntime_providers_obj; everywhere else they go into the static library onnxruntime_providers as before. Only one of the two exists per configuration, and consumers link ${onnxruntime_providers_target}, so the condition lives in onnxruntime_providers_cpu.cmake alone. onnxruntime_unittests.cmake keys off the target's actual TYPE rather than re-testing it.

The archive is skipped there because with onnxruntime_ENABLE_LTO the objects carry whole-program IR, pushing it past the 4 GiB limit lib.exe enforces on the COFF archive format (LNK1248). No lib.exe option raises that limit, and a shared-library build only needs the objects for the final link.

Why compilation and packaging are not separate targets

An earlier revision always created the object library and materialized the archive from $<TARGET_OBJECTS:onnxruntime_providers_obj>. That does not survive the Xcode generator, and failed every iOS job:

libtool: can't open file: .../onnxruntime_providers_obj.build/.../Objects-normal/arm64/utils.o (No such file or directory)

Xcode flattens a target's objects into one directory and CMake hash-suffixes the filenames when source basenames collide, but TARGET_OBJECTS expands to the unsuffixed names. Sources with unique basenames in the same target were written unsuffixed, which is the control. Ninja is unaffected.

Two further constraints:

• The archive cannot link the object library in any form. Even a PRIVATE link is recorded as $<LINK_ONLY:>, and install(EXPORT) then rejects onnxruntime_providers for requiring a target outside the export set.
• Consumers go through a variable rather than being repointed at the object library, because onnxruntime_INTERNAL_LIBRARIES is consumed both as a link list and as a list of archive files — the Apple static-framework prelink in onnxruntime.cmake, and bundle_static_library() in onnxruntime_webassembly.cmake, which collects only STATIC_LIBRARY dependencies and would otherwise drop the providers silently. That is also why the condition stays keyed on MSVC.

Verification

Firefox builds onnxruntime on Windows with cl.exe from MSVC 14.51.36231 and this exact configuration, so it serves as an A/B — same pinned revision both sides, differing only by this patch:

python3 tools/ci_build/build.py --update --parallel --enable_lto --disable_rtti \
  --cmake_generator Ninja --build_dir _build --build --build_shared_lib \
  --skip_submodule_sync --use_lock_free_queue --skip_tests --config MinSizeRel \
  --compile_no_warning_as_error \
  --cmake_extra_defines onnxruntime_BUILD_UNIT_TESTS=OFF \
  --cmake_extra_defines ONNX_USE_LITE_PROTO=ON \
  --disable_exceptions \
  --cmake_extra_defines CMAKE_SHARED_LINKER_FLAGS=/MANIFEST:NO

Without the patch, x86_64 and i686 both fail, with onnxruntime_ENABLE_LTO=ON and onnxruntime_BUILD_SHARED_LIB=ON in the generated cache:

lib.exe /nologo /LTCG /machine:x64 /LTCG /out:onnxruntime_providers.lib @CMakeFiles\onnxruntime_providers.rsp
onnxruntime_providers.lib : fatal error LNK1248: image size (1001DD8D4) exceeds maximum allowable size (FFFFFFFF)

That is 4,296,923,348 bytes, roughly 1.9 MB over; the 32-bit build reports 4,300,722,339. The margin is small, consistent with where this surfaced — the same build linked fine until the toolchain moved to 14.51.

With the patch, onnxruntime_providers.lib is never created, the objects build under onnxruntime_providers_obj, and both DLLs link.

That configuration is trimmed (--disable_rtti, --disable_exceptions, lite proto, MinSizeRel), so a stock shared+LTO build overflows by at least as much. It also sets onnxruntime_BUILD_UNIT_TESTS=OFF with no training or webgpu, so those paths rely on CI here. On Linux the shared, static and forced-object-library configurations all configure and build, with no provider objects in any intermediate archive.

[azure-pipelines]

Azure Pipelines:
There may be pipelines that require an authorized user to comment /azp run to run.

[Copilot]

Pull request overview

Note

Copilot was unable to run its full agentic suite in this review.

This PR changes how the onnxruntime_providers library is built/consumed by introducing an object-library based path (primarily for MSVC shared builds) and updating downstream link dependencies to use a single indirection target variable.

Changes:

• Introduce onnxruntime_providers_obj object library and set ${onnxruntime_providers_target} to either the object library or a materialized static archive depending on configuration.
• Update multiple CMake link lists to use ${onnxruntime_providers_target} instead of the hard-coded onnxruntime_providers target.
• Adjust unit test utility linking to avoid copying object-library objects into an intermediate static archive.

Reviewed changes

Copilot reviewed 9 out of 9 changed files in this pull request and generated 3 comments.

Show a summary per file

File	Description
cmake/onnxruntime_providers_cpu.cmake	Splits providers into object library + conditional archive with ${onnxruntime_providers_target} indirection.
cmake/onnxruntime_unittests.cmake	Switches tests to ${onnxruntime_providers_target} and adds special handling when it’s an object library.
cmake/onnxruntime.cmake	Updates internal library aggregation to use ${onnxruntime_providers_target}.
cmake/onnxruntime_webassembly.cmake	Uses ${onnxruntime_providers_target} when bundling/linking for WebAssembly.
cmake/onnxruntime_training.cmake	Updates training runner deps/linking to ${onnxruntime_providers_target}.
cmake/onnxruntime_python.cmake	Updates pybind module linking to ${onnxruntime_providers_target}.
cmake/onnxruntime_providers_webgpu.cmake	Updates WebGPU provider linking to ${onnxruntime_providers_target}.
cmake/onnxruntime_providers_pch.cmake	Renames PCH target to onnxruntime_providers_obj.
cmake/CMakeLists.txt	Includes providers PCH config only if onnxruntime_providers_obj exists.

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

[Copilot]

Pull request overview

Copilot reviewed 9 out of 9 changed files in this pull request and generated 2 comments.

Suppressed comments (1)

cmake/onnxruntime_providers_cpu.cmake:210

• LINKER_LANGUAGE is only meaningful for linkable targets (e.g., STATIC/SHARED libraries, executables). For an OBJECT_LIBRARY it is effectively ignored, so this property assignment is misleading. Prefer removing it here and (if needed) applying LINKER_LANGUAGE CXX only to the actual archive target (onnxruntime_providers).

set_target_properties(onnxruntime_providers_obj PROPERTIES LINKER_LANGUAGE CXX)

[Ryan VanderMeulen (rvandermeulen)]

Seems I'm at an impasse with the reviewbots. I'll wait for Edward Chen (@edgchen1)'s review for now 😄

[Copilot]

Pull request overview

Copilot reviewed 9 out of 9 changed files in this pull request and generated no new comments.

[Edward Chen (edgchen1)]

Could you confirm that you manually validated the exact MSVC configuration this change targets: --build_shared_lib --enable_lto? Current PR CI does not cover that combination, so recording the command and successful result in the PR description would close the validation gap.

The PR description also appears to describe the earlier two-target design and should be updated to match the current implementation.

🤖 Written by GitHub Copilot on behalf of @edgchen1.

[Ryan VanderMeulen (rvandermeulen)]

Confirmed, and recorded in the description.

Firefox builds onnxruntime on Windows with this exact configuration, so I ran it as an A/B — same pinned revision both sides, differing only by this patch, cl.exe from MSVC 14.51.36231:

python3 tools/ci_build/build.py --update --parallel --enable_lto --disable_rtti \
  --cmake_generator Ninja --build_dir _build --build --build_shared_lib \
  --skip_submodule_sync --use_lock_free_queue --skip_tests --config MinSizeRel \
  --compile_no_warning_as_error \
  --cmake_extra_defines onnxruntime_BUILD_UNIT_TESTS=OFF \
  --cmake_extra_defines ONNX_USE_LITE_PROTO=ON \
  --disable_exceptions \
  --cmake_extra_defines CMAKE_SHARED_LINKER_FLAGS=/MANIFEST:NO

Without the patch, x86_64 and i686 both fail:

lib.exe /nologo /LTCG /machine:x64 /LTCG /out:onnxruntime_providers.lib @CMakeFiles\onnxruntime_providers.rsp
onnxruntime_providers.lib : fatal error LNK1248: image size (1001DD8D4) exceeds maximum allowable size (FFFFFFFF)

With it, no archive is produced and both link.

That build is trimmed (--disable_rtti, --disable_exceptions, lite proto, MinSizeRel), so a stock shared+LTO build overflows by at least as much. It also sets onnxruntime_BUILD_UNIT_TESTS=OFF with no training or webgpu, so those paths rely on CI here.

[Copilot]

Pull request overview

Copilot reviewed 9 out of 9 changed files in this pull request and generated no new comments.

[Edward Chen (edgchen1)]

closing and reopening to restart CI builds

[azure-pipelines]

Azure Pipelines:
There may be pipelines that require an authorized user to comment /azp run to run.