Cherry-picks for rel-1.23.0 #26059
Merged
Cherry-picks for rel-1.23.0 #26059
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Contributor
snnn
commented
Sep 16, 2025
snnn
commented
- Major Refactoring of Azure DevOps Pipelines (Major Refactoring of Azure DevOps Pipelines #26008)
- Convert QNN x64 pipeline to GH (Convert QNN x64 pipeline to GH #26047)
- Upgrade com.diffplug.spotless to 7.2.1 in Java build (Upgrade com.diffplug.spotless to 7.2.1 in Java build #26051)
added 3 commits
September 16, 2025 12:14
This pull request introduces a significant refactoring of the Azure Pipelines CI/CD infrastructure. The primary goals of these changes are to: 1. Solve the problem that vcpkg/cmake version can get changed when the CI build machine image changes, which can make pipeline suddenly broken and interrupt our release process. 2. Reduce the `Zip-Nuget-Java-Nodejs Packaging Pipeline`'s running time by changing how macOS universal2 binaries are built. 3. Add the support for RC releases for Java packages. **1. Standardized Build Tool Setup (`setup-build-tools.yml`)** A new reusable template, `setup-build-tools.yml`, has been created to centralize the setup of essential build tools. * **Pinned Dependencies:** This new template allows us to pin specific versions of `cmake` and `vcpkg`, which was not previously possible. This ensures a more stable and predictable build environment across all pipelines. * **Reduced Redundancy:** By consolidating the setup logic into a single template, we have significantly reduced code duplication across numerous pipeline files, making them cleaner and easier to maintain. Currently this file is only used in macOS and Windows pipelines since most Linux pipelines use docker to manage their environment. **2. Reworked macOS Universal Binary Build Process** The methodology for building macOS `universal2` binaries has been fundamentally changed to improve reliability and flexibility. * **Python Packaging:** The Python packaging pipeline will no longer produce `universal2` wheels. Instead, it will generate separate wheels for `x86_64` and `arm64` architectures. * **NuGet C-API Packaging:** The NuGet C-API pipeline has been updated to first build the `x86_64` and `arm64` binaries independently. These single-architecture binaries are then combined to create the final universal package, rather than building a `universal2` package in a single pass. The change is made mainly because: - Building for both ARCHs in a single pass is too slow, which may take about 5 hours in the ADO machine pool. - A lot of MLAS features are disabled when ORT is built in such a way. **3. Java Packaging and Testing Overhaul** The Download_Java_Tools stage in "Zip-Nuget-Java-Nodejs Packaging Pipeline" is deleted because it is no longer used. Previously it was added to reduce the times of downloading the same java packages again and again , which was to reduce download errors. Now we have setup a private ADO feed for this. Besides, there are some major changes to the pipeline that: 1. MD5 and SHA1 checksum files are provided along with the java package files instead of SHA256. This is because Sonatype's official docs says MD5/SHA1 checkcums are required while the others are optional. See: https://central.sonatype.org/publish/requirements/#supply-javadoc-and-sources . Now the publishing would fail if we don't have the MD5/SHA1 checksum files. 2. The format of the checksum files is changed. Previously we used Linux's sha256sum command to generate such files, so each checksum file contains a hash value and a filename in the file content. However, it was not the expected format. Sonatype expects that the file only contains a hash value. This PR fixes the issue. 3. A few powershell scripts were rewritten in python to improve error check and robustness 4. Added the support for generating RC packages. Previously we had to manually modify the version numbers and manually do GPG sign. 5. Two new files `jar-packaging.yml` and `setup-maven.yml` were added. We will use maven to fetch dependent packages(instead of directly HTTP fetching) to improve supply chain security, because maven allows us using a private feed to do so. **4. Dockerfile Enhancements** The Dockerfiles used in the CI have been updated to use a `BASEIMAGE` argument. This makes them more flexible, allowing the base image to be specified at build time, which simplifies maintenance and updates. It will allow us to using different base image repos in different CI environments. In the future we will change the Github Actions to only fetch base images from public docker repos. Meanwhile, ADO packaging pipelines will continue to use private repos. **5. Improved Release Management** The run_packaging_pipelines.py script has been updated to provide more robust and explicit control over the package versioning for different build scenarios. This clarifies the process for generating nightly, release candidate (RC), and final release packages. The script now handles three distinct cases for package versioning: * Nightly Packages: For regular CI builds (e.g., on the main branch), the script triggers the packaging pipelines in "nightly" mode. This sets the IsReleaseBuild parameter to false and results in packages with a development version suffix (e.g., 1.2.3-dev-20250909-abcdef). * Release Candidate (RC) Builds: To create a pre-release or RC build, the script is run with the --build-mode release flag, along with the --pre-release-suffix-string (e.g., rc) and --pre-release-suffix-number (e.g., 1) arguments. This sets the IsReleaseBuild parameter to true and passes the suffix information to the pipelines, resulting in a semantically versioned pre-release package (e.g., 1.2.3-rc.1). * Final Release Builds: For a final release, the script is run with --build-mode release without any pre-release suffix arguments. This sets IsReleaseBuild to true, and the resulting package will have a clean, final version number (e.g., 1.2.3) based on the VERSION_NUMBER file. Please note: - Java packages still only support the second and third mode. - Python packages only support the first and the last mode.
Convert QNN x64 CI pipeline from ADO pipeline to Github Actions. We shall move all PR pipelines to Github Actions.
This PR upgrades the com.diffplug.spotless Gradle plugin to version 7.2.1 in the java/build.gradle file. This brings in the latest features and bug fixes from the Spotless code formatter.
![github-advanced-security[bot]](https://avatars.githubusercontent.com/in/57789?s=60&v=4){kind=small}
snnn
force-pushed
the
users/snnn/cr
branch
2 times, most recently
from
1ef8a11 to
c9dee4d
Compare
### Description <!-- Describe your changes. --> Fix Mac Catalyst build options. There were some overriding `--target` and `-mmacosx-version-min` options causing build errors. ### Motivation and Context <!-- - Why is this change required? What problem does it solve? - If it fixes an open issue, please link to the issue here. --> Fix #25920
baijumeswani
approved these changes
Sep 18, 2025
natke
approved these changes
Sep 18, 2025
This was referenced Sep 19, 2025
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.