[release/8.0.4xx] Backport maestro and artifact drop infra improvements #9195
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Context: https://github.com/dotnet/arcade/blob/efc3da96e5ac110513e92ebd9ef87c73f44d8540/Documentation/DependencyFlowOnboardingWithoutArcade.md The steps used to publish build asset information to maestro have been updated. With the new `PushToAzureDevOpsArtifacts` task the build pipeline should now create all of the artifacts required for maestro artifact publishing. The `add-build-to-channel` darc command will now trigger a [Build Promotion Pipeline][0] that pushes build assets to the feed that corresponds to the maestro channel that is being updated. We should no longer need to push assets to various NuGet feeds in a separate step. [0]: https://dev.azure.com/devdiv/DevDiv/_build/results?buildId=9577012&view=logs&j=ba23343f-f710-5af9-782d-5bd26b102304&t=c7a8693b-2f9c-5ea8-c909-cde9405ac2e1&l=238
Commit bbac9fe ran into some issues when attempting to publish to maestro: error : Asset 'D:\a\_work\1\a\7dc04dfe-406a-4fa3-aea0-199acc2763fa\MergedManifest.xml' already exists with different contents at assets/manifests/xamarin-xamarin-android/34.99.0-dev/MergedManifest.xml We should be able to fix this by using the long package version which optionally includes pre-release labeling and commit distance info.
Context: xamarin/yaml-templates@8759ec9 Context: xamarin/sdk-insertions#149 Steps to upload release artifacts to custom blob storage have been replaced with azure-artifacts-drop (aka.ms/drop). A new version of nuget-msi-convert has been added that will create a set of artifact drops for the following shipping artifacts: * nugets * vs-components * vs-packs The nugets drop contains all shipping packages that should be pushed to various feeds or NuGet.org. The components and packs drops are used for VS insertions.
We've seen the build promotion pipeline fail when trying to publish
stable package versions:
error : Package 'Microsoft.iOS.Ref.net8.0_17.5' has stable version '17.5.8001' but is targeted at a non-isolated feed 'https://pkgs.dev.azure.com/dnceng/public/_packaging/dotnet8/nuget/v3/index.json'
This is because we were not declaring these packages as stable when
building the build asset registry manifest.
Fix this by passing the `$(IsStableBuild)` property to the build asset
manifest creation task. This property needs to be updated manually when
switching to stable package versioning (see commit 4ea5dbb).
When `$(IsStableBuild)` is set to true, packages will be pushed to an
isolated feed during publishing, such as:
Package [email protected] (Shipping) should go to https://pkgs.dev.azure.com/dnceng/public/_packaging/darc-pub-dotnet-android-b8317b6f/nuget/v3/index.json (Isolated, Public)
Fixes: #9164 Migrates darc/maestro commands to use a passwordless auth flow, as token based authentication is deprecated and will be removed in the future.
Using the $(System.JobAttempt) variable in the drop metadata artifact name is problematic. In some cases the drop artifacts created by the nuget-msi-convert job will be used by a different job, and the job attempt number will not necessarily match if any jobs are re-ran.
|
Oops missed a commit, new test build: https://devdiv.visualstudio.com/DevDiv/_build/results?buildId=10054781&view=results |
jonathanpeppers
approved these changes
Aug 16, 2024
jonathanpeppers
added a commit
that referenced
this pull request
Aug 22, 2024
Changes: 34.0.113...b0fd011 .NET 8 changelog: * [Mono.Android] AndroidMessageHandler should follow HTTP-308 redirects (#8951) * [ci] Fix android source path for MAUI test job (#9030) * [ci] Update checkout path for nightly build (#9028) * [Mono.Android-Tests] Fix repo URL in redirect tests (#9035) * [Xamarin.Android.Build.Tasks] Support VS "Build Acceleration" (#9042) * [xaprepare] Always use release mono bundle (#9106) * [ci] Update sdk-insertions trigger to manual only (#9029) * Bump to dotnet/runtime@4a37e7305c 8.0.8 (#9033) * Bump to dotnet/installer@b638a84fba 8.0.205-servicing.24212.27 (#9032) * [ci] Disable CodeQL on macOS, Linux, non-main jobs (#9111) * [ci] Use DotNetCoreCLI to sign macOS files (#9102) * [tests] fix `InvalidTargetPlatformVersion` for `net8.0` (#9110) * [ci] Run "Push Internal" job on AzurePipelines-EO pool (#8991) * Bump to dotnet/runtime@2d5c0b720c 8.0.8 (#9123) * [Mono.Android] Data sharing and Close() overrides (#9103) * [Xamarin.Android.Build.Tasks] fix `Inputs` for `_Generate*Java*` targets (#9174) * Bump to xamarin/monodroid@e6a7cf474a (#9193) * [release/8.0.4xx] Backport maestro and artifact drop infra improvements (#9195) * Bump to dotnet/runtime@62f69f1e86 8.0.9 (#9191) * Bump to dotnet/runtime@ed13b35174 8.0.9 (#9221) * [build] Update package metadata (#9230) * [Xamarin.Android.Build.Tasks] Rethink default property values (#9155) (#9224) * [Xamarin.Android.Build.Tasks] Fix an issue with incremental builds (#9183) * [ci] Improve push_signed_nugets job condition (#9240)
jonathanpeppers
added a commit
that referenced
this pull request
Aug 23, 2024
Changes: 34.0.113...b0fd011 .NET 8 changelog: * [Mono.Android] AndroidMessageHandler should follow HTTP-308 redirects (#8951) * [ci] Fix android source path for MAUI test job (#9030) * [ci] Update checkout path for nightly build (#9028) * [Mono.Android-Tests] Fix repo URL in redirect tests (#9035) * [Xamarin.Android.Build.Tasks] Support VS "Build Acceleration" (#9042) * [xaprepare] Always use release mono bundle (#9106) * [ci] Update sdk-insertions trigger to manual only (#9029) * Bump to dotnet/runtime@4a37e7305c 8.0.8 (#9033) * Bump to dotnet/installer@b638a84fba 8.0.205-servicing.24212.27 (#9032) * [ci] Disable CodeQL on macOS, Linux, non-main jobs (#9111) * [ci] Use DotNetCoreCLI to sign macOS files (#9102) * [tests] fix `InvalidTargetPlatformVersion` for `net8.0` (#9110) * [ci] Run "Push Internal" job on AzurePipelines-EO pool (#8991) * Bump to dotnet/runtime@2d5c0b720c 8.0.8 (#9123) * [Mono.Android] Data sharing and Close() overrides (#9103) * [Xamarin.Android.Build.Tasks] fix `Inputs` for `_Generate*Java*` targets (#9174) * Bump to xamarin/monodroid@e6a7cf474a (#9193) * [release/8.0.4xx] Backport maestro and artifact drop infra improvements (#9195) * Bump to dotnet/runtime@62f69f1e86 8.0.9 (#9191) * Bump to dotnet/runtime@ed13b35174 8.0.9 (#9221) * [build] Update package metadata (#9230) * [Xamarin.Android.Build.Tasks] Rethink default property values (#9155) (#9224) * [Xamarin.Android.Build.Tasks] Fix an issue with incremental builds (#9183) * [ci] Improve push_signed_nugets job condition (#9240) ~~ Other changes ~~ * Setup Maestro to track .NET8 builds After setting up the appropriate changes in `Version.Details.xml`, I ran: > darc update-dependencies --id 235947 Looking up build with BAR id 235947 Updating 'Microsoft.Android.Sdk.Windows': '1' => '34.0.137' (from build '8.0.4xx- b0fd011-1' of 'https://github.com/dotnet/android') Checking for coherency updates... Using 'Strict' coherency mode. If this fails, a second attempt utilizing 'Legacy' Coherency mode will be made. Local dependencies updated based on build with BAR id 235947 (8.0.4xx-b0fd0113c829edd9bd1bc7d742255a237ff19f69-1 from https://github.com/dotnet/android@release/8.0.4xx)
jonathanpeppers
added a commit
that referenced
this pull request
Aug 23, 2024
Changes: 34.0.113...b0fd011 .NET 8 changelog: * [Mono.Android] AndroidMessageHandler should follow HTTP-308 redirects (#8951) * [ci] Fix android source path for MAUI test job (#9030) * [ci] Update checkout path for nightly build (#9028) * [Mono.Android-Tests] Fix repo URL in redirect tests (#9035) * [Xamarin.Android.Build.Tasks] Support VS "Build Acceleration" (#9042) * [xaprepare] Always use release mono bundle (#9106) * [ci] Update sdk-insertions trigger to manual only (#9029) * Bump to dotnet/runtime@4a37e7305c 8.0.8 (#9033) * Bump to dotnet/installer@b638a84fba 8.0.205-servicing.24212.27 (#9032) * [ci] Disable CodeQL on macOS, Linux, non-main jobs (#9111) * [ci] Use DotNetCoreCLI to sign macOS files (#9102) * [tests] fix `InvalidTargetPlatformVersion` for `net8.0` (#9110) * [ci] Run "Push Internal" job on AzurePipelines-EO pool (#8991) * Bump to dotnet/runtime@2d5c0b720c 8.0.8 (#9123) * [Mono.Android] Data sharing and Close() overrides (#9103) * [Xamarin.Android.Build.Tasks] fix `Inputs` for `_Generate*Java*` targets (#9174) * Bump to xamarin/monodroid@e6a7cf474a (#9193) * [release/8.0.4xx] Backport maestro and artifact drop infra improvements (#9195) * Bump to dotnet/runtime@62f69f1e86 8.0.9 (#9191) * Bump to dotnet/runtime@ed13b35174 8.0.9 (#9221) * [build] Update package metadata (#9230) * [Xamarin.Android.Build.Tasks] Rethink default property values (#9155) (#9224) * [Xamarin.Android.Build.Tasks] Fix an issue with incremental builds (#9183) * [ci] Improve push_signed_nugets job condition (#9240) ~~ Other changes ~~ * Setup Maestro to track .NET8 builds After setting up the appropriate changes in `Version.Details.xml`, I ran: > darc update-dependencies --id 235947 Looking up build with BAR id 235947 Updating 'Microsoft.Android.Sdk.Windows': '1' => '34.0.137' (from build '8.0.4xx- b0fd011-1' of 'https://github.com/dotnet/android') Checking for coherency updates... Using 'Strict' coherency mode. If this fails, a second attempt utilizing 'Legacy' Coherency mode will be made. Local dependencies updated based on build with BAR id 235947 (8.0.4xx-b0fd0113c829edd9bd1bc7d742255a237ff19f69-1 from https://github.com/dotnet/android@release/8.0.4xx)
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Backports the following maestro and artifact storage changes:
bbac9fe
f834b55
9ba9efa
55c2dd8
d050fda
8a85c10