Skip to content

Onboarding to ArPow (arcade-powered source-build) #6387

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
Forgind merged 8 commits into from
Jun 1, 2021
Merged

Onboarding to ArPow (arcade-powered source-build) #6387

Show file tree
Hide file tree
Changes from 6 commits
Commits
Show all changes
8 commits
Select commit Hold shift + click to select a range
1bc455c
ArPow stage 1: local source-build infrastructure
MichaelSimons Apr 29, 2021
5538488
Enable source-build CI
MichaelSimons Apr 29, 2021
f848c41
Integrate source-build patches into product
MichaelSimons Apr 29, 2021
44dda5b
Add source-build leg to official build
MichaelSimons Apr 29, 2021
7f46690
Consume SB arcade intermediate
MichaelSimons May 24, 2021
3847696
Define publish dependency in offical builds
MichaelSimons May 24, 2021
fbc2c99
Update AfterSigning targets to handle ArPow
MichaelSimons May 27, 2021
3c8d829
Merge branch 'main' into ArPow-Stage1
Forgind Jun 1, 2021
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 6 additions & 0 deletions .vsts-dotnet-ci.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -248,3 +248,9 @@ jobs:
ArtifactName: 'MonoOnMac test logs'
continueOnError: true
condition: always()

- template: /eng/common/templates/job/source-build.yml
parameters:
platform:
name: 'Managed'
container: 'mcr.microsoft.com/dotnet-buildtools/prereqs:centos-7-3e800f1-20190501005343'
Comment thread
rainersigwald marked this conversation as resolved.
7 changes: 7 additions & 0 deletions .vsts-dotnet.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -221,12 +221,19 @@ stages:
displayName: Execute cleanup tasks
condition: succeededOrFailed()

- template: /eng/common/templates/job/source-build.yml
parameters:
platform:
name: 'Managed'
container: 'mcr.microsoft.com/dotnet-buildtools/prereqs:centos-7-3e800f1-20190501005343'

- template: /eng/common/templates/job/publish-build-assets.yml
parameters:
enablePublishBuildArtifacts: true
publishUsingPipelines: true
dependsOn:
- Windows_NT
- Source_Build_Managed
Comment thread
rainersigwald marked this conversation as resolved.
pool:
vmImage: vs2017-win2016

Expand Down
2 changes: 1 addition & 1 deletion eng/AfterSigning.targets
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
<Project>
<Target Name="CopyAMD64Symbols" Condition="'$(OfficialBuild)' == 'true'" BeforeTargets="Build">
<Target Name="CopyAMD64Symbols" Condition="'$(OfficialBuild)' == 'true' AND '$(DotNetBuildFromSource)' != 'true'" BeforeTargets="Build">
<Copy
SourceFiles="$(ArtifactsBinDir)MSBuild\x64\Release\net472\MSBuild.pdb"
DestinationFolder="$(ArtifactsSymStoreDirectory)\MSBuild\net472\amd64"
Expand Down
2 changes: 1 addition & 1 deletion eng/Packages.props
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -25,7 +25,7 @@
<PackageReference Update="System.Configuration.ConfigurationManager" Version="4.7.0" />
<PackageReference Update="System.Memory" Version="4.5.4" />
<PackageReference Update="System.Reflection.Metadata" Version="1.6.0" />
<PackageReference Update="System.Resources.Extensions" Version="4.6.0" />
<PackageReference Update="System.Resources.Extensions" Version="$(SystemResourcesExtensionsPackageVersion)" />
Comment thread
rainersigwald marked this conversation as resolved.
<PackageReference Update="System.Runtime.CompilerServices.Unsafe" Version="5.0.0"/>
<PackageReference Update="System.Security.Cryptography.Pkcs" Version="4.7.0" />
<PackageReference Update="System.Security.Cryptography.Xml" Version="4.7.0" />
Expand Down
13 changes: 13 additions & 0 deletions eng/SourceBuild.props
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
<Project>

<PropertyGroup>
<GitHubRepositoryName>msbuild</GitHubRepositoryName>
</PropertyGroup>

<Target Name="ConfigureInnerBuildArgs" BeforeTargets="GetSourceBuildCommandConfiguration">

@benvillalobos benvillalobos May 24, 2021

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A bit more of a design question, should the GetSourceBuildCommandConfiguration target instead have a DependsOnTargets=ConfigureInnerBuildArgs? Or is that step not required of all repos?

I assume it's done this way to prevent repos that don't have the target (that maybe don't care about sourcebuild but use arcade) to outright fail

@MichaelSimons MichaelSimons May 25, 2021

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Correct, ConfigureInnerBuildArgs is not required.

<PropertyGroup>
<InnerBuildArgs>$(InnerBuildArgs) /p:Projects="$(InnerSourceBuildRepoRoot)\MSBuild.SourceBuild.slnf"</InnerBuildArgs>

@benvillalobos benvillalobos May 24, 2021

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How does this file get generated?

@MichaelSimons MichaelSimons May 25, 2021

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you clarify which file you are referring to? The solution filter is checked in - https://github.com/dotnet/msbuild/blob/main/MSBuild.SourceBuild.slnf

</PropertyGroup>
</Target>

</Project>
5 changes: 5 additions & 0 deletions eng/SourceBuildPrebuiltBaseline.xml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
<UsageData>
<IgnorePatterns>
<UsagePattern IdentityGlob="*/*" />
</IgnorePatterns>
</UsageData>
1 change: 1 addition & 0 deletions eng/Version.Details.xml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,7 @@
<Dependency Name="Microsoft.DotNet.Arcade.Sdk" Version="6.0.0-beta.21227.1">
<Uri>https://github.com/dotnet/arcade</Uri>
<Sha>cca78ffe3eefdc217e43c2421f2f23355f16da2d</Sha>
<SourceBuild RepoName="arcade" ManagedOnly="true"/>

@benvillalobos benvillalobos May 24, 2021

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we not need a SourceBuild attribute with ManagedOnly set to true for NuGet.Build.Tasks, Microsoft.Extensions.DependencyModel, or Microsoft.Net.Compilers.Toolset? are they not used for ArPow?

@MichaelSimons MichaelSimons May 25, 2021

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We will be adorning more dependencies with the SourceBuild attribute but not all upstream dependencies have their SourceBuild intermediate NuGet packages created yet or msbuild is not on a version that is source built yet. Once all repos are on ArPow we will make the appropriate updates here to eliminate prebuilts. We will also be turning on prebuilt detection in the source build CI leg to prevent regressions.

@rainersigwald rainersigwald May 25, 2021

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note that of the list, we should only need the Roslyn one. The others are dependencies used to run tests/build a test environment that shouldn't be relevant in sourcebuild.

</Dependency>
<Dependency Name="NuGet.Build.Tasks" Version="5.10.0-rc.7240">
<Uri>https://github.com/nuget/nuget.client</Uri>
Expand Down
1 change: 1 addition & 0 deletions eng/Versions.props
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -23,6 +23,7 @@
</PropertyGroup>
<!-- Production Dependencies -->
<PropertyGroup>
<SystemResourcesExtensionsPackageVersion>4.6.0</SystemResourcesExtensionsPackageVersion>
</PropertyGroup>
<!-- Toolset Dependencies -->
<PropertyGroup>
Expand Down
2 changes: 1 addition & 1 deletion src/MSBuild/MSBuild.csproj
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@
<!-- Set RuntimeIdentifiers so that NuGet will restore for both AnyCPU as well as x86 and x64.
This is important for the MSBuild.VSSetup project, which "references" both the x86 and x64
versions of this project -->
<RuntimeIdentifiers>win7-x86;win7-x64</RuntimeIdentifiers>
<RuntimeIdentifiers Condition="'$(DotNetBuildFromSource)' != 'true'">win7-x86;win7-x64</RuntimeIdentifiers>

<EnableDefaultItems>false</EnableDefaultItems>
<AllowUnsafeBlocks>true</AllowUnsafeBlocks>
Expand Down
2 changes: 1 addition & 1 deletion src/MSBuildTaskHost/MSBuildTaskHost.csproj
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@
<!-- Set RuntimeIdentifiers so that NuGet will restore for both AnyCPU as well as x86 and x64.
This is important for the MSBuild.VSSetup project, which "references" both the x86 and x64
versions of this project -->
<RuntimeIdentifiers>win7-x86;win7-x64</RuntimeIdentifiers>
<RuntimeIdentifiers Condition="'$(DotNetBuildFromSource)' != 'true'">win7-x86;win7-x64</RuntimeIdentifiers>

<EnableDefaultItems>false</EnableDefaultItems>
<DefineConstants>$(DefineConstants);CLR2COMPATIBILITY;TASKHOST</DefineConstants>
Expand Down