Direct package references on System.Collections.Immutable and System.Reflection.Metadata are causing build issues in source-build

[NikolaMilosavljevic]

Related to: dotnet/source-build#4344

With #100595, source-build got broken in release/8.0.1xx, due to package downgrade issue. Source-build consumes MSBuild 17.8.5, while runtime is on 17.8.3. Source-build will update MSBuild package references to live ones (17.8.5), but two runtime projects had direct package references, on S.C.I and S.R.M which caused package downgrade warning (prompted to error in source-build).

The fix would be to remove these direct package references as they will be transitively resolved from MSBuild package reference. This will allow both source-build and repo build to get the correct sets of references.

The change was implemented in source-build as a patch, with dotnet/installer#19554

[dotnet-policy-service]

Tagging subscribers to this area: @dotnet/area-infrastructure-libraries
See info in area-owners.md if you want to be subscribed.

[ViktorHofer]

cc @ericstj

[dotnet-policy-service]

Tagging subscribers to this area: @dotnet/runtime-infrastructure
See info in area-owners.md if you want to be subscribed.

[ericstj]

Yes, I discussed this with @NikolaMilosavljevic and supported this change as the least breaking. Probably someone should have a look at what dependencies build tools like WasmAppBuilder Microsoft.NET.Sdk.WebAssembly.Pack.Tasks and MonoAOTCompiler take and how those behave in both servicing and source build. @lewing @agocke since these are things your team delivers can you help drive the right fix here?

[NikolaMilosavljevic]

I have a slightly better change in https://dev.azure.com/dnceng/internal/_git/dotnet-installer/pullrequest/39154. Two instances are resolved by removing direct package references. Third one is solved by adding missing MSBuild dependencies.

That PR has the following two instances resolved by removing package references:

2024-04-20T01:46:27.0232624Z     /vmr/src/runtime/artifacts/source-build/self/src/src/tasks/WasmAppBuilder/WasmAppBuilder.csproj : error NU1605: Warning As Error: Detected package downgrade: System.Collections.Immutable from 8.0.0 to 7.0.0. Reference the package directly from the project to select a different version.  [/vmr/src/runtime/artifacts/source-build/self/src/Build.proj]
2024-04-20T01:46:27.0233175Z     /vmr/src/runtime/artifacts/source-build/self/src/src/tasks/WasmAppBuilder/WasmAppBuilder.csproj : error NU1605:  WasmAppBuilder -> Microsoft.Build 17.8.5 -> System.Collections.Immutable (>= 8.0.0)  [/vmr/src/runtime/artifacts/source-build/self/src/Build.proj]
2024-04-20T01:46:27.0233652Z     /vmr/src/runtime/artifacts/source-build/self/src/src/tasks/WasmAppBuilder/WasmAppBuilder.csproj : error NU1605:  WasmAppBuilder -> System.Collections.Immutable (>= 7.0.0) [/vmr/src/runtime/artifacts/source-build/self/src/Build.proj]
2024-04-20T01:46:27.0234237Z     /vmr/src/runtime/artifacts/source-build/self/src/src/tasks/WasmAppBuilder/WasmAppBuilder.csproj : error NU1605: Warning As Error: Detected package downgrade: System.Reflection.Metadata from 8.0.0 to 7.0.0. Reference the package directly from the project to select a different version.  [/vmr/src/runtime/artifacts/source-build/self/src/Build.proj]
2024-04-20T01:46:27.0234771Z     /vmr/src/runtime/artifacts/source-build/self/src/src/tasks/WasmAppBuilder/WasmAppBuilder.csproj : error NU1605:  WasmAppBuilder -> Microsoft.Build 17.8.5 -> System.Reflection.Metadata (>= 8.0.0)  [/vmr/src/runtime/artifacts/source-build/self/src/Build.proj]
2024-04-20T01:46:27.0235232Z     /vmr/src/runtime/artifacts/source-build/self/src/src/tasks/WasmAppBuilder/WasmAppBuilder.csproj : error NU1605:  WasmAppBuilder -> System.Reflection.Metadata (>= 7.0.0) [/vmr/src/runtime/artifacts/source-build/self/src/Build.proj]
2024-04-20T01:46:27.0237719Z     /vmr/src/runtime/artifacts/source-build/self/src/src/tasks/AotCompilerTask/MonoAOTCompiler.csproj : error NU1605: Warning As Error: Detected package downgrade: System.Reflection.Metadata from 8.0.0 to 7.0.0. Reference the package directly from the project to select a different version.  [/vmr/src/runtime/artifacts/source-build/self/src/Build.proj]
2024-04-20T01:46:27.0238330Z     /vmr/src/runtime/artifacts/source-build/self/src/src/tasks/AotCompilerTask/MonoAOTCompiler.csproj : error NU1605:  MonoAOTCompiler -> Microsoft.Build 17.8.5 -> System.Reflection.Metadata (>= 8.0.0)  [/vmr/src/runtime/artifacts/source-build/self/src/Build.proj]
2024-04-20T01:46:27.0238868Z     /vmr/src/runtime/artifacts/source-build/self/src/src/tasks/AotCompilerTask/MonoAOTCompiler.csproj : error NU1605:  MonoAOTCompiler -> System.Reflection.Metadata (>= 7.0.0) [/vmr/src/runtime/artifacts/source-build/self/src/Build.proj]

The following was solved by adding the missing 3 MSBuild package dependencies in Version.Details.xml, and corresponding change in Versions.props:

2024-04-20T01:46:27.0235861Z     /vmr/src/runtime/artifacts/source-build/self/src/src/tasks/Microsoft.NET.Sdk.WebAssembly.Pack.Tasks/Microsoft.NET.Sdk.WebAssembly.Pack.Tasks.csproj : error NU1605: Warning As Error: Detected package downgrade: Microsoft.Build.Framework from 17.8.5 to 17.8.3. Reference the package directly from the project to select a different version.  [/vmr/src/runtime/artifacts/source-build/self/src/Build.proj]
2024-04-20T01:46:27.0236512Z     /vmr/src/runtime/artifacts/source-build/self/src/src/tasks/Microsoft.NET.Sdk.WebAssembly.Pack.Tasks/Microsoft.NET.Sdk.WebAssembly.Pack.Tasks.csproj : error NU1605:  Microsoft.NET.Sdk.WebAssembly.Pack.Tasks -> Microsoft.Build 17.8.5 -> Microsoft.Build.Framework (>= 17.8.5)  [/vmr/src/runtime/artifacts/source-build/self/src/Build.proj]
2024-04-20T01:46:27.0237107Z     /vmr/src/runtime/artifacts/source-build/self/src/src/tasks/Microsoft.NET.Sdk.WebAssembly.Pack.Tasks/Microsoft.NET.Sdk.WebAssembly.Pack.Tasks.csproj : error NU1605:  Microsoft.NET.Sdk.WebAssembly.Pack.Tasks -> Microsoft.Build.Framework (>= 17.8.3) [/vmr/src/runtime/artifacts/source-build/self/src/Build.proj]

[lewing]

@akoeplinger can you take a look

[akoeplinger]

We did some improvements around these msbuild tasks in main with 994e272 and 4f09909, but @ViktorHofer has a lot more context on these package version dependencies :)

The change from #101395 (comment) does make sense to me.