[Broken Build]: MSB4166 after upgrading to SDK 8.0.400: 'Child node "2" exited prematurely'

[mm-jhn]

Issue Description

We upgraded to SDK 8.0.400 (with MSBuild 17.11.3) and now the dotnet publish command that creates a Docker image fails with error MSB4166 in the CI, preventing us from deploying new builds.

The failure appears to occur very late in the build process because the building of all C# projects succeeds and the generated Docker image is pushed to the local registry, then it fails without any indication as to where or why.

Steps to Reproduce

The project and the binlog contain information under NDA by the customer and cannot be shared. I can only share a screenshot from the binlog viewer.
The CI pipeline runs in Azure Devops Services (managed agent pool) and uses the UseDotNet@2 task to install the SDK.

Invocations:

• CLI: dotnet publish src/redacted.csproj -c Release --os linux --arch x64 -p:PublishProfile=DefaultContainer -p:SourceRevisionId=$(Build.SourceVersion)
• Internal: /opt/hostedtoolcache/dotnet/sdk/8.0.400/MSBuild.dll -maxcpucount -verbosity:m -nologo -restore -target:Publish --property:_IsPublishing=true --property:PublishProfile=DefaultContainer --property:SourceRevisionId=<some-commit-hash> -property:Configuration=Release -property:RuntimeIdentifier=linux-x64 -property:DOTNET_CLI_DISABLE_PUBLISH_AND_PACK_RELEASE=true /home/vsts/work/1/s/src/redacted.csproj -bl:/home/vsts/work/1/a/build.binlog -distributedlogger:Microsoft.DotNet.Tools.MSBuild.MSBuildLogger,/opt/hostedtoolcache/dotnet/sdk/8.0.400/dotnet.dll*Microsoft.DotNet.Tools.MSBuild.MSBuildForwardingLogger,/opt/hostedtoolcache/dotnet/sdk/8.0.400/dotnet.dll

Expected Behavior

The build succeeds like it did under any previous 8.0 SDK.

Actual Behavior

MSBUILD : error MSB4166: Child node "2" exited prematurely. Shutting down. Diagnostic information may be found in files in "/tmp/MSBuildTempvsts/" and will be named MSBuild_*.failure.txt. This location can be changed by setting the MSBUILDDEBUGPATH environment variable to a different directory.
MSBUILD : error MSB4166: /tmp/MSBuildTempvsts/MSBuild_pid-1799_c78c35fa40c04dadaff20d14eb74ecd3.failure.txt:
MSBUILD : error MSB4166: UNHANDLED EXCEPTIONS FROM PROCESS 1799:
MSBUILD : error MSB4166: =====================
MSBUILD : error MSB4166: 08/14/2024 07:31:41
MSBUILD : error MSB4166: System.ArgumentNullException: Value cannot be null. (Parameter 'value')
MSBUILD : error MSB4166:    at System.ArgumentNullException.Throw(String paramName)
MSBUILD : error MSB4166:    at System.IO.BinaryWriter.Write(String value)
MSBUILD : error MSB4166:    at Microsoft.Build.Framework.TelemetryEventArgs.WriteToStream(BinaryWriter writer)
MSBUILD : error MSB4166:    at Microsoft.Build.Shared.LogMessagePacketBase.WriteToStream(ITranslator translator)
MSBUILD : error MSB4166:    at Microsoft.Build.Shared.LogMessagePacketBase.Translate(ITranslator translator)
MSBUILD : error MSB4166:    at Microsoft.Build.BackEnd.NodeEndpointOutOfProcBase.RunReadLoop(Stream localReadPipe, Stream localWritePipe, ConcurrentQueue`1 localPacketQueue, AutoResetEvent localPacketAvailable, AutoResetEvent localTerminatePacketPump)
MSBUILD : error MSB4166: ===================
MSBUILD : error MSB4166: 
MSBUILD : error MSB4166: 

Ask us questions

No response

[mm-jhn]

The same behaviour also occurs locally (SDK 8.0.400, Windows 11). Of note here is that adding -nodeReuse:false to the command line does not mitigate this problem. The stack trace in the diagnostic file is the same (the node ID can differ though).

Invocation: dotnet publish .\src\redacted.csproj -c Release --os linux --arch x64 -t:PublishContainer -nodeReuse:false /bl

Without a fix or mitigation, this is a blocker for us and we have to downgrade to SDK 8.0.3xx.

[baronfel]

This is a hard crash in MSBuild that we have a hotfix SDK coming out for quite soon, aiming for tomorrow. The only fix for now is to use an older SDK version. Details about this problem can be found at dotnet/sdk#42719 (comment), I'm going to close this issue as a duplicate of it.