-
Notifications
You must be signed in to change notification settings - Fork 4.9k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[Build] Console output lost #97211
Comments
Tagging subscribers to this area: @dotnet/runtime-infrastructure Issue DetailsAfter update to the newest tools, the output from msbuild has changed and now auto-updates an existing line in terminal instead of printing all output out. As a result, the important information about failing tests is lost and this is what remains: dotnet build tests/FunctionalTests/ -t:test
MSBuild version 17.10.0-preview-24060-03+1725b247e for .NET
Restore complete (2.6s)
TestUtilities net8.0 succeeded (4.2s) → /home/manicka/repositories/runtime/artifacts/bin/TestUtilities/Debug/net8.0/TestUtilities.dll
System.Net.Quic.Functional.Tests net9.0-linux failed with errors (30.3s) → /home/manicka/repositories/runtime/artifacts/bin/System.Net.Quic.Functional.Tests/Debug/net9.0-linux/System.Net.Quic.Functional.Tests.dll
/home/manicka/repositories/runtime/eng/testing/tests.targets(195,5): error : One or more tests failed while running tests from 'System.Net.Quic.Functional.Tests'. [/home/manicka/repositories/runtime/src/libraries/System.Net.Quic/tests/FunctionalTests/System.Net.Quic.Functional.Tests.csproj]
Build failed with errors in 40.1s No failing test name, no failure details, no clickable link to I do not know if this is an issue in our infra (the way we call msbuild) or arcade or msbuild itself, so feel free to transfer this issue.
|
That's the new msbuild terminal logger (dotnet/sdk#37492), you can disable it via Looks like there are some improvements coming for dotnet test as well: dotnet/sdk#38098. |
Yes, please. I just got bit by this as well. Why is not showing the test failures a better experience, @rainersigwald? |
Local workaroundSet the environment variable Committable workaround
You can't do this within MSBuild based on requested target; you'll need to disable -tl entirely (easiest way would be to put Ok but what's going on?
They would not, the critical change there is what logging/warning/error APIs the task running the tests calls, and Arcade tests do that all manually.
It isn't. What happened is that build output is so polluted by "high" priority messages that the terminal logger mostly disregards them. Here, it sounds like all of the debugging test output is given in a message, rather than in the error, which combines badly with that. The plan for It should be possible to change the repo/Arcade test infrastructure to provide more details in a way that is more MSBuild-y. I'll take a look. |
There's already logic that tries to include the TRX in the failure message, which works for me when I artificially induce a failure in STJ tests. runtime/eng/testing/tests.targets Lines 189 to 195 in 5a6135e
❯ dotnet build -t:test -bl
MSBuild version 17.10.0-preview-24060-03+1725b247e for .NET
Restore complete (1.0s)
...
System.Text.Json net462 succeeded (0.7s) → S:\runtime\artifacts\bin\System.Text.Json\Debug\net462\System.Text.Json.dll
System.Text.Json.Tests net462 failed with errors (1.4s) → S:\runtime\artifacts\bin\System.Text.Json.Tests\Debug\net462\System.Text.Json.Tests.dll
S:\runtime\eng\testing\tests.targets(195,5): error : One or more tests failed while running tests from 'System.Text.Json.Tests'. Please check S:\runtime\artifacts\bin\System.Text.Json.Tests\Debug\net462\testResults.xml for details! [S:\runtime\src\libraries\System.Text.Json\tests\System.Text.Json.Tests\System.Text.Json.Tests.csproj]
Build failed with errors in 8.4s I don't know why that didn't trigger for you, @ManickaP. Of course, that's still less data than the super verbose dump you had before
Trying to see if we can conditionally emit that on test failure. |
It says that there are errors, but not what those errors are. That's my problem. To actually see the errors I then need to go load up the XML file from xunit and search through it looking for what went wrong. |
Thanks! |
How would y'all feel about this? text version
It loses the leading whitespace (dotnet/msbuild#9668) but is fairly close to the prior output and can be done with a small patch in this repo (15c21fe). |
note that CI test runs do not preserver the XML any more AFAIK. e.g. the console output is only once clue we have in many cases. |
do you have example with deeper stack @rainersigwald ? |
The terminal logger should not be involved in CI test runs--we should detect that output is being redirected and fall back to the older output behavior. If you see otherwise that is a bug in MSBuild. But this is good to know--does it mean you would want this output all the time, not just on test failure? |
Sure: |
Not ideal, since without color the whitespace is the main indicator of where one failure ends and the next begins. It's a lot harder to skim the output for pertinent info. But certainly better than no test output at all. |
My failure was a crash ( |
For reference, a code base at my work has been struggling with the lack of test run failure information in CI for like half a year, and that has been building against .NET 6. I haven't been able to find a single log file or anything that even points to why the CI |
@akoeplinger, could you help disable the new thing repo wide until these issues are addressed? Thanks! @rainersigwald, let's make sure crashes/asserts are visible as well. Thanks in advance for fixing it! |
* Disable msbuild terminal logger #97211 * Update Directory.Build.rsp
* Disable msbuild terminal logger dotnet#97211 * Update Directory.Build.rsp
We plan to make terminal logger to respect verbosity level. After this feature is done, high-importance messages would be accessible if one set verbosity to detailed or diagnostic (default dotnet verbosity setting is Here is the example of the more verbose run with the terminal logger:
Currently the verbosity level is set to |
This is more question for our infra @akoeplinger @ViktorHofer. |
We definitely can't switch to |
I talked to @rainersigwald and we have got one more idea here: we will allow to set a specific verbosity for the Terminal Logger with |
Closing this issue as the initial problem has been resolved. New issues can be tracked separately. |
After update to the newest tools, the output from msbuild has changed and now auto-updates an existing line in terminal instead of printing all output out. As a result, the important information about failing tests is lost and this is what remains:
No failing test name, no failure details, no clickable link to
testResults.xml
.I do not know if this is an issue in our infra (the way we call msbuild) or arcade or msbuild itself, so feel free to transfer this issue.
The text was updated successfully, but these errors were encountered: