`dotnet test` help is broken for MTP when run on solution

[Youssef1313]

xunit/xunit#3375 (comment)

[Youssef1313]

@bradwilson This might be an issue on your side where a class lib project has IsTestingPlatformApplication=true, causing dotnet test to try to initiate its help, and failing. Can you double check that please?

NOTE: Microsoft.Testing.Platform NuGet package sets the property to true in props file, but you need to manually set it to false for class libs. This might also be an existing visible issue for you in IDE Solution Explorer where a non-test project is showing with the test project icon.

The thing on our side though is that when printing this error, we should indicate which project it's complaining about.

[bradwilson]

The list comes up twice. One is obviously xunit.v3.runner.inproc.console since that's where all the MTP code lives. Fixing that one only solves half the problem, though:

I guess I'm left waiting for your error update to figure out which project it's upset about. (FWIW, I don't see the same issue when using just a standard class library and test project.)

[Youssef1313]

@bradwilson I think the left over is from xunit.v2.tests.csproj. The error message is still confusing (even after we will add the project name).

The correct error message should say that xunit.v2.tests.csproj is a "VSTest" project which isn't supported by MTP's dotnet test.

[bradwilson]

That's interesting that it thinks that, since there are no references anywhere to either Microsoft.NET.Test.Sdk nor xunit.runner.visualstudio. Did it... guess?

[Youssef1313]

The expected behavior is if a project has IsTestProject=true and IsTestingPlatformApplication!=true, we consider it as "VSTest". So, when opting-in via dotnet.config, we don't support such project. So, we error on xunit.v2.tests.csproj. But today, we are producing the "wrong" error for this case.

[bradwilson]

Also, the MSBuild override does not seem to have done anything, if your guess was correct:

[Youssef1313]

@bradwilson To clarify more, xunit.v2.tests.csproj already has IsTestProject=true and IsTestingPlatformApplication!=true. We incorrectly attempt to calculate the run properties on it (which it fails). We should skip calculating run properties on it, and then at a later stage a different error will be produced saying "This is a VSTest project, it's not supported when using MTP".