Exception: Sequence contains more than one element

[skrysm]

Description

In version 2.2.3 there is a regression (compared to 2.1.2) for this code:

[TestClass]
public sealed class MyTestClass : MyTestClassBase
{
    [TestMethod]
    [TestCategory("MyTests")]
    public new void TestSomething()
    {
        base.TestSomething();
    }
}

public abstract class MyTestClassBase
{
    protected void TestSomething()
    {
        // Do something
    }
}

Executing this via:

"C:\Program Files (x86)\Microsoft Visual Studio\2019\Enterprise\Common7\IDE\CommonExtensions\Microsoft\TestWindow\vstest.console.exe" "MyTests.dll" /TestCaseFilter:"TestCategory=MyTests"

Leads to this error (I had to manually translate it because the error is in German on my machine):

Exception in call of Executor "executor://mstestadapter/v2": The sequence contains more than one element.

This does not happen in version 2.1.2.

Steps to reproduce

1. Create a modern-SDK test project
2. Select net48 as target framework
3. Add code above
4. Reference NuGet packages like this:

    <PackageReference Include="Microsoft.NET.Test.Sdk" Version="16.9.1" />
    <PackageReference Include="MSTest.TestAdapter" Version="2.2.3" />
    <PackageReference Include="MSTest.TestFramework" Version="2.2.3" />

Run the tests via vstest.console (like shown above).

Expected behavior

There should be no exception.

Actual behavior

The exception above is thrown.

Environment

• Windows 10
• Visual Studio 2019 16.7.12
• vstest.console 16.7.0

AB#1335565

[Haplois]

I was able to reproduce this. This issue is because of the Test Platform itself, with its next release this should be fixed.

An exception occurred while invoking executor 'executor://mstestadapter/v2': Sequence contains more than one element
Stack trace:

Server stack trace: 
   at System.Linq.Enumerable.SingleOrDefault[TSource](IEnumerable`1 source)
   at Microsoft.TestPlatform.AdapterUtilities.ManagedNameUtilities.ManagedNameHelper.FindMethod(Type type, String methodName, Int32 methodArity, String[] parameterTypes)
   at Microsoft.TestPlatform.AdapterUtilities.ManagedNameUtilities.ManagedNameHelper.GetMethod(Assembly assembly, String managedTypeName, String managedMethodName)
   at Microsoft.VisualStudio.TestPlatform.MSTest.TestAdapter.Execution.TypeCache.GetMethodInfoUsingManagedNameHelper(TestMethod testMethod, TestClassInfo testClassInfo)
   at Microsoft.VisualStudio.TestPlatform.MSTest.TestAdapter.Execution.TypeCache.GetMethodInfoForTestMethod(TestMethod testMethod, TestClassInfo testClassInfo)
   at Microsoft.VisualStudio.TestPlatform.MSTest.TestAdapter.Execution.TypeCache.ResolveTestMethod(TestMethod testMethod, TestClassInfo testClassInfo, ITestContext testContext, Boolean captureDebugTraces)
   at Microsoft.VisualStudio.TestPlatform.MSTest.TestAdapter.Execution.TypeCache.GetTestMethodInfo(TestMethod testMethod, ITestContext testContext, Boolean captureDebugTraces)
   at Microsoft.VisualStudio.TestPlatform.MSTest.TestAdapter.Execution.UnitTestRunner.RunSingleTest(TestMethod testMethod, IDictionary`2 testContextProperties)
   at System.Runtime.Remoting.Messaging.StackBuilderSink._PrivateProcessMessage(IntPtr md, Object[] args, Object server, Object[]& outArgs)
   at System.Runtime.Remoting.Messaging.StackBuilderSink.SyncProcessMessage(IMessage msg)

Exception rethrown at [0]: 
   at System.Runtime.Remoting.Proxies.RealProxy.HandleReturnMessage(IMessage reqMsg, IMessage retMsg)
   at System.Runtime.Remoting.Proxies.RealProxy.PrivateInvoke(MessageData& msgData, Int32 type)
   at Microsoft.VisualStudio.TestPlatform.MSTest.TestAdapter.Execution.UnitTestRunner.RunSingleTest(TestMethod testMethod, IDictionary`2 testContextProperties)
   at Microsoft.VisualStudio.TestPlatform.MSTest.TestAdapter.Execution.TestExecutionManager.ExecuteTestsWithTestRunner(IEnumerable`1 tests, IRunContext runContext, ITestExecutionRecorder testExecutionRecorder, String source, IDictionary`2 sourceLevelParameters, UnitTestRunner testRunner)
   at Microsoft.VisualStudio.TestPlatform.MSTest.TestAdapter.Execution.TestExecutionManager.ExecuteTestsInSource(IEnumerable`1 tests, IRunContext runContext, IFrameworkHandle frameworkHandle, String source, Boolean isDeploymentDone)
   at Microsoft.VisualStudio.TestPlatform.MSTest.TestAdapter.Execution.TestExecutionManager.ExecuteTests(IEnumerable`1 tests, IRunContext runContext, IFrameworkHandle frameworkHandle, Boolean isDeploymentDone)
   at Microsoft.VisualStudio.TestPlatform.MSTest.TestAdapter.Execution.TestExecutionManager.RunTests(IEnumerable`1 tests, IRunContext runContext, IFrameworkHandle frameworkHandle, TestRunCancellationToken runCancellationToken)
   at Microsoft.VisualStudio.TestPlatform.MSTest.TestAdapter.MSTestExecutor.RunTests(IEnumerable`1 tests, IRunContext runContext, IFrameworkHandle frameworkHandle)
   at Microsoft.VisualStudio.TestPlatform.CrossPlatEngine.Execution.RunTestsWithTests.InvokeExecutor(LazyExtension`2 executor, Tuple`2 executorUri, RunContext runContext, IFrameworkHandle frameworkHandle)
   at Microsoft.VisualStudio.TestPlatform.CrossPlatEngine.Execution.BaseRunTests.<>c__DisplayClass48_0.<RunTestInternalWithExecutors>b__0()
   at Microsoft.VisualStudio.TestPlatform.PlatformAbstractions.PlatformThread.<>c__DisplayClass0_0.<Run>b__0()
--- End of stack trace from previous location where exception was thrown ---
   at Microsoft.VisualStudio.TestPlatform.PlatformAbstractions.PlatformThread.Run(Action action, PlatformApartmentState apartmentState, Boolean waitForCompletion)
   at Microsoft.VisualStudio.TestPlatform.CrossPlatEngine.Execution.BaseRunTests.TryToRunInSTAThread(Action action, Boolean waitForCompletion)
   at Microsoft.VisualStudio.TestPlatform.CrossPlatEngine.Execution.BaseRunTests.RunTestInternalWithExecutors(IEnumerable`1 executorUriExtensionMap, Int64 totalTests)

[skrysm]

This issue is because of the Test Platform itself, with its next release this should be fixed.

So I've update the "Microsoft.NET.Test.Sdk" package to 16.10.0 and "MSTest.TestAdapter" and "MSTest.TestFramework" to 2.2.4 - but the error is still happening. Am I missing something here?

What exactly is the "Test Platform"?

[Haplois]

Microsoft.NET.Test.Sdk package is the Test Platform. I'll look into this and let you know about the progress.

[Haplois]

The issue is in Microsoft.TestPlatform.AdapterUtilities.ManagedNameUtilities.GetMethod, this method doesn't support method hiding. This will be fixed in our 16.11 release. Moving this issue to https://github.com/microsoft/vstest.

[skrysm]

The strange thing is that it's working with MSTest.TestFramework/MSTest.TestFramework in version 2.1.2 - it's just broken in version 2.2.x. But I guess they've changed something in the test framework, right?

[Haplois]

Yes, we switched to a more robust test resolving mechanism provided by the Test Platform v16.10, but it doesn't handle hidden methods well. That needs to be fixed for this to work.

For now, as a workaround, you can make the base method and override it if possible, or you can rename your test.

[skrysm]

This will be fixed in our 16.11 release.

So, now that 16.11 is released, I was trying to see whether the bug is finally fixed. But it's not yet fixed. Did the fix not make it into the 16.11 release?

[skrysm]

Just as a side note: Apparently some change in either the "MSTest.TestAdapter" or the "MSTest.TestFramework" package in version 2.2.4 changed the behavior so that we now get a warning rather than an error. Unfortunately, this also means that tests that run into this problem are being "lost" because a CI pipeline can no longer detect this error.

[deWisch]

Any news on this issue? It is still present in the latest version.

[Naxemar]

Any news ?