Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Unit Test] Cannot run unit tests from VSCode when using NUnit's TestFixtureSource #1734

Closed
Strachu opened this issue Aug 23, 2017 · 5 comments

Comments

@Strachu
Copy link

Strachu commented Aug 23, 2017

I'm not sure whether this is a C# extension's bug or NUnit, filing here as when running dotnet test everything works.

Environment data

dotnet --info output:

.NET Command Line Tools (1.0.4)

Product Information:
Version: 1.0.4
Commit SHA-1 hash: af1e6684fd

Runtime Environment:
OS Name: ubuntu
OS Version: 16.04
OS Platform: Linux
RID: ubuntu.16.04-x64
Base Path: /usr/share/dotnet/sdk/1.0.4

VS Code version: Version 1.15.1
C# Extension version: 1.12.1

Steps to reproduce

Extract VSCodeBug.zip and open it in VS Code. After restoring packages click "run test" or "debug test" on method Test.

The project uses NUnit and its TestFixtureSourceAttribute to run the test fixture multiple times with different data.

Expected behavior

When placing breakpoint at the beginning of the mehod and clicking "debug test" the debugger should hit the breakpoint.
When clicking "run test" the test should be executed 10 times with results in log.

Actual behavior

When I click "debug test" I see:

Debugging method 'AA.UnitTest1.Test'...

Microsoft (R) Build Engine version 15.1.1012.6693
Copyright (C) Microsoft Corporation. All rights reserved.

VSCodeDebuggerBug -> /home/strachu/VSCodeDebuggerBug/bin/Debug/netcoreapp1.1/VSCodeDebuggerBug.dll

Build succeeded.
0 Warning(s)
0 Error(s)

Time Elapsed 00:00:01.94

NUnit Adapter 3.8.0.0: Test discovery starting
NUnit Adapter 3.8.0.0: Test discovery complete

and nothing happens, no breakpoint is hit, after clicking "debug test" again I get:

Microsoft (R) Build Engine version 15.1.1012.6693
Microsoft (R) Build Engine version 15.1.1012.6693
Copyright (C) Microsoft Corporation. All rights reserved.
Copyright (C) Microsoft Corporation. All rights reserved.

VSCodeDebuggerBug -> /home/strachu/VSCodeDebuggerBug/bin/Debug/netcoreapp1.1/VSCodeDebuggerBug.dll
VSCodeDebuggerBug -> /home/strachu/VSCodeDebuggerBug/bin/Debug/netcoreapp1.1/VSCodeDebuggerBug.dll

Build succeeded.
Build succeeded.
0 Warning(s)
0 Warning(s)
0 Error(s)
0 Error(s)

Time Elapsed 00:00:00.98
Time Elapsed 00:00:00.98

Notice the duplicate entries, also a popup with "Failed to start debugger: null" error shows. A new entry is added after every run (after 5 runs, the messages are duplicated 5 times).

When I click ''run tests" I see:

Running test AA.UnitTest1.Test...

Microsoft (R) Build Engine version 15.1.1012.6693
Copyright (C) Microsoft Corporation. All rights reserved.

VSCodeDebuggerBug -> /home/strachu/VSCodeDebuggerBug/bin/Debug/netcoreapp1.1/VSCodeDebuggerBug.dll

Build succeeded.
0 Warning(s)
0 Error(s)

Time Elapsed 00:00:00.98

NUnit Adapter 3.8.0.0: Test discovery starting
NUnit Adapter 3.8.0.0: Test discovery complete

Total tests: 0. Passed: 0. Failed: 0. Skipped: 0

So VS Code haven't found any test to run.

When executing the same code with dotnet test I see:

Build started, please wait...
Build completed.

Test run for /home/strachu/VSCodeDebuggerBug/bin/Debug/netcoreapp1.1/VSCodeDebuggerBug.dll(.NETCoreApp,Version=v1.1)
Microsoft (R) Test Execution Command Line Tool Version 15.0.0.0
Copyright (c) Microsoft Corporation. All rights reserved.

Starting test execution, please wait...
NUnit Adapter 3.8.0.0: Test execution started
Running all tests in /home/strachu/VSCodeDebuggerBug/bin/Debug/netcoreapp1.1/VSCodeDebuggerBug.dll
NUnit3TestExecutor converted 10 of 10 NUnit test cases
NUnit Adapter 3.8.0.0: Test execution complete

Total tests: 10. Passed: 10. Failed: 0. Skipped: 0.
Test Run Successful.
Test execution time: 0.9933 Seconds

So from command line everything works.

When I remove the TestFixtureSourceAttribute and a ctor from the class and invoke it from VSCode, 1 test is run, so this case is working correctly, the problem is when using TestFixtureSourceAttribute.

@Strachu
Copy link
Author

Strachu commented Aug 23, 2017

Just noting that the duplicate entries in logs and a popup with "Failed to start debugger: null" error on a second try with "debug this" seems to be an unrelated bug as the same happens when the debugger fails to start due to a compilation error or when clicking again "debug this" too fast on a test not triggering original bug.
After this it is impossible to debug again without restarting Visual Studio Code.

@Defee
Copy link

Defee commented Sep 16, 2019

Is there any updates on it? It seems I face the same issue now.

@ManfredLange
Copy link

ManfredLange commented Dec 11, 2020

This is still a problem and prevents the use of NUnit's TestFixtureSource in combination with Omnisharp. Here is another example (gist) that works with dotnet test but fails with the following error when clicking the debug line in the source code:

{
  "Request_seq": 135,
  "Command": "/v2/debugtest/getstartinfo",
  "Running": true,
  "Success": false,
  "Message": "\"System.NullReferenceException: Object reference not set to an instance of an object\\n  at OmniSharp.DotNetTest.VSTestManager.DebugGetStartInfoAsync (System.String[] methodNames, System.String runSettings, System.String testFrameworkName, System.String targetFrameworkVersion, System.Threading.CancellationToken cancellationToken) [0x00113] in <4f61ed4d73fa416e988571e27def90ec>:0 \\n  at OmniSharp.Endpoint.EndpointHandler`2[TRequest,TResponse].GetFirstNotEmptyResponseFromHandlers (OmniSharp.Endpoint.Exports.ExportHandler`2[TRequest,TResponse][] handlers, TRequest request) [0x00099] in <def28aea2ee84c5d8f408bdb08c9c4ba>:0 \\n  at OmniSharp.Endpoint.EndpointHandler`2[TRequest,TResponse].HandleRequestForLanguage (System.String language, TRequest request, OmniSharp.Protocol.RequestPacket packet) [0x00163] in <def28aea2ee84c5d8f408bdb08c9c4ba>:0 \\n  at OmniSharp.Endpoint.EndpointHandler`2[TRequest,TResponse].Process (OmniSharp.Protocol.RequestPacket packet, OmniSharp.Endpoint.LanguageModel model, Newtonsoft.Json.Linq.JToken requestObject) [0x0024b] in <def28aea2ee84c5d8f408bdb08c9c4ba>:0 \\n  at OmniSharp.Stdio.Host.HandleRequest (System.String json, Microsoft.Extensions.Logging.ILogger logger) [0x000e7] in <bc69fa58846f4d298e971dab810bd026>:0 \"",
  "Body": null,
  "Seq": 254,
  "Type": "response"
}

My configuration:

dotnet --info output:

.NET SDK (reflecting any global.json):
 Version:   5.0.100
 Commit:    5044b93829

Runtime Environment:
 OS Name:     debian
 OS Version:  10
 OS Platform: Linux
 RID:         debian.10-x64
 Base Path:   /usr/share/dotnet/sdk/5.0.100/

Host (useful for support):
  Version: 5.0.0
  Commit:  cf258a14b7

.NET SDKs installed:
  5.0.100 [/usr/share/dotnet/sdk]

.NET runtimes installed:
  Microsoft.AspNetCore.App 5.0.0 [/usr/share/dotnet/shared/Microsoft.AspNetCore.App]
  Microsoft.NETCore.App 5.0.0 [/usr/share/dotnet/shared/Microsoft.NETCore.App]

To install additional .NET runtimes or SDKs:
  https://aka.ms/dotnet-download

VS Code version: 1.51.1
C# extension version: 1.23.7
NUnit version 3.12.0

Restarting Omnisharp and/or VS Code makes no difference.

@kseeman
Copy link

kseeman commented Mar 17, 2023

I'm currently also experiencing this same issue with the following configuration:

VS Code Version: 1.74.3
C# extension version: 1.25.4
NUnit version: 3.13.3

Has there been any progress on this or at least a work around?

@JoeRobich
Copy link
Member

The C# extension has a minimal unit testing experience that may not work with every testing framework feature. The C# DevKit has a more robust unit testing experience.

@JoeRobich JoeRobich closed this as not planned Won't fix, can't repro, duplicate, stale Nov 8, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

7 participants