NUnit3TestAdapter 3.8.0-alpha1 - Win32Exception: Permission denied on TravisCI for DesktopClr(net461)

[ghost]

Thanks for releasing this. I tried it out today and it works really well on Windows but when I use it on Linux I get a Win32Exception.

https://travis-ci.org/fir3pho3nixx/Core/builds/229245536#L1899

Test run for /home/travis/build/fir3pho3nixx/Core/src/Castle.Core.Tests/bin/Debug/net461/Castle.Core.Tests.dll(.NETFramework,Version=v4.6.1)
Microsoft (R) Test Execution Command Line Tool Version 15.0.0.0
Copyright (c) Microsoft Corporation.  All rights reserved.
Starting test execution, please wait...
Unhandled Exception: System.ComponentModel.Win32Exception: Permission denied
   at Interop.Sys.ForkAndExecProcess(String filename, String[] argv, String[] envp, String cwd, Boolean redirectStdin, Boolean redirectStdout, Boolean redirectStderr, Int32& lpChildPid, Int32& stdinFd, Int32& stdoutFd, Int32& stderrFd)
   at System.Diagnostics.Process.StartCore(ProcessStartInfo startInfo)
   at System.Diagnostics.Process.Start()
   at Microsoft.VisualStudio.TestPlatform.CrossPlatEngine.Helpers.ProcessHelper.LaunchProcess(String processPath, String arguments, String workingDirectory, Action`1 exitCallback)
   at Microsoft.VisualStudio.TestPlatform.CrossPlatEngine.Hosting.DefaultTestHostManager.LaunchTestHost(TestProcessStartInfo testHostStartInfo)
   at Microsoft.VisualStudio.TestPlatform.CrossPlatEngine.Client.ProxyOperationManager.SetupChannel(IEnumerable`1 sources)
   at Microsoft.VisualStudio.TestPlatform.CrossPlatEngine.Client.ProxyExecutionManager.InitializeExtensions(IEnumerable`1 sources)
   at Microsoft.VisualStudio.TestPlatform.CrossPlatEngine.Client.ProxyExecutionManager.Initialize()
   at Microsoft.VisualStudio.TestPlatform.Client.TestPlatform.CreateTestRunRequest(TestRunCriteria testRunCriteria)
   at Microsoft.VisualStudio.TestPlatform.CommandLine.TestPlatformHelpers.TestRequestManager.RunTests(TestRunCriteria testRunCriteria, ITestRunEventsRegistrar testRunEventsRegistrar)
   at Microsoft.VisualStudio.TestPlatform.CommandLine.TestPlatformHelpers.TestRequestManager.RunTests(TestRunRequestPayload testRunRequestPayload, ITestHostLauncher testHostLauncher, ITestRunEventsRegistrar testRunEventsRegistrar)
   at Microsoft.VisualStudio.TestPlatform.CommandLine.Processors.RunTestsArgumentExecutor.RunTests(IEnumerable`1 sources)
   at Microsoft.VisualStudio.TestPlatform.CommandLine.Processors.RunTestsArgumentExecutor.Execute()
   at Microsoft.VisualStudio.TestPlatform.CommandLine.Executor.ExecuteArgumentProcessor(IArgumentProcessor processor, Int32& exitCode)
   at Microsoft.VisualStudio.TestPlatform.CommandLine.Executor.Execute(String[] args)
   at Microsoft.VisualStudio.TestPlatform.CommandLine.Program.Main(String[] args)

I can also reproduce the error on Ubuntu 16.04 with the following dotnet --info:

.NET Command Line Tools (1.0.1)

Product Information:
 Version:            1.0.1
 Commit SHA-1 hash:  005db40cd1

Runtime Environment:
 OS Name:     ubuntu
 OS Version:  16.04
 OS Platform: Linux
 RID:         ubuntu.16.04-x64
 Base Path:   /usr/share/dotnet/sdk/1.0.1

The command I used after following your article @ http://www.alteridem.net/2017/05/04/test-net-core-nunit-vs2017/ was dotnet test ./src/Castle.Core.Tests/Castle.Core.Tests-VS2017.csproj.

If you need any more info happy to help.

[ghost]

Also, here is a reference to my csproj file: https://github.com/fir3pho3nixx/Core/blob/refactor-build-travisci-nunit-adapter/src/Castle.Core.Tests/Castle.Core.Tests-VS2017.csproj#L43

[rprouse]

Thanks for the report. Odd that nothing in that callstack is part of the adapter, in fact it looks like the dotnet test process spawning the vstest process. I am going to mark this as confirm for now until we track down where the bug is.

[ghost]

Just pinging back to find out how this is going?

[rprouse]

Sorry, we haven't looked at this yet, I have been busy with an NUnit Framework release. Can you test a simple NUnit and xUnit test project the same way on Linux? I would like to confirm that the bug is with our adapter and not VSTest which it looks like from your Callstack.

[ghost]

Let me check this out and get back to you.

[ghost]

@rprouse - Just an update, we have been chasing a castle core/windsor release to get some bug fixes out after moving our builds to appveyor/travisci. Looks like I might have bandwidth to start looking at this again in the coming week.

[rprouse]

Thanks for the update @Fir3pho3nixx. Any help you can provide would be appreciated.

[ghost]

@rprouse - I have some good news. I have a multi-targeting solution compiling on both windows and linux using both nunit and xunit. Turns out this is kicking out the same error for both testing frameworks using the DesktopCLR. NUnit's error reporting is slightly better it gives a stacktrace. XUnit does not.

The current gold standard I could come up with for multi-targeting on both windows and linux with tests for doing this can be found here: https://github.com/fir3pho3nixx/nunit-adapter-vs-xunit

Here is the output from linux:

Test run for /home/gavin/code/nunit-adapter-vs-xunit/nunit-target/bin/Debug/net462/nunit-target.dll(.NETFramework,Version=v4.6.2)
Microsoft (R) Test Execution Command Line Tool Version 15.0.0.0
Copyright (c) Microsoft Corporation.  All rights reserved.

Starting test execution, please wait...

Unhandled Exception: System.ComponentModel.Win32Exception: Permission denied
   at Interop.Sys.ForkAndExecProcess(String filename, String[] argv, String[] envp, String cwd, Boolean redirectStdin, Boolean redirectStdout, Boolean redirectStderr, Int32& lpChildPid, Int32& stdinFd, Int32& stdoutFd, Int32& stderrFd)
   at System.Diagnostics.Process.StartCore(ProcessStartInfo startInfo)
   at System.Diagnostics.Process.Start()
   at Microsoft.VisualStudio.TestPlatform.CrossPlatEngine.Helpers.ProcessHelper.LaunchProcess(String processPath, String arguments, String workingDirectory, Action`1 exitCallback)
   at Microsoft.VisualStudio.TestPlatform.CrossPlatEngine.Hosting.DefaultTestHostManager.LaunchTestHost(TestProcessStartInfo testHostStartInfo)
   at Microsoft.VisualStudio.TestPlatform.CrossPlatEngine.Client.ProxyOperationManager.SetupChannel(IEnumerable`1 sources)
   at Microsoft.VisualStudio.TestPlatform.CrossPlatEngine.Client.ProxyExecutionManager.InitializeExtensions(IEnumerable`1 sources)
   at Microsoft.VisualStudio.TestPlatform.CrossPlatEngine.Client.ProxyExecutionManager.Initialize()
   at Microsoft.VisualStudio.TestPlatform.Client.TestPlatform.CreateTestRunRequest(TestRunCriteria testRunCriteria)
   at Microsoft.VisualStudio.TestPlatform.CommandLine.TestPlatformHelpers.TestRequestManager.RunTests(TestRunCriteria testRunCriteria, ITestRunEventsRegistrar testRunEventsRegistrar)
   at Microsoft.VisualStudio.TestPlatform.CommandLine.TestPlatformHelpers.TestRequestManager.RunTests(TestRunRequestPayload testRunRequestPayload, ITestHostLauncher testHostLauncher, ITestRunEventsRegistrar testRunEventsRegistrar)
   at Microsoft.VisualStudio.TestPlatform.CommandLine.Processors.RunTestsArgumentExecutor.RunTests(IEnumerable`1 sources)
   at Microsoft.VisualStudio.TestPlatform.CommandLine.Processors.RunTestsArgumentExecutor.Execute()
   at Microsoft.VisualStudio.TestPlatform.CommandLine.Executor.ExecuteArgumentProcessor(IArgumentProcessor processor, Int32& exitCode)
   at Microsoft.VisualStudio.TestPlatform.CommandLine.Executor.Execute(String[] args)
   at Microsoft.VisualStudio.TestPlatform.CommandLine.Program.Main(String[] args)


Test run for /home/gavin/code/nunit-adapter-vs-xunit/nunit-target/bin/Debug/netcoreapp1.1/nunit-target.dll(.NETCoreApp,Version=v1.1)
Microsoft (R) Test Execution Command Line Tool Version 15.0.0.0
Copyright (c) Microsoft Corporation.  All rights reserved.

Starting test execution, please wait...
NUnit Adapter 3.8.0.0: Test execution started
Running all tests in /home/gavin/code/nunit-adapter-vs-xunit/nunit-target/bin/Debug/netcoreapp1.1/nunit-target.dll
NUnit3TestExecutor converted 1 of 1 NUnit test cases
NUnit Adapter 3.8.0.0: Test execution complete

Total tests: 1. Passed: 1. Failed: 0. Skipped: 0.
Test Run Successful.
Test execution time: 3.5235 Seconds


Build started, please wait...
Build completed.

Test run for /home/gavin/code/nunit-adapter-vs-xunit/xunit-target/bin/Debug/net462/xunit-target.dll(.NETFramework,Version=v4.6.2)
Microsoft (R) Test Execution Command Line Tool Version 15.0.0.0
Copyright (c) Microsoft Corporation.  All rights reserved.

The path '/home/gavin/code/nunit-adapter-vs-xunit/xunit-target/bin/Debug/net462' specified in the 'TestAdapterPath' does not contain any test adapters, provide a valid path and try again.

Starting test execution, please wait...
Permission denied



Test run for /home/gavin/code/nunit-adapter-vs-xunit/xunit-target/bin/Debug/netcoreapp1.1/xunit-target.dll(.NETCoreApp,Version=v1.1)
Microsoft (R) Test Execution Command Line Tool Version 15.0.0.0
Copyright (c) Microsoft Corporation.  All rights reserved.

Starting test execution, please wait...
[xUnit.net 00:00:00.8625283]   Discovering: xunit-target
[xUnit.net 00:00:01.0791276]   Discovered:  xunit-target
[xUnit.net 00:00:01.1748117]   Starting:    xunit-target
[xUnit.net 00:00:01.7009607]   Finished:    xunit-target

Total tests: 1. Passed: 1. Failed: 0. Skipped: 0.
Test Run Successful.
Test execution time: 4.3841 Seconds

[ghost]

Further down the rabbit hole we go :)

[rprouse]

@Fir3pho3nixx tell me if I am reading this wrong, but it sounds like this is a .NET Core issue since it appears in both NUnit and xUnit. May we close?

[ghost]

You would be right. Scheduled fix for 15.3.