ComponentNotRegisteredException when executing test cases using NUnit Console 3.16.3 (Release)

[marossi7]

We recently lifted .NET Framework project to .NET 7. All went without major complications, but we encountered an issue when we were trying to execute test cases from a .NET 7 test library using NUnit Console 3.16.3 (Release). The interesting fact is that the tests are running fine when ran from Visual Studio, but as longs as we run them from the console runner the following error is encountered.

1) Error : TestRunnerModule.Integration.Tests.ConnectionTestRunnerIntegrationTests.TestRunnerPortTestUpdate_UpdatesWizardData(READyCollector)
Autofac.Core.Registration.ComponentNotRegisteredException : The requested service 'MediatorModule.IMediator' has not been registered. To avoid this exception, either register a component to provide the service, check for service registration using IsRegistered(), or use the ResolveOptional() method to resolve an optional dependency.

See https://autofac.rtfd.io/help/service-not-registered for more info.
   at Autofac.ResolutionExtensions.ResolveService(IComponentContext context, Service service, IEnumerable`1 parameters)
   at Autofac.ResolutionExtensions.Resolve(IComponentContext context, Type serviceType, IEnumerable`1 parameters)
   at Autofac.ResolutionExtensions.Resolve[TService](IComponentContext context, IEnumerable`1 parameters)
   at Autofac.ResolutionExtensions.Resolve[TService](IComponentContext context)
   at TestRunnerModule.Integration.Tests.ConnectionTestRunnerIntegrationTests.TestRunnerPortTestUpdate_UpdatesWizardData(DeviceType deviceType) in C:\Users\MHR\source\concentratortool\Tests\DotNet\IntegrationTests\TestRunnerModule.Integration.Tests\ConnectionTestRunnerIntegrationTests.cs:line 241
   at System.RuntimeMethodHandle.InvokeMethod(Object target, Void** arguments, Signature sig, Boolean isConstructor)
   at System.Reflection.MethodInvoker.Invoke(Object obj, IntPtr* args, BindingFlags invokeAttr)

In the test case we use IoC (Autofac 7.0.1) and we are loading assembly modules using Module Scanning. Module scanning is performed with the RegisterAssemblyModules() registration method, which does exactly what its name suggests.

var path = Path.GetDirectoryName(Assembly.GetExecutingAssembly().Location);
List<Assembly> allAssemblies = new List<Assembly>();
foreach (string dll in Directory.GetFiles(path, "*MediatorModule.dll"))
{
    allAssemblies.Add(Assembly.LoadFrom(dll));
}
foreach (string dll in Directory.GetFiles(path, "*WizardDataModule.dll"))
{
    allAssemblies.Add(Assembly.LoadFrom(dll));
}

allAssemblies.Add(Assembly.GetExecutingAssembly());

var builder = new ContainerBuilder();
builder.RegisterAssemblyModules(allAssemblies.ToArray());
Container = builder.Build();

When trying to resolve registered assembly / ies (see below) the exception mentioned in the beginning is thrown.

using (var scope = Container.BeginLifetimeScope())
{
    var mediator = scope.Resolve<IMediator>();
    var wizardDataModule = scope.Resolve<IWizardDataModule>();
...

I am not sure this is the correct place to report this, but the whole thing works fine in the Visual Studio environment, but not through the NUnit Console 3.16.3 (Release).

[OsirisTerje]

Can you please check the 3.17 version.

The 3.16 series are abandoned, and 3.17 is a continuation of 3.15, which is what is used in the adapter, and thus dotnet and Visual Studio.

[marossi7]

Updated to 3.17. It provides me with the same results.

[OsirisTerje]

Ok, that means the error is in the console code, not in the engine code.

Or, that you need to add something to the Autofac registration, that is, scanning is not enough.
Perhaps also raise a question with the Autofac guys.

Suggest you also upload a small repro that reproduce this error.

And, do you really need to use the console, can you use dotnet test instead ?

[marossi7]

Raised the question with the Autofac guys, but they dismissed it saying that it is most likely a problem with environment, which is quite fair argument I guess.

Our existing pipeline has been using console runner and I was just hoping it would still work onwards. However, thanks for the tip - I just tested it with dotnet test and it works like charm. We will switch to this strategy instead.

Do you want me to set up small repo to preproduce this issue that could help you trace what could be wrong with it ?

[OsirisTerje]

Good to hear that dotnet test works for you.

Do you want me to set up small repo to preproduce this issue that could help you trace what could be wrong with it ?

It would be awesome, then we having something to check when we get to this.

[CharliePoole]

@marossi7 Any more info on this? If not I will close it.

BTW, @OsirisTerje commented that the problem is probably not with the engine, because the vs adapter uses the engine. That's not 100% certain as the adapter uses only a subset of the engine's features. In particular, the adapter doesn't create processes, since VS has already created the process used by the tests.

[rodrigo455]

I see the same issue with version 3.18.1.

dotnet test works with autofac, but nunit console tool does not. I am using Autofac 8.0.0

[CharliePoole]

Do you reference the VS adapter? If so, can you run a test removing that reference? That will help to verify that no engine version conflict is occurring.

[rodrigo455]

I have the csproj with the following

  <ItemGroup>
    <PackageReference Include="Microsoft.NET.Test.Sdk" Version="17.10.0" />
    <PackageReference Include="NUnit" Version="4.1.0" />
    <PackageReference Include="NUnit3TestAdapter" Version="4.5.0" />
    <PackageReference Include="NUnit.Analyzers" Version="4.2.0" />
    <PackageReference Include="coverlet.collector" Version="6.0.2" />
  </ItemGroup>

[CharliePoole]

@rodrigo455 Eliminate NUnit3TestAdapter and try a test run. Ideally, your test project should only have the NUnit reference, so you may consider eliminating all other references except for NUnit temporarily.

[CharliePoole]

Also... where are you invoking code that registers assemblies? Is it located in the SUT or in the test assembly? Are you certain that the registration code is actually invoked when running under NUnit3-Console?

[rodrigo455]

with just <PackageReference Include="NUnit" Version="4.1.0" />, the project build fine, but it fails to run:

NUnit Console 3.17.0+685c5b542b5e9ba632c905f0bd514a773d9758af (Release)
Copyright (c) 2022 Charlie Poole, Rob Prouse
Monday, July 22, 2024 8:02:43 AM

Runtime Environment
   OS Version: Ubuntu 20.04.6 LTS
  Runtime: .NET 8.0.5

Test Files
    output/Debug/test/nunit/bin/net8.0/IntegrationTests.dll

NUnit.Engine.NUnitEngineException : Failed to load the NUnit Framework in the test assembly
  ----> System.IO.FileNotFoundException : Could not load file or assembly 'nunit.framework, Version=4.0.1.0, Culture=neutral, PublicKeyToken=2638cd05610744eb'. The system cannot find the file specified.


--NUnitEngineException
Failed to load the NUnit Framework in the test assembly
   at NUnit.Engine.Drivers.NUnitNetCore31Driver.Load(String assemblyPath, IDictionary`2 settings)
   at NUnit.Engine.Runners.DirectTestRunner.LoadDriver(IFrameworkDriver driver, String testFile, TestPackage subPackage)
   at NUnit.Engine.Runners.DirectTestRunner.LoadPackage()
   at NUnit.Engine.Runners.DirectTestRunner.EnsurePackageIsLoaded()
   at NUnit.Engine.Runners.DirectTestRunner.RunTests(ITestEventListener listener, TestFilter filter)
   at NUnit.Engine.Runners.AbstractTestRunner.Run(ITestEventListener listener, TestFilter filter)
   at NUnit.Engine.Runners.MasterTestRunner.RunTests(ITestEventListener listener, TestFilter filter)
   at NUnit.Engine.Runners.MasterTestRunner.Run(ITestEventListener listener, TestFilter filter)
   at NUnit.ConsoleRunner.ConsoleRunner.RunTests(TestPackage package, TestFilter filter)
--
FileNotFoundException
Could not load file or assembly 'nunit.framework, Version=4.0.1.0, Culture=neutral, PublicKeyToken=2638cd05610744eb'. The system cannot find the file specified.

   at System.Reflection.RuntimeAssembly.InternalLoad(AssemblyName assemblyName, StackCrawlMark& stackMark, AssemblyLoadContext assemblyLoadContext, RuntimeAssembly requestingAssembly, Boolean throwOnFileNotFound)
   at System.Runtime.Loader.AssemblyLoadContext.LoadFromAssemblyName(AssemblyName assemblyName)
   at NUnit.Engine.Drivers.NUnitNetCore31Driver.Load(String assemblyPath, IDictionary`2 settings)

where are you invoking code that registers assemblies?

for my case, It is happening during the OneTimeSetUp function for now, when a class that is being used is statically initialized.

[rodrigo455]

The problem for me is that AppDomain.CurrentDomain.BaseDirectory points to the project binary folder with dotnet test,
but that is not the case when running from nunit console tool (where it points to the tool installation directory).

I can get the registering code to run from Assembly.GetExecutingAssembly().Location, with this the dlls are loaded ok,
but It fails to instantiate a type with:

var type = assem.GetType(typeName, true);
Activator.CreateInstance(type);

because it can't find other dependent libraries, although they are loaded to the current AppDomain and available at project binary folder.

I get FileNotFoundException from Assembly.GetType, funny thing, is that the missing dll is the one where the above code is executed.

as you might notice, it is not an autofac problem..., but a misconfigured environment. not sure how to solve that yet.
The AssemblyLoadContext.Default is not able to find dlls in the test project binary directory.

related to #1313

[CharliePoole]

You mentioned #1313 as related. We closed #1313 without action for reasons given here: #1313 (comment)

I can look into this issue further but a simple repro example would be of great help.

[rodrigo455]

My issue is that a dependency of one of the dlls loaded with autofac, use a custom AssemblyLoadContext to find other dependencies... and those ones could not be found in the Default AssemblyLoadContext, because it searches into the nunit console context (base directory)...

Now, I have a workaround that it is working for me, I had to add a Resolving event, so that custom context looks for dlls into the test project binary directory, if it does not find the Default one.

[CharliePoole]

So, would you consider this resolved or should we keep it?

IMO, if the assembly that uses the custom AssemblyLoadContext , is your assembly_ then it's not a bug. The workaround is the solution. But if it's autofac that uses the custom context, then it's a bug... maybe ours, maybe autofac's. :-)

In any case, I'm happy to play with it some more if you can give me a very simple repro project to work with.

[rodrigo455]

Is it fair to assume that the default AssemblyLoadContext should be able to find assemblies where the test libraries are located?
I would think so... if not me, someone else might encounter the same issue.

anyway, I am fine if you consider this resolved.

[CharliePoole]

@rodrigo455 I recognize it's a real problem for you but I have to triage issues to choose those which (1) impact the most people and (2) give me enough information to work on this. The latter is why I asked for a repro project.

@rodrigo455 @marossi7 If either of you can provide a repro project, I'll debug through it using the latest main and see what can be done. Lacking that, it will require somebody familiar with autofac to step up and volunteer to do a PR.

[JulianJmu]

Hi, stumbled across this bug and created a simple repro
NunitApplicationBaseTest.zip

[CharliePoole]

Thanks @JulianJmu

[CharliePoole]

I think this may be related to #1488, which deals with incorrect AppContext.BaseDirectory. Moving to the 3.18.3 milestone.

[CharliePoole]

@marossi7 @JulianJmu @rodrigo455

The dev builds in our myget feed now provide the correct AppContext.BaseDirectory (version 3.8.3-dev00009 or later). It would be great if one of you could try it with AutoFac and let me know so I can include this issue in 3.18.3.