ConfigureTestContainer not working with GenericHost

[alefranz]

Describe the bug

When using GenericHost, in tests ConfigureTestContainer is not executed.

To Reproduce

I've added a test in alefranz@282c153#diff-589b0cebe9e796b47e521f0318393df2R194-R208 to reproduce

Expected behavior

The delegate provided with ConfigureTestContainer should be executed

Additional context

This happen in 3.x

I'm looking at providing a fix for this but I would probably need some hint on the right place to address this.
Could you also confirm the behavior is not intentional?

@Tratcher I've seen you have worked to add support to ConfigureTestServices with GenericHost in https://github.com/aspnet/AspNetCore/pull/6585/files#diff-48af505b9d348e7e52da534c4590aef1R36 - Do you think it would need a similar logic to address this as well?

Thank you,
Alessio

[Tratcher]

There were reasons... I don't think ConfigureTestContainer was compatible. @javiercn might remember.

[javiercn]

@alefranz thanks for contacting us.

In previous versions the reason we had ConfigureTest(Services|Container) was due to the fact that the Configure(Services|Container) in Startup ran after all the ConfigureTest(Services|Container) methods applied to the host builder and we needed a way to run a Configure(Services|Container) callback after that happened to ensure services could be overriden for testing purposes.

When we moved to generic host, we changed that behavior and now the callbacks are queued in the order in which you register them, so when you do UseStartup the callback gets immediately queued. For that reason, if you are using generic host you shouldn't need the ConfigureTestContainer overload, you can simply call hostBuilder.ConfigureContainer after the call to UseStartup to tweak the container configuration.

I don't know what work @Tratcher did to make it work for ConfigureTestServices. I think the main reason we didn't do it is because we might have thought it was a much less common use case.

If ConfigureTestServices is supported on generic host, @Tratcher is ok with it, and it's not a big change, I'm supportive of a PR to make ConfigureTestContainer work too, if you want to give it a try.

I don't think we will fix ourselves as it's not a common scenario and there is an alternative way to achieve the same thing as described above, so this only helps you in migration scenarios.

Otherwise, I simply recommend closing the issue.

Hope this helps.

[alefranz]

I will investigate the scope of a change to make it work.
If this is too big of a change, what do you think about throwing an exception that clearly indicates that it is not supported in this scenario and what to use instead?
With the current behavior there is a risk to spend time in trying to understand why you don't have the expected services in the container (talking from experience 😅).

[roend83]

@javiercn I'm having a similar issue and have tried to make the updates that you suggested above. However, I'm having an issue because my Startup.ConfigureContainer method is always being called after my call to IHostBuilder.ConfigureContainer. It doesn't seem to matter what order I set them up with.

[roend83]

@javiercn I did confirm that if I remove Startup.ConfigureContainer and instead put that code into a call to IHostBuilder.ConfigureContainer then it does appear to all get executed in the right order.

[javiercn]

@roend83 It's impossible for us to diagnose what's going on in your case without a minimal repro project. How DI works with the new container is explained a bit better here:
https://github.com/aspnet/AspNetCore.Docs/issues/14973

[roend83]

AspNetCoreConfigurationIssue.zip
@javiercn Here's a minimal repro of what I'm seeing.

[javiercn]

@roend83 Thanks for the repro.

@Tratcher I took a quick look at the repro and @roend83 seems legit. Can you take a look?

        public static IHostBuilder CreateHostBuilder(string[] args) =>
            Host.CreateDefaultBuilder(args)
                .UseServiceProviderFactory(new AutofacServiceProviderFactory())
                .ConfigureWebHostDefaults(webBuilder => { webBuilder.UseStartup<Startup>(); })
                .ConfigureContainer<ContainerBuilder>(builder =>
                {
                    if (!Startup.IsContainerConfigured)
                    {
                        throw new Exception("Startup.ConfigureContainer hasn't been called yet'");
                    }
                });

My understanding is that that should never throw as Startup.ConfigureContainer should run before the ConfigureContainer after it, isn't that the case?

[Tratcher]

This gets hit first:

>	AspNetCoreConfigurationIssue.dll!AspNetCoreConfigurationIssue.Program.CreateHostBuilder.AnonymousMethod__1_1(Autofac.ContainerBuilder builder) Line 27	C#
 	Microsoft.Extensions.Hosting.dll!Microsoft.Extensions.Hosting.HostingHostBuilderExtensions.ConfigureContainer.AnonymousMethod__0(Microsoft.Extensions.Hosting.HostBuilderContext context, System.__Canon builder)	Unknown
 	Microsoft.Extensions.Hosting.dll!Microsoft.Extensions.Hosting.Internal.ConfigureContainerAdapter<Autofac.ContainerBuilder>.ConfigureContainer(Microsoft.Extensions.Hosting.HostBuilderContext hostContext, object containerBuilder)	Unknown
 	Microsoft.Extensions.Hosting.dll!Microsoft.Extensions.Hosting.HostBuilder.CreateServiceProvider()	Unknown
 	Microsoft.Extensions.Hosting.dll!Microsoft.Extensions.Hosting.HostBuilder.Build()	Unknown
 	AspNetCoreConfigurationIssue.dll!AspNetCoreConfigurationIssue.Program.Main(string[] args) Line 18	C#

And this second:

>	AspNetCoreConfigurationIssue.dll!AspNetCoreConfigurationIssue.Startup.ConfigureContainer(Autofac.ContainerBuilder builder) Line 56	C#
 	[Native to Managed Transition]	
 	[Managed to Native Transition]	
 	Microsoft.AspNetCore.Hosting.dll!Microsoft.AspNetCore.Hosting.ConfigureContainerBuilder.InvokeCore(object instance, object container)	Unknown
 	Microsoft.AspNetCore.Hosting.dll!Microsoft.AspNetCore.Hosting.ConfigureContainerBuilder.Invoke.__StartupConfigureContainer|0(object containerBuilder)	Unknown
 	Microsoft.AspNetCore.Hosting.dll!Microsoft.AspNetCore.Hosting.ConfigureContainerBuilder.Invoke(object instance, object container)	Unknown
 	Microsoft.AspNetCore.Hosting.dll!Microsoft.AspNetCore.Hosting.ConfigureContainerBuilder.Build.AnonymousMethod__0(object container)	Unknown
 	Microsoft.AspNetCore.Hosting.dll!Microsoft.AspNetCore.Hosting.GenericWebHostBuilder.ConfigureContainer<System.__Canon>(Microsoft.Extensions.Hosting.HostBuilderContext context, System.__Canon container)	Unknown
 	Microsoft.Extensions.Hosting.dll!Microsoft.Extensions.Hosting.Internal.ConfigureContainerAdapter<Autofac.ContainerBuilder>.ConfigureContainer(Microsoft.Extensions.Hosting.HostBuilderContext hostContext, object containerBuilder)	Unknown
 	Microsoft.Extensions.Hosting.dll!Microsoft.Extensions.Hosting.HostBuilder.CreateServiceProvider()	Unknown
 	Microsoft.Extensions.Hosting.dll!Microsoft.Extensions.Hosting.HostBuilder.Build()	Unknown
 	AspNetCoreConfigurationIssue.dll!AspNetCoreConfigurationIssue.Program.Main(string[] args) Line 18	C#

ConfigureContainer actions are called in the order registered:
https://github.com/aspnet/Extensions/blob/b3a158ac055f1f8f76ed4a1a1c6ef860068cf63e/src/Hosting/Hosting/src/HostBuilder.cs#L228-L231
This is where generic web host finds and calls ConfigureContainer:
https://github.com/aspnet/AspNetCore/blob/c7972d45f83aab461d23616a9c17fe5fe3ad2226/src/Hosting/Hosting/src/GenericHost/GenericWebHostBuilder.cs#L260-L263
Which is called inline...

>	AspNetCoreConfigurationIssue.dll!AspNetCoreConfigurationIssue.Program.CreateHostBuilder.AnonymousMethod__1_0(Microsoft.AspNetCore.Hosting.IWebHostBuilder webBuilder) Line 24	C#
 	Microsoft.AspNetCore.dll!Microsoft.Extensions.Hosting.GenericHostBuilderExtensions.ConfigureWebHostDefaults.AnonymousMethod__0(Microsoft.AspNetCore.Hosting.IWebHostBuilder webHostBuilder)	Unknown
 	Microsoft.AspNetCore.Hosting.dll!Microsoft.Extensions.Hosting.GenericHostWebHostBuilderExtensions.ConfigureWebHost(Microsoft.Extensions.Hosting.IHostBuilder builder, System.Action<Microsoft.AspNetCore.Hosting.IWebHostBuilder> configure)	Unknown
 	Microsoft.AspNetCore.dll!Microsoft.Extensions.Hosting.GenericHostBuilderExtensions.ConfigureWebHostDefaults(Microsoft.Extensions.Hosting.IHostBuilder builder, System.Action<Microsoft.AspNetCore.Hosting.IWebHostBuilder> configure)	Unknown
 	AspNetCoreConfigurationIssue.dll!AspNetCoreConfigurationIssue.Program.CreateHostBuilder(string[] args) Line 22	C#
 	AspNetCoreConfigurationIssue.dll!AspNetCoreConfigurationIssue.Program.Main(string[] args) Line 18	C#

Ah, but UseStartup is delayed until ConfigureServices runs.
https://github.com/aspnet/AspNetCore/blob/c7972d45f83aab461d23616a9c17fe5fe3ad2226/src/Hosting/Hosting/src/GenericHost/GenericWebHostBuilder.cs#L203-L209
That explains the order inversion. Here's how it works out:

• Program: ConfigureWebHostDefaults, UseStartup
• Program: ConfigureContainer (register callback)
• Build, CreateServiceProvider
• ConfigureServices
  • UseStartup registers Startup.ConfigureContainer callback
• ConfigureContainer
  • Program ConfigureContainer callback
  • Startup ConfigureContainer callback

Trying to fix this would likely require moving the delayed call to UseStartup out of ConfigureServices. Not sure what other consequences that would have.

[GODBS]

I have the exact same issue. Is there a workaround that would allow us to bypass this issue?

[cdibbs]

@GODBS I ended up going the super-hacky route, for now. I hope this is fixed in 3.1.

internal static Action<ContainerBuilder> IntegTestOverrides = null;
public void ConfigureContainer(ContainerBuilder cb)
{
    cb.RegisterModule(new ApiRootModule());
    cb.RegisterType<StartupHttpPipeline>().As<IStartupHTTPPipeline>();
    IntegTestOverrides?.Invoke(cb);
}

[Tratcher]

@cdibbs no changes are planned for 3.1.

[cdibbs]

Will a fix come with a later patch to 3.1? Requiring hacks to override dependencies in your tests seems like a pretty undesirable thing to enter LTS with.

[Tratcher]

Unlikely. Patching is risky and anything with a viable workaround is unlikely to be patched.

[seesharper]

Take a look at this article for more information about AspNet Core 3.0 integration tests
https://andrewlock.net/converting-integration-tests-to-net-core-3/

[seesharper]

As @roend83 mentioned it works when adding third party DI registrations at the time the IHostBuilder is bein set up.
Take a look here for an example

https://github.com/seesharper/heatkeeper/blob/master/src/HeatKeeper.Server.Host/Program.cs

And the customization of the container during test is here

https://github.com/seesharper/heatkeeper/blob/master/src/HeatKeeper.Server.WebApi.Tests/TestBase.cs

[dahovey]

The resolution I am using is below.

          public class TestApplicationFactory : WebApplicationFactory<Startup>
        {
            protected override IHost CreateHost(IHostBuilder builder)
            {
                builder.ConfigureContainer<ContainerBuilder>(containerBuilder =>
                    {
           
                    });

                return base.CreateHost(builder);
            }
        }

The ConfigureContainer registered inside WebApplicationFactory is called after ConfigureContainer by the generic host builder inside Program.cs.

[forktrucka]

The resolution I am using is below.

          public class TestApplicationFactory : WebApplicationFactory<Startup>
        {
            protected override IHost CreateHost(IHostBuilder builder)
            {
                builder.ConfigureContainer<ContainerBuilder>(containerBuilder =>
                    {
           
                    });

                return base.CreateHost(builder);
            }
        }

The ConfigureContainer registered inside WebApplicationFactory is called after ConfigureContainer by the generic host builder inside Program.cs.

I tried this. Seemed to face the same problem in my implementation... What am i missing?

Using 3.1, we're using autofac and calling ConfigureContainer(ContainerBuilder builder) from startup. This seems to execute in the same order...

[Pvlerick]

All due respect, I am sorry but the provided answers to this issues are simply not satisfactory.

When we moved to generic host, we changed that behavior and now the callbacks are queued in the order in which you register them, so when you do UseStartup the callback gets immediately queued. For that reason, if you are using generic host you shouldn't need the ConfigureTestContainer overload, you can simply call hostBuilder.ConfigureContainer after the call to UseStartup to tweak the container configuration.

This does not work, as demonstrated in Tratcher's post here: #14907 (comment)

@javiercn 's description in #14907 (comment) implies to be the intended behaviour is that Startup.ConfigureContainer should be called after Program.ConfigureContainer if the later is called after UseStarup. This is not the case which means it is a plain bug.

None of the technique above are working, meaning that there is no apparent nor easy workaround. I'm excluding @cdibbs 's techniques which prevents testing in parallel.

This issue prevents from having a proper way for tests to override dependencies, which is a huge issue for us (and I suspect many other). We have tons of tests doing exactly that and this bug prevents us from moving from 2.1 to 3.1

This is a big show-stopper for anyone who wants to properly test his application.

[alefranz]

I have investigated a fix but unfortunately, as well explained by @Tratcher , to fix this would have potential side effects so I parked my work.
@anurse would a PR be considered or it is to risky to do change this behavior as 3.1 has already been released?
Should the effort instead be in documenting a working hack?
Thank you!

[cdibbs]

I agree with @Pvlerick that my workaround doesn't work when you run the tests in parallel (which is the norm). So, is my workaround really "viable?" Maybe not.

It sounds like any fix would have side effects, but perhaps that just builds a case for doubling up on the normal amount of testing done for a patch like this.

[Pvlerick]

@cdibbs exactly. It's an ugly fix to try to make things work temporarily, this is not a solution that will be implemented across projects; I don't see myself explaining to other developers that this is how they should inject theirs mocks from now on, moving from a super-clean solution in 2.1 to a hack in 3.1.

We need this addressed badly :-(

[ssunkari]

I got it working by implementing custom factory for my container setup. In my case its Autofac. I have tweaked the IServiceProviderFacotry implementation of Autofac library.

Code

public class CustomAutofacServiceProviderFactory : IServiceProviderFactory<ContainerBuilder>
{
    private readonly Action<ContainerBuilder> _configurationOverride;
    private readonly Action<ContainerBuilder> _configurationAction;
    /// <summary>
    /// Initializes a new instance of the <see cref="AutofacServiceProviderFactory"/> class.
    /// </summary>
    /// <param name="testStartup"></param>
    /// <param name="configurationAction">Action on a <see cref="ContainerBuilder"/> that adds component registrations to the conatiner.</param>
    public CustomAutofacServiceProviderFactory(Action<ContainerBuilder> configurationAction = null, Action<ContainerBuilder> configurationOverride = null)
    {
        _configurationOverride = configurationOverride;
        _configurationAction = configurationAction ?? (builder => { });
    }



    /// <summary>
    /// Creates a container builder from an <see cref="IServiceCollection" />.
    /// </summary>
    /// <param name="services">The collection of services.</param>
    /// <returns>A container builder that can be used to create an <see cref="IServiceProvider" />.</returns>
    public ContainerBuilder CreateBuilder(IServiceCollection services)
    {
        var builder = new ContainerBuilder();

        builder.Populate(services);

        _configurationAction(builder);

        return builder;
    }

    /// <summary>
    /// Creates an <see cref="IServiceProvider" /> from the container builder.
    /// </summary>
    /// <param name="containerBuilder">The container builder.</param>
    /// <returns>An <see cref="IServiceProvider" />.</returns>
    public IServiceProvider CreateServiceProvider(ContainerBuilder containerBuilder)
    {
        if (containerBuilder == null) throw new ArgumentNullException(nameof(containerBuilder));

        _configurationOverride(containerBuilder); // Added this to override container overrides
        var container = containerBuilder.Build();

        return new AutofacServiceProvider(container);
    }
}

Test Setup

public class CustomWebApplicationFactory : WebApplicationFactory where
TStartup:class
{

    protected override IHost CreateHost(IHostBuilder builder)
    {
        builder.ConfigureContainer<ContainerBuilder>(cb =>
        {
        }).UseServiceProviderFactory(
            new CustomAutofacServiceProviderFactory(
                b => { },
                OverrideContainer));
 

        return base.CreateHost(builder);
    }

}
Refer to Autofac docs https://autofaccn.readthedocs.io/en/latest/integration/aspnetcore.html for service registration

[Pvlerick]

@ssunkari I haven't tried this yet, but again I don't see this as a viable workaround.

@anurse @davidfowl @DamianEdwards any views on this, do you think this can/will be patched?

[ghost]

Thanks for contacting us.
We're moving this issue to the Next sprint planning milestone for future evaluation / consideration. Because it's not immediately obvious that this is a bug in our framework, we would like to keep this around to collect more feedback, which can later help us determine the impact of it. We will re-evaluate this issue, during our next planning meeting(s).
If we later determine, that the issue has no community involvement, or it's very rare and low-impact issue, we will close it - so that the team can focus on more important and high impact issues.
To learn more about what to expect next and how this issue will be handled you can read more about our triage process here.

[oliverhanappi]

You can override container registrations in the following way, using Autofac as example:

Configure your container in Program instead of Startup.ConfigureContainer(ContainerBuilder):

public static IHostBuilder CreateHostBuilder (string[] args)
{
  return Host.CreateDefaultBuilder (args)
    .UseServiceProviderFactory (new AutofacServiceProviderFactory())
    .ConfigureContainer<ContainerBuilder>(b => { /* configure container here */ })
    .ConfigureWebHostDefaults(builder => builder.UseStartup<Startup>());
}

Override container registrations using ContainerBuilder in a derived WebApplicationFactory:

public class TestWebApplicationFactory : WebApplicationFactory<Startup>
{
  protected override IHost CreateHost(IHostBuilder builder)
  {
    builder.ConfigureContainer<ContainerBuilder>(b => { /* test overrides here */ });
    return base.CreateHost(builder);
  }
}

[satrapu]

You can override container registrations in the following way, using Autofac as example:
...

I would replace overriding TestWebApplicationFactory class with an action for added flexibility, like this:

public class TestWebApplicationFactory : WebApplicationFactory<Startup>
{
    ....
    private Action<ContainerBuilder> setupMockServicesAction;

    public TestWebApplicationFactory WithMockServices(Action<ContainerBuilder> setupMockServices)
    {
        setupMockServicesAction = setupMockServices;
        return this;
    }

    protected override IHost CreateHost(IHostBuilder builder)
    {
        builder.ConfigureContainer<ContainerBuilder>(containerBuilder =>
        {
            setupMockServicesAction?.Invoke(containerBuilder);
        });

        return base.CreateHost(builder);
   }
    ....
}

Then, my test method which resides inside an NUnit test fixture which needs to instantiate TestWebApplicationFactory class looks like this:

[TestFixture]
public class CustomExceptionHandlerTests
{
    [Test]
    public async Task HandleException_WhenApiThrowsException_MustConvertExceptionToInstanceOfProblemDetailsClass()
    {
         // Arrange phase
         ...
         using var testWebApplicationFactory =
                new TestWebApplicationFactory(...)
                    .WithMockServices(containerBuilder =>
                    {
                        // Ensure a mock implementation will be injected whenever a service requires an instance of the
                        // IGenerateJwtFlow interface.
                        containerBuilder
                            .RegisterType<GenerateJwtFlowWhichThrowsException>()
                            .As<IGenerateJwtFlow>()
                            .InstancePerLifetimeScope();
                    });

         using HttpClient httpClient = testWebApplicationFactory.CreateClient();
         ...
    }
}

One can employ WithMockServices only when needing to replace a registered service with something else, e.g. a class which will throw an exception whenever one of its methods are called, like in my above example.

The full source code can be found here .

[jsdevtom]

Hi @jr01. That solution doesn't work for me:

Here is a workaround for Autofac that will call any ConfigureTestContainer(...) method, also in .WithWebHostBuilder(...) calls.

It builds on @ssunkari's workaround

public class CustomWebApplicationFactory<TStartup> : WebApplicationFactory<TStartup>
        where TStartup : class
{
	protected override void ConfigureWebHost(IWebHostBuilder webHostBuilder)
	{
		webHostBuilder.ConfigureTestContainer<Autofac.ContainerBuilder>(builder =>
		{
			// called after Startup.ConfigureContainer
		});
	}

	protected override IHost CreateHost(IHostBuilder builder)
	{
		builder.UseServiceProviderFactory(new CustomServiceProviderFactory());
		return base.CreateHost(builder);
	}
}

public class CustomServiceProviderFactory : IServiceProviderFactory<CustomContainerBuilder>
{
	public CustomContainerBuilder CreateBuilder(IServiceCollection services) => new CustomContainerBuilder(services);

	public IServiceProvider CreateServiceProvider(CustomContainerBuilder containerBuilder) =>
		new AutofacServiceProvider(containerBuilder.CustomBuild());
}

public class CustomContainerBuilder : Autofac.ContainerBuilder
{
	private readonly IServiceCollection services;

	public CustomContainerBuilder(IServiceCollection services)
	{
		this.services = services;
		this.Populate(services);
	}

	public Autofac.IContainer CustomBuild()
	{
		var sp = this.services.BuildServiceProvider();
#pragma warning disable CS0612 // Type or member is obsolete
		var filters = sp.GetRequiredService<IEnumerable<IStartupConfigureContainerFilter<Autofac.ContainerBuilder>>>();
#pragma warning restore CS0612 // Type or member is obsolete

		foreach (var filter in filters)
		{
			filter.ConfigureContainer(b => { })(this);
		}

		return this.Build();
	}
}

Usage

var factory = new CustomWebApplicationFactory<Startup>();

var client1 = factory.CreateClient();


var client2 = factory.WithWebHostBuilder(b =>
{
	b.ConfigureTestContainer<Autofac.ContainerBuilder>(builder =>
	{
		// Called after Startup.ConfigureContainer and after CustomWebApplicationFactory's ConfigureTestContainer
	});
}).CreateClient();

In AutoFac version 6.2.0 I get Cannot inherit from sealed class 'ContainerBuilder'. Do you have any ideas?

UPDATE: Solved through this comment

[ghost]

Thanks for contacting us.

We're moving this issue to the .NET 8 Planning milestone for future evaluation / consideration. We would like to keep this around to collect more feedback, which can help us with prioritizing this work. We will re-evaluate this issue, during our next planning meeting(s).
If we later determine, that the issue has no community involvement, or it's very rare and low-impact issue, we will close it - so that the team can focus on more important and high impact issues.
To learn more about what to expect next and how this issue will be handled you can read more about our triage process here.

[tuskajozsef]

Cleanest solution I could achieve based on previous answers:

Create overrides:

public class TestAutofacModule : Module
{
    protected override void Load(ContainerBuilder builder)
    {
        builder.RegisterType<TestService>().As<IService>();
    }
}

Create new ServiceProviderFactory:

public class TestServiceProviderFactory<T> : IServiceProviderFactory<ContainerBuilder> where T: Module, new()
{
    private readonly AutofacServiceProviderFactory _factory = new();

    public ContainerBuilder CreateBuilder(IServiceCollection services)
    {
        return _factory.CreateBuilder(services);
    }

    public IServiceProvider CreateServiceProvider(ContainerBuilder containerBuilder)
    {
        containerBuilder.RegisterModule(new T());
        return _factory.CreateServiceProvider(containerBuilder);
    }
}

Use new Factory:

var builder = new HostBuilder()
                .UseServiceProviderFactory(new TestServiceProviderFactory<TestAutofacModule>())
                ...

[halter73]

Based on #51381, this unsurprisingly also affects WebApplicationBuilder. We should make sure WAB works too when we fix this.