Add IHttpModule and pipeline infrastructure to emulate it

[twsouthwick]

This change adds support for HttpApplication and IHttpModule and is emulated as best as possible on the ASP.NET Core pipeline. This is not tied to IIS and will work on Kestrel or any other host by using middleware to invoke the expected events at the times that best approximate the timing from ASP.NET Core. An attempt has been made to get the events to fire at the appropriate time, but because of the substantial difference between ASP.NET and ASP.NET Core there may still be unexpected behavior.

In order to register either an HttpApplication or IHttpModule instance, use the following pattern:

using System.Web;
using ModulesLibrary;

var builder = WebApplication.CreateBuilder(args);

builder.Services.AddSystemWebAdapters()
    // Non-generic version available if no custom HttpApplication is needed
    .AddHttpApplication<MyApp>(options =>
    {
        // Size of pool for HttpApplication instances. Should be what the expected concurrent requests will be
        options.PoolSize = 10;
        // Register a module by name
        options.RegisterModule<MyModule>("Module");
    });

var app = builder.Build();

app.UseSystemWebAdapters();

app.Run();

class MyApp : HttpApplication
{
  protected void Application_Start()
  {
    ...
  }
  protected void Session_Start()
  {
    ...
  }
  protected void Begin_Request()
  {
    ...
  }
  ...
}

class MyModule : IHttpModule
{
  public void Init(HttpApplication app)
  {
    ...
  }
  
  public void Dispose()
  {
  }
}

The normal .UseSystemWebAdapters() middleware builder will enable majority of the events. However, the authentication and authorization events require two additional middleware calls in order to enable them if you want the events to fire in the expected order. If they are omitted, they will be called at the point UseSystemWebAdapters() is added.

app.UseRouting();
app.UseAuthentication();
+ app.UseRaiseAuthenticationEvents();
app.UseAuthorization();
+ app.UseRaiseAuthorizationEvents();
app.UseSystemWebAdapters();

When should this be used?

Most of the time, this should not be used. Prefer direct ASP.NET Core middleware if possible.

This is intended mostly for scenarios where a module needs to be run on ASP.NET Core but is unable to be migrated easily. Ideally, the code in a module should be restructured to be used as middleware. This is especially recommended when only a single or few events are used; those can usually be migrated in a straightfoward way.

However, if a module has many thousands of line of code and many events being used (the initial driver of this feature), this can provide a stepping stone to migrating that functionality to ASP.NET Core.

Emulated Events

For details on how this worked in .NET Framework, see the official documentation

The IIS event pipeline that is expected by IHttpModule and HttpApplication is emulated using middleware by the adapters. As part of this, it will add additional middleware that will invoke the events. This is done via a feature that is inserted early on in the adapter pipeline. This exposes the HttpApplication for the request, as well as the ability to raise events on it.

Events have a prescribed order which is replicated with these emulated events. However, because the rest of the ASP.NET Core pipeline is unaware of these events and so some of the state of the request may not be exactly replicated.

A common pattern is to be able to call HttpRequest.End() or HttpApplication.CompleteRequest(). Both of these are supported, as well as continuing to raise the events that are raised in IIS with this (including EndRequest and the logging events).

Note: In the cases in which no modules or HttpApplication type is registered, the emulated pipeline is not added to the middleware chain.

HttpApplication lifetime

On ASP.NET Framework, each request would get an individual HttpApplication instance. This object contains the following information:

• Event callbacks registered either on the HttpApplication type itself or on registered modules
• Any state contained in the HttpApplication instance or its registered modules

In order to support this, one of the first middlewares invoked will retrieve an instance of HttpApplication. This uses a PooledObjectPolicy<HttpApplication> that will create an instance of the application's HttpApplication type and register all modules on it. When the request is exiting that middleware, it will return the HttpApplication instance to the pool which will also remove the HttpContext instance assigned to it.

This can potentially create a number of instances of HttpApplication that are only used a limited number of times. The pool can be controlled by customizing the HttpApplicationOptions.PoolSize option or providing a custom implementation of ObjectPool<HttpApplication> that can use the PooledObjectPolicy<HttpApplication> provided to override the pooling behavior.

[twsouthwick]

@davidfowl would you prefer to start without pooling? Any other concerns to get this in?

[CZEMacLeod]

@twsouthwick, @davidfowl

To my mind, System.Web does this with pooling, and we now have excellent pooling mechanisms. As long as there are suitable tests for this, I don't see an issue.
I feel like the overheads, if you have a lot of modules/events and/or a complex Application class defined, are going to make this very slow without it.

If it does need to be disabled (at least by default, hopefully with an opt-in mechanism) could we do some kind of factory, and be able to register the PoolingApplicationFactory or a SimpleApplicationFactory. This would feel much better to me that just deleting the code outright.

It could allow other (customer supplied) implementations too by inheriting a base class, perhaps to spin up x instances at startup and anything that goes through System.Web is throttled to that many concurrent requests.

I don't want to make more work for you, and I want to see this next version get out (so we can get even more cool things added in), but if you are going to simplify this for the sake of maintainability/testability or otherwise, please make it a mechanism that can be overridden with a better implementation "at the consumer's risk".

[davidfowl]

What's the pool size on System.Web?

[twsouthwick]

There doesn't appear to be a specified number. They create on demand, then this method is called when there's idle time to trim 3% of the pool on each call

[twsouthwick]

This kind of dynamic pool size doesn't seem easy to implement on top of ObjectPool<> as-is, although I did play around with a custom implementation that would use ML.NET and its forecasting APIs to dynamically adapt the target pool size based on the historical concurrent request count :)

[twsouthwick]

@davidfowl anything else?