Richer Extensibility for routes and clusters

[samsp-msft]

Problem Statement

Today if you want to extend the route or clusters, you can only do it through the metadata property on each, which is a Dictionary<string, string>. If you want to be able to have structured data its not possible without you forcing it into a string and then parsing it when needed. There are scenarios like A/B testing, or authenticating with back end servers (not pass thru) where you want to be able to store a structure of data in config, and have it available at runtime on the route/cluster objects.

Design for this feature has been moved to #1796

[samsp-msft]

Two things we should consider from #87:

• What happens to state objects that are not directly derived from config. For example load balancing counters. You wouldn't want them to be reset any time configuration changes. So there probably needs to be a way for them to persist, and have a notification that either they have been orphaned as config has changed, or automatically move them to the new state objects, and have notification that they may want to pickup the new configuration, along with a pointer to the config data.
• Thread safety is a concern for these objects. Its probably mostly a documentation problem rather than anything we can directly fix. However the collection that contains them needs to be thread safe.

[karelz]

Triage: Pretty open ended. We need agreement first on the design.

[ayrloong]

@karelz Hi I have seen this design https://github.com/microsoft/reverse-proxy/blob/main/docs/designs/route-extensibility.md
I think this function is very meaningful. I have implemented this part of the function. Can I submit a PR?

[karelz]

@ayrloong I don't think the linked doc covers this issue, does it?

[ayrloong]

@karelz Thanks for your reply Here is my initial implementation. when getting Extensions, I can't link like above. I need to use As

{
    "ReverseProxy": {
        "Routes": {
            "route1": {
                "ClusterId": "cluster1",
                "Match": {
                    "Path": "{**catch-all}"
                },
                "Extensions": {
                    "UserModel": {
                        "Name": "lq1"
                    },
                    "ABTest": {
                        "CE": 3,
                        "Names": [
                            "1",
                            "2",
                            "3"
                        ]
                    }
                }
            }
        },
        "Clusters": {
            "cluster1": {
                "Destinations": {
                    "destination1": {
                        "Address": "https://example.com/"
                    }
                }
            }
        }
    }
}

Startup.cs

builder.Services.AddReverseProxy()
    .LoadFromConfig(builder.Configuration.GetSection("ReverseProxy"), [typeof(UserModel), typeof(ABTest)]);

app.UseEndpoints(endpoints =>
{
    // We can customize the proxy pipeline and add/remove/replace steps
    endpoints.MapReverseProxy(proxyPipeline =>
    {
        // Use a custom proxy middleware, defined below
        proxyPipeline.Use((context, next) =>
        {
            var proxyFeature = context.Features.Get<IReverseProxyFeature>();
            var user = proxyFeature.Route.Config.Extensions[typeof(UserModel)] as UserModel;
            
            var abTest = proxyFeature.Route.Config.Extensions[typeof(ABTest)] as ABTest;

            Console.WriteLine(abTest.CE);
            return next();
        });
        proxyPipeline.UseSessionAffinity();
        proxyPipeline.UseLoadBalancing();
    });
});

ReverseProxyServiceCollectionExtensions.cs

    public static IReverseProxyBuilder LoadFromConfig(this IReverseProxyBuilder builder, IConfiguration config,
        Type[] extensionsTypes)
    {
        if (config is null)
        {
            throw new ArgumentNullException(nameof(config));
        }

        builder.Services.AddSingleton<IProxyConfigProvider>(sp =>
        {
            // This is required because we're capturing the configuration via a closure
            return new ConfigurationConfigProvider(sp.GetRequiredService<ILogger<ConfigurationConfigProvider>>(),
                config, extensionsTypes);
        });

        return builder;
    }

RouteModel.cs

  public IReadOnlyDictionary<Type, object>? Extensions { get; init; }

ConfigurationConfigProvider.cs

private static IReadOnlyDictionary<Type, object>? CreateExtensions(IConfigurationSection section)
    {
        if (section.GetChildren() is var children && !children.Any())
        {
            return null;
        }


        var mutable = new Dictionary<Type, object>();
        
        foreach (var extensions in extensionsTypes)
        {
            var instance = section.GetSection(extensions.Name).Get(extensions);
            if (instance != null)
            {
                mutable[extensions] = instance;
            }
        }

        return new ReadOnlyDictionary<Type, object>(mutable);
    }

[ayrloong]

Hello, I have spent some time working on this and implemented this part, submitting a draft PR for review.