DI with IChatCompletionService with multiple providers

[saroshwadia]

I have a use case where the user would like to choose either OpenAI or AzureOpenAI or Ollama and then Chat with that ChatCompletionService.

In Program.cs I have
builder.Services.AddAzureOpenAIChatCompletion(deploymentName: builder.Configuration["AzureOpenAI:DeploymentName"]!, endpoint: builder.Configuration["AzureOpenAI:Endpoint"]!, apiKey: builder.Configuration["AzureOpenAI:ApiKey"]!); builder.Services.AddOpenAIChatCompletion(modelId: builder.Configuration["OpenAI:ChatModel"]!, apiKey: builder.Configuration["OpenAI:ApiKey"]!); builder.Services.AddOllamaChatCompletion(modelId: builder.Configuration["OllamaAI:ChatModel"]!, endpoint: new Uri(builder.Configuration["OllamaAI:Endpoint"]!));

now using DI "IChatCompletionService chatCompletionService" how can I get the correct chatCompletionService based on what the user has selected (AzureOpenAI or OpenAI or Ollama)?

Thanks

[evchaki]

@saroshwadia - take a look at this notebook - https://github.com/microsoft/semantic-kernel/blob/main/dotnet/notebooks/00-getting-started.ipynb you can see the multiple services and you could have the user provide input on the one they want to use and then enable the service based on that input

[saroshwadia]

Will give it a try. Thx

[RogerBarreto]

We also have a Demonstration on how to use and route between different Connectors and Models with Semantic Kernel.

1. https://github.com/microsoft/semantic-kernel/blob/main/dotnet/samples/Demos/AIModelRouter/Program.cs

2. Another approach is using the ModelId that matches the ModelName in your added connector in the PromptExecutionSettings passed to the kernel.InvokeAsync methods.

3. A third option can be thru filters, changing the modelId based on any information in the FilterContext.
   https://github.com/microsoft/semantic-kernel/blob/main/dotnet/samples/Concepts/Filtering/RetryWithFilters.cs

[saroshwadia]

Will give it a try. Thx

[saroshwadia]

Finally used (DI) keyed services and did the following: (serviceId:)

builder.Services.AddAzureOpenAIChatCompletion(
    deploymentName: builder.Configuration["AzureOpenAI:DeploymentName"]!, 
    endpoint: builder.Configuration["AzureOpenAI:Endpoint"]!,  
    apiKey: builder.Configuration["AzureOpenAI:ApiKey"]!, 
    serviceId: "Azure");
    
builder.Services.AddOpenAIChatCompletion(
    modelId: builder.Configuration["OpenAI:ChatModel"]!, 
    apiKey: builder.Configuration["OpenAI:ApiKey"]!, 
    serviceId: "OpenAI");
    
builder.Services.AddOllamaChatCompletion(
    modelId: builder.Configuration["OllamaAI:ChatModel"]!, 
    endpoint: new Uri(builder.Configuration["OllamaAI:Endpoint"]!), 
    serviceId: "Ollama");

// Then injected IServiceProvider serviceProvider 

private string selectedService = "OpenAI";
private IChatCompletionService? chatCompletionService { get; set; } 

// and used 

var chatCompletionService = serviceProvider.GetRequiredKeyedService<IChatCompletionService>(selectedService);
var response = chatCompletionService.GetStreamingChatMessageContentsAsync(chatHistory: history);

Please let me know if there is a better/cleaner/faster way to do this.

[RogerBarreto]

That's correct, is a good approach if you want use your own IServiceCollection.

One way to make it even less verbose would be to use FluentApi approach, and don't necessarily use the KernelBuilder, just the IServiceCollection directly.

i.e:

services
    .AddAzureOpenAIChatCompletion(...)
    .AddOpenAIChatCompletion(...)
    .AddOllamaChatCompletion(...)