Add ToChatCompletion{Async} methods for combining StreamingChatCompletionUpdates

[stephentoub]

WithMessageAddedAsync enables writing code like:

List<ChatMessage> messages = ...;
await foreach (var update in client.CompleteStreamingAsync(messages).WithMessageAddedAsync(messages))
{
    Console.Write(update);
}

and upon completion of the loop, messages will contain a ChatMessage merged from all of the updates.

ToChatCompletion is what's used to achieve that, but is also exposed on its own so that code which has an enumerable of StreamingChatCompletionUpdates can merge them into a ChatCompletion. This enables a non-streaming implementation to easily be authored in terms of a streaming one.

Microsoft Reviewers: Open in CodeFlow

[eiriktsarpalis]

WithMessageAddedAsync enables writing code like:

I hadn't realized that CompleteStreamingAsync doesn't update the chat history after the IAE has been enumerated. I'm guessing that this is influenced by the deferred execution of the resultant IAE, but it does seem possible we could go the other way and require coalescence to be a part of CompleteStreamingAsync for consistency with CompleteAsync.

[stephentoub]

WithMessageAddedAsync enables writing code like:

I hadn't realized that CompleteStreamingAsync doesn't update the chat history after the IAE has been enumerated. I'm guessing that this is influenced by the deferred execution of the resultant IAE, but it does seem possible we could go the other way and require coalescence to be a part of CompleteStreamingAsync for consistency with CompleteAsync.

Neither of them update the chat history with the resulting message.

[SteveSandersonMS]

The naming of WithMessageAddedAsync feels odd. Phrasing it in the past tense sounds as if the message has already been added in some way, but it adds the message in the future after the streaming completes.

Do any of the following feel right to you?

await foreach (var update in client.CompleteStreamingAsync(messages).AppendMessageTo(messages))
await foreach (var update in client.CompleteStreamingAsync(messages).AppendResultTo(messages))
await foreach (var update in client.CompleteStreamingAsync(messages).AppendResponseTo(messages))
await foreach (var update in client.CompleteStreamingAsync(messages).WithAppendTo(messages))
await foreach (var update in client.CompleteStreamingAsync(messages).ThenAppendTo(messages))

I mildly lean towards AppendResultTo or AppendResponseTo but don't feel strongly.

[eiriktsarpalis]

I tend to associate the Append prefix with methods that return void; With more accurately reflects we're decorating the underlying IAE with behaviour; Then is typically used when appending callbacks in the style of Task.ContinueWith which isn't precisely what this method is doing.

[stephentoub]

The hard part here is the ToChatCompletion. With that, someone can do the equivalent of WithMessageAddedAsync with:

List<StreamingChatCompletionUpdate> updates = [];
await foreach (var update in client.CompleteStreamingAsync(messages))
{
    Console.Write(update);
    updates.Add(update);
}
messages.Add(updates.ToChatCompletion().Message);

Given that all of the raised concerns are about WithMessageAddedAsync, I'll just remove that for now.

[stephentoub]

This is now just:

public static ChatCompletion ToChatCompletion(
    this IEnumerable<StreamingChatCompletionUpdate> updates, bool coalesceContent = true)

[stephentoub]

(though now that I write that out, I'm wondering if I should add one for IAsyncEnumerable as well)

[stephentoub]

Refactored to have ToChatCompletion{Async} and to have them in the Abstractions library. Having them there means a leaf client can choose to use e.g. ToChatCompletionAsync to implement its CompleteAsync method around its CompleteStreamingAsync, if desired.