Investigation: Azure and unbranded clients by annelo-msft · Pull Request #7 · annelo-msft/openai-in-typespec

annelo-msft · 2024-04-22T22:47:43Z

This PR explores letting extending models write their own ToBinaryContent methods

…ctionary

…ions

annelo-msft · 2024-04-23T23:13:13Z

+EndProject
+Project("{9A19103F-16F7-4668-BE54-9A1E7A4F7556}") = "AzureOpenAI", "azoai\AzureOpenAI\AzureOpenAI.csproj", "{FB61EB83-7171-48F4-9423-E64B99BDAA5A}"
+EndProject
+Project("{9A19103F-16F7-4668-BE54-9A1E7A4F7556}") = "System.ClientModel", "..\..\..\..\azure-sdk-for-net\sdk\core\System.ClientModel\src\System.ClientModel.csproj", "{8364F37B-2B3D-4CAA-8F20-5B808E19C6CE}"


Points to the version of ClientModel in this PR: Prototype of JsonModel abstract class

annelo-msft · 2024-04-23T23:13:57Z

+    // TODO: Do something with output
+}
+
+void CallAzureService()


Illustrates application call to Azure OpenAI service.

annelo-msft · 2024-04-23T23:14:10Z

+
+namespace AzureOpenAI;
+
+internal class AzureChatClient : Chat


Sub-client is internal.

annelo-msft · 2024-04-23T23:14:40Z

+        Argument.AssertNotNull(createChatCompletionRequest, nameof(createChatCompletionRequest));
+
+        using BinaryContent content = createChatCompletionRequest.ToBinaryContent();
+        ClientResult result = await CreateChatCompletionAsync(createChatCompletionRequest.Model.ToString(), content, context: default).ConfigureAwait(false);


Overrides convenience method to call Azure-specific protocol method.

annelo-msft · 2024-04-23T23:14:56Z

+
+        using BinaryContent content = createChatCompletionRequest.ToBinaryContent();
+        ClientResult result = await CreateChatCompletionAsync(createChatCompletionRequest.Model.ToString(), content, context: default).ConfigureAwait(false);
+        var value = ModelReaderWriter.Read<CreateChatCompletionResponse>(result.GetRawResponse().Content)!;


Uses ModelReaderWriter to create response model.

annelo-msft · 2024-04-23T23:15:21Z

+        return ClientResult.FromValue(value, result.GetRawResponse());
+    }
+
+    public override Task<ClientResult> CreateChatCompletionAsync(BinaryContent content, RequestOptions context = null)


Overrides non-functional protocol method and throws from it.

annelo-msft · 2024-04-23T23:15:58Z

+        return ClientResult.FromResponse(Pipeline.ProcessMessage(message, context));
+    }
+
+    public async Task<ClientResult> CreateChatCompletionAsync(string model, BinaryContent content, RequestOptions context = null)


Overloads protocol method with Azure-specific protocol signature.

annelo-msft · 2024-04-23T23:16:57Z

+        return ClientResult.FromResponse(await Pipeline.ProcessMessageAsync(message, context).ConfigureAwait(false));
+    }
+
+    private PipelineMessage CreateCreateChatCompletionRequest(string model, BinaryContent content, RequestOptions context)


Implements private request-creation helper that creates request with Azure-specific URI and path parameters. Note here that request content has already been serialized by the model types.

annelo-msft · 2024-04-23T23:17:44Z

+    {
+        Argument.AssertNotNull(createChatCompletionRequest, nameof(createChatCompletionRequest));
+
+        using BinaryContent content = createChatCompletionRequest.ToBinaryContent();


Extension method on third-party type creates BinaryContent instance that writes content via internal Azure model wrapping third-party type.

annelo-msft · 2024-04-23T23:18:15Z

+        _apiKeyCredential = credential;
+    }
+
+    public override Chat GetChatClient()


Overrides subclient factory method to return Azure-specific subclient.

annelo-msft · 2024-04-23T23:18:36Z

+public static class ChatClientExtensions
+{
+    // Expose overloads of protocol methods as extensions to the base client
+    public static ClientResult CreateChatCompletion(this Chat client, string model, BinaryContent content, RequestOptions options = default)


Expose overloads of protocol methods as extensions to the base client.

annelo-msft · 2024-04-23T23:19:13Z

+namespace AzureOpenAI.Models;
+
+[PersistableModelProxy(typeof(UnknownAzureChatExtensionConfiguration))]
+public partial class AzureChatExtensionConfiguration : IJsonModel<AzureChatExtensionConfiguration>


Models specific to Azure are generated in standard way.

annelo-msft · 2024-04-23T23:19:50Z

+
+namespace AzureOpenAI.Models;
+
+internal class AzureCreateChatCompletionRequest : IJsonModel<AzureCreateChatCompletionRequest>


Internal model wraps third-party model to enable serializing model with Azure-added properties.

annelo-msft · 2024-04-23T23:20:23Z

+        writer.WriteStartObject();
+
+        // Note that we write the base model differently in this context
+        ((IJsonModel<object>)_request).Write(writer, new ModelReaderWriterOptions("W*"));


Writes third-party model using "W*"/partial format.

annelo-msft · 2024-04-23T23:23:32Z

+
+namespace AzureOpenAI;
+
+internal class JsonModelList<TModel> : List<TModel>, IJsonModel<JsonModelList<TModel>>


Proposal for emitted type to simplify reading/writing collections.

annelo-msft · 2024-04-23T23:24:07Z

+
+public static class CreateChatCompletionRequestExtensions
+{
+    public static IList<AzureChatExtensionConfiguration> GetDataSources(this CreateChatCompletionRequest request)


To be replaced with DataSources extension property in the future.

annelo-msft · 2024-04-23T23:24:32Z

+
+        JsonModelList<AzureChatExtensionConfiguration> dataSources;
+
+        if (model.AdditionalProperties.TryGetValue("data_sources", out object? value))


Retrieves value from AdditionalProperties dictionary, or creates it new and adds it to the dictionary.

annelo-msft · 2024-04-23T23:25:05Z

+
+namespace AzureOpenAI.Models;
+
+public partial class AzureChatExtensionDataSourceResponseCitation :  IJsonModel<AzureChatExtensionDataSourceResponseCitation>


public output models specific to Azure are generated normally.

annelo-msft · 2024-04-23T23:25:22Z

+public static class ChatCompletionResponseMessageExtensions
+{
+    // Output property
+    public static AzureChatExtensionsMessageContext? GetAzureExtensionsContext(this ChatCompletionResponseMessage message)


To be replaced with extension property AzureExtensionsContext in the future.

annelo-msft · 2024-04-23T23:25:47Z

+        // It's either deserialized already or not. Find out now.
+        // This should work for all cases because we know the contract regarding
+        // BinaryData values.
+        if (value is BinaryData serializedValue)


Deserialized value if needed and returns from extension property.

annelo-msft · 2024-04-23T23:26:15Z

        /// <param name="keyCredential"> The key credential to copy. </param>
        /// <param name="endpoint"> OpenAI Endpoint. </param>
-        internal Chat(ClientPipeline pipeline, ApiKeyCredential keyCredential, Uri endpoint)
+        protected internal Chat(ClientPipeline pipeline, ApiKeyCredential keyCredential, Uri endpoint)


Third-party subclient constructor becomes protected internal.

annelo-msft · 2024-04-23T23:26:43Z

 namespace OpenAI.Models
 {
-    public partial class ChatCompletionResponseMessage : IJsonModel<ChatCompletionResponseMessage>
+    public partial class ChatCompletionResponseMessage : JsonModel<ChatCompletionResponseMessage>


Inherits from JsonModel abstract class in System.ClientModel to add AdditionalProperties dictionary.

annelo-msft · 2024-04-23T23:27:05Z

-                    }
-#endif
-                }
+                WriteUnknownProperties(writer, options);


Writes unknown properties through protected method on the base type.

annelo-msft · 2024-04-23T23:27:44Z

-        /// </list>
-        /// </para>
-        /// </summary>
-        private IDictionary<string, BinaryData> _serializedAdditionalRawData;


Serialized values are now handled through AdditionalProperties dictionary on base JsonModel type.

annelo-msft · 2024-04-23T23:28:45Z

            }

-            writer.WriteStartObject();
+            if (format == "J")


Allow writing partial content instead of full object to enable writing content with additions in the extended type.

annelo-msft added 30 commits April 8, 2024 14:09

Add Azure client

df95c53

updates

60575d9

updates

45b680a

beginning work on extension methods

51e2eec

WIP

0ac5efe

Initial implementation of settable extension property

8ca386f

functional prototype for inputs; with lots of open questions

443c709

updates

43b4afe

starting the model-inheritance design exploration

99f8265

update to dictionary approach

9140b1d

nits

fa9fbff

Updates to JsonModelList, input extension

9699f63

add 'emitted' subtype of BinaryData

0829d5c

update 3rd-party model serialization method

d6dcaca

add Azure response subtypes

8ab3c85

updates

f10b1bb

use dictionary for output model

09e5bc6

functional e2e, using 'AW' approach - this doesn't work

5d6b7b2

make e2e work by separating which types of values are written from di…

56639be

…ctionary

WIP - trying Mads's suggestion

b2ac716

use local ClientModel; remove BinaryData subtype; update input extens…

04c0255

…ions

rework input model wrapper type

0652870

handle output models; no change really

24124c9

nits

687a88a

nits

8226cf8

updates

f629e00

nits and updates

d97c28c

rework things a bit

6f2b486

revert unneeded changes

b124eff

update clients

4cf6fa6

rename file

6ae64d3

annelo-msft commented Apr 23, 2024

View reviewed changes

nits

dd32b09

annelo-msft commented Apr 23, 2024

View reviewed changes


		namespace AzureOpenAI.Models;

		internal class AzureCreateChatCompletionRequest : IJsonModel<AzureCreateChatCompletionRequest>


		namespace AzureOpenAI;

		internal class JsonModelList<TModel> : List<TModel>, IJsonModel<JsonModelList<TModel>>


		JsonModelList<AzureChatExtensionConfiguration> dataSources;

		if (model.AdditionalProperties.TryGetValue("data_sources", out object? value))


		namespace AzureOpenAI.Models;

		public partial class AzureChatExtensionDataSourceResponseCitation : IJsonModel<AzureChatExtensionDataSourceResponseCitation>

Conversation

annelo-msft commented Apr 22, 2024

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant