TypeSpec for Azure AI Inference #28438
Open
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…rl internal, since its using the ImageUrl class
…is-beta-2-updates
dargilco
requested review from
vicancy and
bexxx
and removed request for
a team
Next Steps to MergeNext steps that must be taken to merge this PR:
|
Swagger Validation Report
|
| Compared specs (v2.2.2) | new version | base version |
|---|---|---|
| package-2024-05-01-preview | package-2024-05-01-preview(dda6da8) | default(main) |
[must fix]The following errors/warnings are introduced by current PR:
️️✔️Avocado succeeded [Detail] [Expand]
Validation passes for Avocado.
️️✔️SwaggerAPIView succeeded [Detail] [Expand]
️️✔️TypeSpecAPIView succeeded [Detail] [Expand]
️️✔️ModelValidation succeeded [Detail] [Expand]
Validation passes for ModelValidation.
️️✔️SemanticValidation succeeded [Detail] [Expand]
Validation passes for SemanticValidation.
️️✔️PoliCheck succeeded [Detail] [Expand]
Validation passed for PoliCheck.
️️✔️SpellCheck succeeded [Detail] [Expand]
Validation passes for SpellCheck.
️️✔️Lint(RPaaS) succeeded [Detail] [Expand]
Validation passes for Lint(RPaaS).
️️✔️PR Summary succeeded [Detail] [Expand]
Validation passes for Summary.
️️✔️Automated merging requirements met succeeded [Detail] [Expand]
Swagger Generation Artifacts
|
Generated ApiView
|
AzureRestAPISpecReview
added
BreakingChangeReviewRequired
This was referenced Mar 25, 2024
Closed
…l only support function
6 tasks
…ted models to use the work `choice` instead of selection, similar to OAI TypeSpec
…at it only works for MaaS/MaaP
|
Hi, @dargilco. Your PR has no update for 14 days and it is marked as stale PR. If no further update for over 14 days, the bot will close the PR. If you want to refresh the PR, please remove |
microsoft-github-policy-service
bot
added
the
no-recent-activity
|
Hi, @dargilco. The PR will be closed since the PR has no update for 28 days. If you still need the PR review to proceed, please reopen it and @ mention PR assignee. |
4 tasks
Comment on lines
+18
to
+60
| @doc(""" | ||
| Gets chat completions for the provided chat messages. | ||
| Completions support a wide variety of tasks and generate text that continues from or "completes" | ||
| provided prompt data. The method makes a REST API call to the `/chat/completions` route | ||
| on the given endpoint. | ||
| """) | ||
| @actionSeparator("/") | ||
| @route("chat/completions") | ||
| op getChatCompletions is Azure.Core.RpcOperation< | ||
| { | ||
| ...ChatCompletionsOptions; | ||
| ...AdditionalRequestHeaders; | ||
| }, | ||
| ChatCompletions | ||
| >; | ||
|
|
||
| @doc(""" | ||
| Return the embedding vectors for given text prompts. | ||
| The method makes a REST API call to the `/embeddings` route on the given endpoint. | ||
| """) | ||
| @actionSeparator("/") | ||
| @route("embeddings") | ||
| op getEmbeddings is Azure.Core.RpcOperation< | ||
| { | ||
| ...EmbeddingsOptions; | ||
| ...AdditionalRequestHeaders; | ||
| }, | ||
| EmbeddingsResult | ||
| >; | ||
|
|
||
| @doc(""" | ||
| Return the embedding vectors for given images. | ||
| The method makes a REST API call to the `/images/embeddings` route on the given endpoint. | ||
| """) | ||
| @actionSeparator("/") | ||
| @route("images/embeddings") | ||
| op getImageEmbeddings is Azure.Core.RpcOperation< | ||
| { | ||
| ...ImageEmbeddingsOptions; | ||
| ...AdditionalRequestHeaders; | ||
| }, | ||
| EmbeddingsResult | ||
| >; |
There was a problem hiding this comment.
Please check if you can put the ###Options (also make it a model instead of alias) with the explicit @body, e.g.
op getChatCompletions is Azure.Core.RpcOperation<
{
@body body: ChatCompletionsOptions;
...AdditionalRequestHeaders;
},
ChatCompletions
>;The reason is that without @body, typespec would let SDK emitter spread the body properties to method signature. (see spread).
However, SDK emitter cannot handle the ...Record<unknown> in the ###Options in above "spread" scenario. -- ...Record<unknown> means any key-value can be set to the request body, but that won't work if the request body properties now on the method signature.
Therefore, this is going to cause problem to SDK emitters, and the output code would be incorrect.
There was a problem hiding this comment.
Basically, the request body can have properties
messages
frequency_penalty
stream
...
model
<any undocumented properties, as the Record>
But after the spread in method (the way of writing the alias ChatCompletionsOptions in getChatCompletions without @body), the SDK method signature would become
getChatCompletions(messages, frequency_penalty, stream, ..., model) without a way to specify undocumented properties.
dargilco
removed
the
no-recent-activity
|
Assigned to @dargilco |
|
Thank you for your comment @weidongxu-microsoft , I will look into this when I get back to working on this SDK, as part of releasing another beta version. |
|
Hi, @dargilco. Your PR has no update for 14 days and it is marked as stale PR. If no further update for over 14 days, the bot will close the PR. If you want to refresh the PR, please remove |
microsoft-github-policy-service
bot
added
the
no-recent-activity
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Data Plane API - Pull Request
Azure AI Studio now hosts a catalog of Large Language Models (LLM) for chat completion, embeddings and image generation, as two offerings called Model-as-a-Service (MaaS, aka "Pay as you go") and Model-as-a-Platform (MaaP, aka "Real-time endpoint"). This TypeSpec defines the common REST API for endpoints hosting these models. It is checked in here to facilitate auto-geration of the client libraries (Python, JS and C# at the moment). Other programing languages will follow. The service team uses Python code which at run-time auto-generates a Swagger file for the service. They do not use the TypeSpec files in this PR. But I am verifying that their Swagger and the one created by this TypeSpec are equivalent, and the emitted client libraries work properly when connecting to this service.
For more information on the catalog see Explore the model catalog in Azure AI Studio.
This TypeSpec is a subset of the Azure OpenAI Inference TypeSpec and additional changes. We use the same model and property names for now, but that may be subject to change. It implements four routes /chat/completions, /embeddings, /image-generation/ and /info.
Is this review for (select one):