[OpenAI] Whisper support

.vscode/cspell.json

@@ -330,6 +330,7 @@
     "Mockito",
     "Mordor",
     "mosca",
+    "mpga",
     "msal",
     "msix",
     "MSRC",

eng/code-quality-reports/src/main/resources/spotbugs/spotbugs-exclude.xml

@@ -2697,4 +2697,18 @@
     <Method name="transfer"/>
     <Bug pattern="NP_NULL_ON_SOME_PATH"/>
   </Match>
+
+   <!-- Returning a new copy of the object is not necessary -->
+  <Match>
+    <Class name="~com\.azure\.ai\.openai\.models\.(AudioTranscriptionOptions|AudioTranslationOptions)"/>
+    <Method name="&lt;init&gt;"/>
+    <Bug pattern="EI_EXPOSE_REP2"/>
+  </Match>
+
+  <!-- False positive, it throws an IllegalArgumentException if non-exist method parameter type passed.-->
+  <Match>
+    <Class name="com.azure.ai.openai.implementation.MultipartDataHelper"/>
+    <Method name="serializeRequest"/>
+    <Bug pattern="BC_UNCONFIRMED_CAST_OF_RETURN_VALUE"/>
+  </Match>
 </FindBugsFilter>

sdk/openai/azure-ai-openai/CHANGELOG.md

@@ -4,6 +4,10 @@
 
 ### Features Added
 
+- Support for `Whisper` endpoints was added.
+- Translation and Transcription of audio files is available
+- The above features are available both in Azure and non-Azure OpenAI
+
 ### Breaking Changes
 
 ### Bugs Fixed

sdk/openai/azure-ai-openai/README.md

@@ -19,6 +19,8 @@ For concrete examples you can have a look at the following links. Some of the mo
 * [Streaming chat completions sample](#streaming-chat-completions "Streaming chat completions")
 * [Embeddings sample](#text-embeddings "Text Embeddings")
 * [Image Generation sample](#image-generation "Image Generation")
+* [Audio Transcription sample](#audio-transcription "Audio Transcription")
+* [Audio Translation sample](#audio-translation "Audio Translation")
 
 If you want to see the full code for these snippets check out our [samples folder][samples_folder].
 
@@ -150,6 +152,8 @@ The following sections provide several code snippets covering some of the most c
 * [Streaming chat completions sample](#streaming-chat-completions "Streaming chat completions")
 * [Embeddings sample](#text-embeddings "Text Embeddings")
 * [Image Generation sample](#image-generation "Image Generation")
+* [Audio Transcription sample](#audio-transcription "Audio Transcription")
+* [Audio Translation sample](#audio-translation "Audio Translation")
 
 ### Text completions
 
@@ -286,6 +290,44 @@ for (ImageLocation imageLocation : images.getData()) {
 
 For a complete sample example, see sample [Image Generation][sample_image_generation].
 
+### Audio Transcription
+The OpenAI service starts supporting `audio transcription` with the introduction of `Whisper` models. 
+The following code snippet shows how to use the service to transcribe audio.
+
+```java readme-sample-audioTranscription
+String fileName = "{your-file-name}";
+Path filePath = Paths.get("{your-file-path}" + fileName);
+
+byte[] file = BinaryData.fromFile(filePath).toBytes();
+AudioTranscriptionOptions transcriptionOptions = new AudioTranscriptionOptions(file)
+    .setResponseFormat(AudioTranscriptionFormat.JSON);
+
+AudioTranscription transcription = client.getAudioTranscription("{deploymentOrModelId}", fileName, transcriptionOptions);
+
+System.out.println("Transcription: " + transcription.getText());
+```
+For a complete sample example, see sample [Audio Transcription][sample_audio_transcription].
+Please refer to the service documentation for a conceptual discussion of [Whisper][microsoft_docs_whisper_model].
+
+### Audio Translation
+The OpenAI service starts supporting `audio translation` with the introduction of `Whisper` models.
+The following code snippet shows how to use the service to translate audio.
+
+```java readme-sample-audioTranslation
+String fileName = "{your-file-name}";
+Path filePath = Paths.get("{your-file-path}" + fileName);
+
+byte[] file = BinaryData.fromFile(filePath).toBytes();
+AudioTranslationOptions translationOptions = new AudioTranslationOptions(file)
+    .setResponseFormat(AudioTranscriptionFormat.JSON);
+
+AudioTranscription translation = client.getAudioTranslation("{deploymentOrModelId}", fileName, translationOptions);
+
+System.out.println("Translation: " + translation.getText());
+```
+For a complete sample example, see sample [Audio Translation][sample_audio_translation].
+Please refer to the service documentation for a conceptual discussion of [Whisper][microsoft_docs_whisper_model].
+
 ## Troubleshooting
 ### Enable client logging
 You can set the `AZURE_LOG_LEVEL` environment variable to view logging statements made in the client library. For
@@ -327,6 +369,7 @@ For details on contributing to this repository, see the [contributing guide](htt
 [logLevels]: https://github.com/Azure/azure-sdk-for-java/blob/main/sdk/core/azure-core/src/main/java/com/azure/core/util/logging/ClientLogger.java
 [microsoft_docs_openai_completion]: https://learn.microsoft.com/azure/cognitive-services/openai/how-to/completions
 [microsoft_docs_openai_embedding]: https://learn.microsoft.com/azure/cognitive-services/openai/concepts/understand-embeddings
+[microsoft_docs_whisper_model]: https://learn.microsoft.com/azure/ai-services/openai/whisper-quickstart?tabs=command-line
 [non_azure_openai_authentication]: https://platform.openai.com/docs/api-reference/authentication
 [performance_tuning]: https://github.com/Azure/azure-sdk-for-java/wiki/Performance-Tuning
 [product_documentation]: https://azure.microsoft.com/services/
@@ -342,6 +385,8 @@ For details on contributing to this repository, see the [contributing guide](htt
 [sample_get_completions_streaming]: https://github.com/Azure/azure-sdk-for-java/blob/main/sdk/openai/azure-ai-openai/src/samples/java/com/azure/ai/openai/usage/GetCompletionsStreamSample.java
 [sample_get_embedding]: https://github.com/Azure/azure-sdk-for-java/blob/main/sdk/openai/azure-ai-openai/src/samples/java/com/azure/ai/openai/usage/GetEmbeddingsSample.java
 [sample_image_generation]: https://github.com/Azure/azure-sdk-for-java/blob/main/sdk/openai/azure-ai-openai/src/samples/java/com/azure/ai/openai/usage/GetImagesSample.java
+[sample_audio_transcription]: https://github.com/Azure/azure-sdk-for-java/blob/main/sdk/openai/azure-ai-openai/src/samples/java/com/azure/ai/openai/usage/AudioTranscriptionSample.java
+[sample_audio_translation]: https://github.com/Azure/azure-sdk-for-java/blob/main/sdk/openai/azure-ai-openai/src/samples/java/com/azure/ai/openai/usage/AudioTranslationSample.java
 [openai_client_async]: https://github.com/Azure/azure-sdk-for-java/blob/main/sdk/openai/azure-ai-openai/src/main/java/com/azure/ai/openai/OpenAIAsyncClient.java
 [openai_client_builder]: https://github.com/Azure/azure-sdk-for-java/blob/main/sdk/openai/azure-ai-openai/src/main/java/com/azure/ai/openai/OpenAIClientBuilder.java
 [openai_client_sync]: https://github.com/Azure/azure-sdk-for-java/blob/main/sdk/openai/azure-ai-openai/src/main/java/com/azure/ai/openai/OpenAIClient.java

sdk/openai/azure-ai-openai/assets.json

@@ -2,5 +2,5 @@
   "AssetsRepo": "Azure/azure-sdk-assets",
   "AssetsRepoPrefixPath": "java",
   "TagPrefix": "java/openai/azure-ai-openai",
-  "Tag": "java/openai/azure-ai-openai_57107e7a09"
+  "Tag": "java/openai/azure-ai-openai_3c34d9f076"
 }