[BUG] GetChatCompletionsStreamAsyncSample code not working

[ventaglio]

Describe the bug
I am not able to get to a working asynchronous chat stream example using GPT-4 through Azure SDK for Java.
I started from the example code provided here
https://github.com/Azure/azure-sdk-for-java/blob/azure-ai-openai_1.0.0-beta.9/sdk/openai/azure-ai-openai/src/samples/java/com/azure/ai/openai/usage/GetChatCompletionsStreamAsyncSample.java
but it does not work out of the box.
When I executed that code, providing the necessary parameters, I got an error.

Exception or Stack Trace
There was an error getting chat completions.java.lang.NullPointerException: The mapper [it.celi.dialog.GetChat$$Lambda$143/0x00000008002a0000] returned a null value.

To Reproduce
Just execute the example code provided here:
https://github.com/Azure/azure-sdk-for-java/blob/azure-ai-openai_1.0.0-beta.9/sdk/openai/azure-ai-openai/src/samples/java/com/azure/ai/openai/usage/GetChatCompletionsStreamAsyncSample.java

Code Snippet
I tried to fix the issue by modifying these lines:
if (CoreUtils.isNullOrEmpty(chatCompletions.getChoices())) { return null; }
this way
if (CoreUtils.isNullOrEmpty(chatCompletions.getChoices())) { return ""; }

This change allowed me to execute the code without NPE errors, but the final behavior is different from what I expected it to be.

Expected behavior
I expected to get the response token by token in an asynchronous way, but I instead received only
Role = assistant

Then I increased the timeout from 10 to 60 seconds
TimeUnit.SECONDS.sleep(60);

This way I got:
Role = assistant immediately
... the full response after more or less 15 seconds.
So the streaming is not working in an asynchronous way.

Screenshots

Setup (please complete the following information):

• OS: macOS Sonoma 14.4.1
• IDE: IntelliJ IDEA 2024.1.3
• Library/Libraries: com.azure:azure-ai-openai:1.0.0-beta.9
• Java version: 17.0.9 Temurin
• App Server/Environment: -
• Frameworks: -

Additional context

Information Checklist
Kindly make sure that you have added all the following information above and checkoff the required fields otherwise we will treat the issuer as an incomplete report

• Bug Description Added
• Repro Steps Added
• Setup information Added

[ventaglio]

Update: I managed to get the asynchronous streaming working by using another deployment even if I have not yet understood what is the difference between the two deployments.
Is there something I can check to understand why one is not working in asynchronous mode and the other it is, using the same exact Java code?

Anyway the example code contains at least a bug in the return null line.

[ventaglio]

Update: we finally understood what was wrong with the first deployment. It had a custom filter that was not configured in "async mode" so it was blocking the asynchronous response.
I leave the issue open just to fix the return null line error.

[mssfang]

Thank you for reporting this issue. Will be fixed soon