Skip to content

fix: make sure choices is not nil or None to make it compatible with OpenAI#1975

Merged
yuzisun merged 2 commits intoenvoyproxy:mainfrom
hustxiayang:fix-choices
Mar 20, 2026
Merged

fix: make sure choices is not nil or None to make it compatible with OpenAI#1975
yuzisun merged 2 commits intoenvoyproxy:mainfrom
hustxiayang:fix-choices

Conversation

@hustxiayang
Copy link
Contributor

@hustxiayang hustxiayang commented Mar 19, 2026

Description
Currently, it sets omitempty for choices, it would cause choices to be None, which is not compatible with OpenAI and caused programs to crash.
see openai-go: https://github.com/openai/openai-go/blob/6723d0b464f1d6898d7242efad6b94cc3ad7dfe6/chatcompletion.go#L183-L184

Fix: remove the omitempty

@hustxiayang hustxiayang requested a review from a team as a code owner March 19, 2026 21:01
@dosubot dosubot bot added the size:S This PR changes 10-29 lines, ignoring generated files. label Mar 19, 2026
@dosubot
Copy link

dosubot bot commented Mar 19, 2026

Related Documentation

1 document(s) may need updating based on files changed in this PR:

Envoy's Space

connect-providers /ai-gateway/blob/main/site/docs/capabilities/llm-integrations/connect-providers.md
View Suggested Changes
@@ -304,8 +304,18 @@
 4. **Authentication**: The AIServiceBackend's security policy provides credentials for upstream authentication
 5. **Schema Transformation**: If needed, the request is transformed from the input schema to the backend's expected schema
 6. **Provider Communication**: The request is forwarded to the actual AI provider with proper authentication
-7. **Response Processing**: The provider's response is transformed back to the unified schema format
+7. **Response Processing**: The provider's response is transformed back to the unified schema format with strict OpenAI API compliance
 8. **Client Response**: The standardized response is returned to the client
+
+### OpenAI API Compatibility
+
+The AI Gateway ensures strict OpenAI API compliance when transforming responses from various providers. This guarantees compatibility with OpenAI client libraries and tools that expect standard OpenAI response structures:
+
+- **Response Schema Compliance**: All response fields match OpenAI's schema exactly, regardless of the backend provider
+- **Streaming Response Format**: In streaming chat completion responses, the `choices` field is always present as an array, even when empty (never null or omitted)
+- **Consistent Structure**: Streaming chunks follow OpenAI's format precisely, ensuring client applications can reliably parse responses without encountering unexpected null values
+
+This strict compliance prevents issues where client programs might crash when encountering non-standard response formats from different providers.
 
 ## Common Configuration Patterns
 

[Accept] [Decline]

Note: You must be authenticated to accept/decline updates.

How did I do? Any feedback?  Join Discord

@codecov-commenter
Copy link

codecov-commenter commented Mar 19, 2026

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 84.33%. Comparing base (0ef2687) to head (f609f0e).

Additional details and impacted files
@@           Coverage Diff           @@
##             main    #1975   +/-   ##
=======================================
  Coverage   84.33%   84.33%           
=======================================
  Files         130      130           
  Lines       17986    17987    +1     
=======================================
+ Hits        15169    15170    +1     
  Misses       1873     1873           
  Partials      944      944           

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@hustxiayang
Copy link
Contributor Author

/retest

Signed-off-by: yxia216 <yxia216@bloomberg.net>
@dosubot dosubot bot added size:M This PR changes 30-99 lines, ignoring generated files. and removed size:S This PR changes 10-29 lines, ignoring generated files. labels Mar 20, 2026
@hustxiayang
Copy link
Contributor Author

/retest

@yuzisun yuzisun merged commit 98200a4 into envoyproxy:main Mar 20, 2026
55 of 57 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

size:M This PR changes 30-99 lines, ignoring generated files.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants