fix: surface image generation errors instead of returning empty HTTP 200 - #13000
Draft
marckarp wants to merge 2 commits into
Draft
fix: surface image generation errors instead of returning empty HTTP 200#13000marckarp wants to merge 2 commits into
marckarp wants to merge 2 commits into
Conversation
A failed /v1/images/generations request previously returned HTTP 200 with
data=[] and no error message. Two layers caused this:
- The SGLang image diffusion handler caught every exception and yielded
{"data": [], "error": msg}. The OpenAI ImagesResponse schema has no
error field, so the message was silently dropped during
deserialization and the client saw a successful empty response.
- The frontend images route collapsed any stream error into a hardcoded
generic 500, discarding the worker's error message.
Fix:
- Handler: let exceptions propagate; the runtime converts a raised
exception into an error event on the response stream, which the
frontend folds into a non-200 HTTP error.
- Handler: raise typed InvalidArgument for request validation failures
(blank input_reference, malformed size) so they surface as HTTP 400
with the message, matching the existing processor pattern.
- Frontend: route the images stream fold error through
ErrorMessage::from_anyhow so InvalidArgument keeps 400-with-message
semantics while internal errors remain sanitized 500s.
Verified with unit tests (23/23 in the runtime image) and a live
deployment: a worker-side failure now returns a non-200 response
instead of an empty 200.
Signed-off-by: Marc Karp <mkarp@nvidia.com>
Co-Authored-By: Claude <noreply@anthropic.com>
marckarp
temporarily deployed
to
external_collaborator
August 11, 2026 05:30 — with
GitHub Actions
Inactive
marckarp
temporarily deployed
to
external_collaborator
August 11, 2026 05:30 — with
GitHub Actions
Inactive
Contributor
|
👋 Hi marckarp! Thank you for contributing to ai-dynamo/dynamo. Just a reminder: The 🚀 |
|
Signed-off-by: Marc Karp <mkarp@nvidia.com> Co-Authored-By: Claude <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Overview
A failed
/v1/images/generationsrequest returns HTTP 200 withdata: []and no error message. The client has no way to tell a failed generation from an empty one. Easy to reproduce: send any request that makes the worker raise (e.g. an out-of-range parameter or a backend failure) — the response is a successful-looking empty body.Two layers cause it:
{"data": [], "error": str(e)}. The OpenAIImagesResponseschema has noerrorfield, so the message is silently dropped during deserialization and the frontend returns HTTP 200 with empty data."Failed to fold images stream"), so the worker's message is only visible in server logs.What this PR changes
Handler (
image_diffusion_handler.py):Annotated::from_err), which the frontend folds into a non-200 HTTP error.input_reference, malformedsize) now raise the typedInvalidArgumentinstead of bareValueError, matching the existing pattern insglang_processor.py.InvalidArgumentmaps to HTTP 400 with the message visible to the client; other exceptions remain sanitized 500s.Frontend (
lib/llm/src/http/service/openai.rs):ErrorMessage::from_anyhowinstead of the hardcoded generic 500. This follows the route's existing sanitization policy:InvalidArgument→ 400 with message, internal errors → sanitized 500 (details stay server-side).Tests: the unit test that asserted the old swallow behavior now asserts propagation; added tests for the
InvalidArgumentvalidation paths.Verification
sglang-runtime:1.3.0with the patched handler (the 8 behavior-dependent tests fail against the stock handler, confirming they detect the old behavior).input_reference) → HTTP 500 instead of the old empty HTTP 200cargo checkintroduces no new errors. A live 400-with-message demonstration requires a rebuilt frontend image (the released frontend predates this change), so that path relies on CI + the unit-level guarantees offrom_anyhow(which existing chat/completions routes already use).Notes for reviewers
_parse_sizeguard is currently unreachable from HTTP because the frontend'ssizefield is a fixed enum that rejects free-form values first. It becomes load-bearing with the planned change to accept arbitraryWxHsizes (needed for diffusion models; the videos route already takes a free-form string), and is worth having defensively regardless.nparameter in the SGLang image diffusion handler #12970: both touchtest_sglang_image_diffusion_handler.pyin different places; whichever merges second rebases trivially. fix: honor the OpenAInparameter in the SGLang image diffusion handler #12970's live test also demonstrated this bug: requestingn=11returnedHTTP 200withdata=[]and an empty error on the wire.Part of a series of fixes for the SGLang image diffusion path (see #12970 for the first).
🤖 Generated with Claude Code
Part of DYN-3974