Skip to content

fix: surface image generation errors instead of returning empty HTTP 200 - #13000

Draft
marckarp wants to merge 2 commits into
ai-dynamo:mainfrom
marckarp:fix/sglang-image-error-surfacing
Draft

fix: surface image generation errors instead of returning empty HTTP 200#13000
marckarp wants to merge 2 commits into
ai-dynamo:mainfrom
marckarp:fix/sglang-image-error-surfacing

Conversation

@marckarp

@marckarp marckarp commented Aug 11, 2026

Copy link
Copy Markdown
Contributor

Overview

A failed /v1/images/generations request returns HTTP 200 with data: [] and no error message. The client has no way to tell a failed generation from an empty one. Easy to reproduce: send any request that makes the worker raise (e.g. an out-of-range parameter or a backend failure) — the response is a successful-looking empty body.

Two layers cause it:

  1. Worker (Python): the SGLang image diffusion handler catches every exception and yields {"data": [], "error": str(e)}. The OpenAI ImagesResponse schema has no error field, so the message is silently dropped during deserialization and the frontend returns HTTP 200 with empty data.
  2. Frontend (Rust): even when an exception does propagate as an error event on the response stream, the images route discards it and returns a hardcoded generic 500 ("Failed to fold images stream"), so the worker's message is only visible in server logs.

What this PR changes

Handler (image_diffusion_handler.py):

  • Remove the catch-and-yield: exceptions now propagate. The runtime converts a raised exception into an error event on the response stream (Annotated::from_err), which the frontend folds into a non-200 HTTP error.
  • Request-validation failures (blank input_reference, malformed size) now raise the typed InvalidArgument instead of bare ValueError, matching the existing pattern in sglang_processor.py. InvalidArgument maps to HTTP 400 with the message visible to the client; other exceptions remain sanitized 500s.

Frontend (lib/llm/src/http/service/openai.rs):

  • Route the images stream fold error through ErrorMessage::from_anyhow instead of the hardcoded generic 500. This follows the route's existing sanitization policy: InvalidArgument → 400 with message, internal errors → sanitized 500 (details stay server-side).

Tests: the unit test that asserted the old swallow behavior now asserts propagation; added tests for the InvalidArgument validation paths.

Verification

  • Unit tests: 23/23 pass inside sglang-runtime:1.3.0 with the patched handler (the 8 behavior-dependent tests fail against the stock handler, confirming they detect the old behavior).
  • Live e2e on a single-GPU FLUX.1-schnell deployment with the patched worker:
    • valid request → HTTP 200 with image (no regression)
    • worker-side failure (blank input_reference) → HTTP 500 instead of the old empty HTTP 200
  • Rust half: cargo check introduces no new errors. A live 400-with-message demonstration requires a rebuilt frontend image (the released frontend predates this change), so that path relies on CI + the unit-level guarantees of from_anyhow (which existing chat/completions routes already use).

Notes for reviewers

Part of a series of fixes for the SGLang image diffusion path (see #12970 for the first).

🤖 Generated with Claude Code

Part of DYN-3974

A failed /v1/images/generations request previously returned HTTP 200 with
data=[] and no error message. Two layers caused this:

- The SGLang image diffusion handler caught every exception and yielded
  {"data": [], "error": msg}. The OpenAI ImagesResponse schema has no
  error field, so the message was silently dropped during
  deserialization and the client saw a successful empty response.
- The frontend images route collapsed any stream error into a hardcoded
  generic 500, discarding the worker's error message.

Fix:
- Handler: let exceptions propagate; the runtime converts a raised
  exception into an error event on the response stream, which the
  frontend folds into a non-200 HTTP error.
- Handler: raise typed InvalidArgument for request validation failures
  (blank input_reference, malformed size) so they surface as HTTP 400
  with the message, matching the existing processor pattern.
- Frontend: route the images stream fold error through
  ErrorMessage::from_anyhow so InvalidArgument keeps 400-with-message
  semantics while internal errors remain sanitized 500s.

Verified with unit tests (23/23 in the runtime image) and a live
deployment: a worker-side failure now returns a non-200 response
instead of an empty 200.

Signed-off-by: Marc Karp <mkarp@nvidia.com>
Co-Authored-By: Claude <noreply@anthropic.com>
@copy-pr-bot

copy-pr-bot Bot commented Aug 11, 2026

Copy link
Copy Markdown

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

@marckarp
marckarp temporarily deployed to external_collaborator August 11, 2026 05:30 — with GitHub Actions Inactive
@marckarp
marckarp temporarily deployed to external_collaborator August 11, 2026 05:30 — with GitHub Actions Inactive
@github-actions

Copy link
Copy Markdown
Contributor

👋 Hi marckarp! Thank you for contributing to ai-dynamo/dynamo.

Just a reminder: The NVIDIA Test Github Validation CI runs an essential subset of the testing framework to quickly catch errors.Your PR reviewers may elect to test the changes comprehensively before approving your changes.

🚀

@github-actions github-actions Bot added external-contribution Pull request is from an external contributor fix backend::sglang Relates to the sglang backend frontend `python -m dynamo.frontend` and `dynamo-run in=http|text|grpc` labels Aug 11, 2026
@datadog-official

Copy link
Copy Markdown

Pipelines

⚠️ Warnings

🚦 2 Pipeline jobs failed

Pre Merge | pre-commit   View in Datadog   GitHub Actions

🔧 Fix in code. This looks caused by changes in this PR. isort failed due to code format issues in image_diffusion_handler.py requiring fixes.

Pre Merge | pre-merge-status-check   View in Datadog   GitHub Actions

📋 Copy prompt for your agent
CI on my pull request is failing. Help me find and fix the root cause of each failing job below — they were flagged as caused by changes in this PR, so focus on the diff. For each job, explain the failure and propose a fix.

Branch: fix/sglang-image-error-surfacing

Pre Merge | pre-commit
Commit: 4e1ea13b68d9b881ec4724c8326c0f20352b3b71
Error (code / quality):
isort failed due to code format issues in image_diffusion_handler.py requiring fixes.
CI job: https://github.com/ai-dynamo/dynamo/actions/runs/31461853106/job/93686788645

Useful? React with 👍 / 👎

This comment will be updated automatically if new data arrives.
🔗 Commit SHA: 4e1ea13 | Docs | Datadog PR Page | Give us feedback!

Signed-off-by: Marc Karp <mkarp@nvidia.com>
Co-Authored-By: Claude <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

backend::sglang Relates to the sglang backend external-contribution Pull request is from an external contributor fix frontend `python -m dynamo.frontend` and `dynamo-run in=http|text|grpc` size/L

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant