Skip to content

Classify malformed-multimodal rejects as invalid_request - #27451

Merged
merrymercy merged 2 commits into
mainfrom
classify-malformed-multimodal-invalid-request
Jun 6, 2026
Merged

merrymercy merged 2 commits into
mainfrom
classify-malformed-multimodal-invalid-request

Conversation

@merrymercy

@merrymercy merrymercy commented Jun 6, 2026 •

Copy link
Copy Markdown
Contributor

Summary

  • When a multimodal request fails validation (e.g. bad base64), return a structured error with type=invalid_request_error and code=400 instead of a bare error message.
  • Detect client disconnects in the streaming path and exit cleanly instead of emitting a 400.
  • In the multimodal base processor, re-raise ValueError separately from RuntimeError so callers can distinguish bad input (4xx) from internal failures (5xx), and chain the original exception via from e.

Original commits

  • 1bce350b1

Test plan

  • Send a streaming request with a malformed base64 image and verify the SSE error event contains "type": "invalid_request_error" and "code": 400.
  • Send a streaming multimodal request and disconnect the client mid-flight; verify the server logs an info-level disconnect message and does not emit a 400 error event.
  • Send a valid multimodal request and confirm it still succeeds end-to-end.

CI States

Latest PR Test (Base): 🚫 Run #27066493925
Latest PR Test (Extra): ❌ Run #27066493871

When a multimodal request fails validation (e.g. bad base64 input),
return a structured error with type=invalid_request_error and code=400
instead of a bare error message. Also detect client disconnects in the
streaming path and exit cleanly instead of emitting a 400.

In the multimodal processor, re-raise ValueError separately from
RuntimeError so callers can distinguish bad input (4xx) from internal
failures (5xx), and chain the original exception for better traceability.

Co-authored-by: cctry <cctry@meta.com>
Co-authored-by: cctry <cctry@fb.com>
@merrymercy

Copy link
Copy Markdown
Contributor Author

/tag-and-rerun-ci

@github-actions github-actions Bot added the run-ci CI: run the baseline test suite on this PR label Jun 6, 2026

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request improves error handling in the HTTP server and multimodal processors. Specifically, it handles client disconnects gracefully during streaming, structures streaming ValueError responses as 400 errors, and propagates ValueErrors during multimodal data loading to avoid 500 server errors. Feedback highlights two main areas for improvement: first, truncating the raw data string in base_processor.py exception messages to prevent log bloat and memory overhead from large base64 payloads; second, unifying the error response format between the streaming and non-streaming paths in http_server.py to ensure API consistency.

Important

The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.

Comment thread python/sglang/srt/multimodal/processors/base_processor.py Outdated
Comment thread python/sglang/srt/entrypoints/http_server.py
Large base64-encoded inputs (e.g. high-res images) can produce massive
error messages. Truncate the data representation to 100 characters in
both ValueError and RuntimeError paths.
@merrymercy
merrymercy merged commit 88a7b0f into main Jun 6, 2026
125 of 152 checks passed
@merrymercy
merrymercy deleted the classify-malformed-multimodal-invalid-request branch June 6, 2026 17:19
Chronostasys pushed a commit to MindLab-Research/sglang that referenced this pull request Aug 24, 2026
…#27451)

Co-authored-by: cctry <cctry@meta.com>
Co-authored-by: cctry <cctry@fb.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

run-ci CI: run the baseline test suite on this PR

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant