fix(sglang): cherry pick conditional disagg bypass httperror fix - #12616
Conversation
Signed-off-by: Karen Chung <karenc@nvidia.com>
![devin-ai-integration[bot]](https://avatars.githubusercontent.com/in/811515?s=60&v=4){kind=small}
| raise HttpError( | ||
| 400, | ||
| f"Detected request annotation {BYPASS_REMOTE_PREFILL_ANNOTATION!r}, but " | ||
| "SGLang backend does not support conditional disaggregation yet. " | ||
| "Use vLLM or TensorRT-LLM for conditional disaggregation." | ||
| "Use vLLM or TensorRT-LLM for conditional disaggregation.", |
There was a problem hiding this comment.
🔍 HttpError propagation path verified
dynamo.llm.HttpError (defined in lib/bindings/python/src/dynamo/llm/exceptions.py) validates code range 0-599 and is duck-typed extracted by the Rust bridge (lib/bindings/python/rust/http.rs:190-201) into http_error::HttpError, so raising with code 400 surfaces a client error rather than a 500. Note the extraction path applies to the HTTP engine bridge; for worker-endpoint requests the exception travels back through the runtime error channel — worth confirming manually that the frontend renders 400 rather than a generic internal error for SGLang decode workers.
Was this helpful? React with 👍 or 👎 to provide feedback.
![dynamo-review-agent[bot]](https://avatars.githubusercontent.com/u/201626793?s=60&v=4){kind=small}
|
🎯 Code Coverage (details) 🔗 Commit SHA: 557c482 | Docs | Datadog PR Page | Give us feedback! |
2 participants
Overview:
cherry-pick for #12578
Details:
Where should the reviewer start?
Related Issues
🔗 This PR is linked to an issue:
🚫 This PR is NOT linked to an issue: