Skip to content

[API] Fix DeepSeek V4.1 /v1/responses empty prompt routing - #38985

Merged
hnyls2002 merged 1 commit into
sgl-project:dsv4.1from
hnyls2002:codex/dsv41-responses-prompt-ids
Sep 11, 2026
Merged

hnyls2002 merged 1 commit into
sgl-project:dsv4.1from
hnyls2002:codex/dsv41-responses-prompt-ids

Conversation

@hnyls2002

@hnyls2002 hnyls2002 commented Sep 11, 2026

Copy link
Copy Markdown
Collaborator
  • Fix /v1/responses returning 400: texts cannot be empty and tokenizer must be initialized for text requests on multimodal DeepSeek V4.1 checkpoints: forward the encoder's prompt_ids, matching Chat Completions.
  • Target the dsv4.1 branch from dsv4.1: remaining model and runtime integration #38798; preserve image payloads and encoded image placeholders without requiring --language-model-only or changing PD configuration.
  • Add regression coverage using the V4.1 encoder for text-only and image inputs; both cases reproduced empty prompt routing before the fix.
  • Related: [Responses] Fix empty-prompt routing for token-first chat encoders (kimi_k3, inkling) #35486 fixes the same routing issue on main for kimi_k3 and inkling. The dsv4.1 branch does not yet contain that helper; when integrating it, dsv41 must also be included in spec_renders_prompt_ids.
  • Validation: 45 tests passed across test_serving_responses.py, test_serving_responses_stream.py, and test_encoding_dsv41.py; pre-commit checks passed. Real PD GPU inference has not been run.

CI States

Latest PR Test (Base): ❌ Run #34557724636
Latest PR Test (Extra): ❌ Run #34557724152
Latest PR Test (AMD ROCm 10): ❌ Run #34557724624

@hnyls2002
hnyls2002 merged commit e087e66 into sgl-project:dsv4.1 Sep 11, 2026
81 of 91 checks passed
@qqtang-code

Copy link
Copy Markdown

Cross-branch heads-up for whenever dsv4.1 lands on main.

This fix keys the empty-prompt routing on the encoding spec in serving_responses._make_request. main has since refactored that same block (ad7f57c, #35486) into a shared helper:

# serving_chat.py
def _engine_prompt(self, processed_messages, is_multimodal):
    if is_multimodal and not chat_encoding.spec_renders_prompt_ids(self.chat_encoding_spec):
        return "text", processed_messages.prompt
    ...
# chat_encoding.py
def spec_renders_prompt_ids(spec):
    return spec in ("inkling", "kimi_k3")

That whitelist does not contain "dsv41". If the merge keeps main's helper and drops this guard, DeepSeek-V4.1 goes straight back to the bug this PR fixes: the V4.1 encoder is token-first, so processed_messages.prompt is empty and /v1/responses returns

400 texts cannot be empty and tokenizer must be initialized

Adding "dsv41" to spec_renders_prompt_ids (or keeping this guard) resolves it. Verified on 8× RTX 6000D serving deepseek-ai/DeepSeek-V4.1-Flash: with the guard, plain text / instructions / streaming / function tools all return 200; without it, all five input spellings return 400.

wonderisland pushed a commit to shinemo-ai/sglang that referenced this pull request Sep 11, 2026
trilog-inc pushed a commit to trilog-inc/sglang that referenced this pull request Sep 11, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants