[API] Fix DeepSeek V4.1 /v1/responses empty prompt routing - #38985
Conversation
|
Cross-branch heads-up for whenever This fix keys the empty-prompt routing on the encoding spec in # serving_chat.py
def _engine_prompt(self, processed_messages, is_multimodal):
if is_multimodal and not chat_encoding.spec_renders_prompt_ids(self.chat_encoding_spec):
return "text", processed_messages.prompt
...
# chat_encoding.py
def spec_renders_prompt_ids(spec):
return spec in ("inkling", "kimi_k3")That whitelist does not contain Adding |
…ject#38985) (cherry picked from commit e087e66)
/v1/responsesreturning400: texts cannot be empty and tokenizer must be initializedfor text requests on multimodal DeepSeek V4.1 checkpoints: forward the encoder'sprompt_ids, matching Chat Completions.dsv4.1branch from dsv4.1: remaining model and runtime integration #38798; preserve image payloads and encoded image placeholders without requiring--language-model-onlyor changing PD configuration.mainforkimi_k3andinkling. Thedsv4.1branch does not yet contain that helper; when integrating it,dsv41must also be included inspec_renders_prompt_ids.test_serving_responses.py,test_serving_responses_stream.py, andtest_encoding_dsv41.py; pre-commit checks passed. Real PD GPU inference has not been run.CI States
Latest PR Test (Base): ❌ Run #34557724636
Latest PR Test (Extra): ❌ Run #34557724152
Latest PR Test (AMD ROCm 10): ❌ Run #34557724624