fix(frontend): return 400 instead of 500 when chat template rendering fails

[KrishnanPrash]

Summary

When a chat template refuses to render a request (for example via raise_exception), the frontend returns HTTP 500 "Failed to generate completions" and drops the template's own error message. The failure is caused by the request, so it should be a client error.

This PR classifies every chat template render failure as HTTP 400 and forwards the template's message to the client. vLLM does the same for template render failures. The fix is @rmccorm4's, taken over from rmccormick/template-error-400: OpenAIPreprocessor::map_prompt_render_error now maps all render failures to ErrorType::InvalidArgument, which the HTTP layer already turns into 400.

Before / after (model whose template rejects a history with no user message):

POST /v1/chat/completions
{"model": "<model>", "max_tokens": 1,
 "messages": [{"role": "assistant", "content": "prefill"}]}

Before:

HTTP/1.1 500 Internal Server Error
{"message":"Failed to generate completions","type":"Internal Server Error","code":500}

After:

HTTP/1.1 400 Bad Request
{"message":"invalid operation: No user query found in messages. (in default:1)","type":"Bad Request","code":400}

Notes:

• stream: true behaves the same: a JSON 400, not an SSE stream. Rendering fails before the response stream opens.
• Requests the template accepts are unchanged. An assistant-only history against a template that renders it still returns 200. The earlier revision of this PR rejected user-less requests at the frontend; that was dropped because vLLM and SGLang both accept such requests when the model's template does. Whether a history is servable is the template's decision; the frontend's job is to report it with the right status code.
• The fix covers /v1/chat/completions, /v1/responses, and /v1/messages, which all use the same renderer.
• Same bug class as [BUG]: /v1/messages returns 500 for Claude Code ≥2.1.154 — mid-conversation role:"system" turns fail chat-template rendering #11762 (mid-conversation role: "system" turns make Qwen and Llama templates raise, surfacing as 500).
• Render failures are logged at debug level, so a server-side template misconfiguration that fails at render time stays diagnosable.

Validation

cargo test -p dynamo-llm --test chat_template_render_errors
cargo test -p dynamo-llm --lib
cargo fmt --all --check && cargo clippy -p dynamo-llm --all-targets

• New end-to-end test (lib/llm/tests/chat_template_render_errors.rs): drives real HTTP through a real preprocessor and template. A raising template returns 400 with a JSON body for both stream: false and stream: true; an accepting template returns 200 for the same assistant-only request.
• Unit tests pin the error classification directly.
• Reverting the fix makes the e2e test fail with left: 500, right: 400 on both streaming modes, so the test covers the regression, not the implementation.

[devin-ai-integration]

Devin Review found 1 potential issue.

[coderabbitai]

Walkthrough

The OpenAI chat-completions validator now requires at least one user message. Unit and HTTP tests cover non-user histories, streaming modes, status codes, content type, and error payloads.

Changes

Chat completion validation

Layer / File(s)	Summary
User-message validation rule lib/llm/src/http/service/openai.rs	The validator rejects non-empty message arrays without a user role, while accepting arrays containing user messages. Unit tests cover the supported role combinations.
HTTP validation coverage lib/llm/tests/http-service.rs	Integration tests verify assistant-only requests fail for both streaming modes with HTTP 400 and the expected JSON error fields.

Estimated code review effort: 3 (Moderate) | ~20 minutes

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name	Status	Explanation	Resolution
Description check	⚠️ Warning	The description is detailed, but it misses required template sections like Overview, Details, Where should reviewer start?, and Related Issues.	Add the missing template headings and include the required issue reference, or explicitly mark that no related issue exists.

✅ Passed checks (4 passed)

Check name	Status	Explanation
Docstring Coverage	✅ Passed	Docstring coverage is 100.00% which is sufficient. The required threshold is 80.00%.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Title check	✅ Passed	The title matches the main HTTP status change, though it describes the cause differently than the implemented validation fix.

---

_{Comment @coderabbitai help to get the list of available commands.}

[datadog-official]

🎯 Code Coverage (details)
• Patch Coverage: 100.00%
• Overall Coverage: 45.51% (+3.39%)

_{This comment will be updated automatically if new data arrives.
🔗 Commit SHA: b642340 | Docs | Datadog PR Page | Give us feedback!}

[rmccorm4]

Native vLLM 0.26.0 does not reject the assistant-only request in this PR. I served Qwen/Qwen3-0.6B directly and received HTTP 200 with generated output. The streaming form also returned HTTP 200, normal SSE deltas, and [DONE].

Example vLLM 0.26.0 serve command

CUDA_VISIBLE_DEVICES=0 vllm serve Qwen/Qwen3-0.6B   --host 127.0.0.1   --port 18105   --dtype half   --max-model-len 4096   --gpu-memory-utilization 0.35   --enforce-eager

Request:

curl -i   -X POST http://127.0.0.1:18105/v1/chat/completions   -H 'Content-Type: application/json'   --data-binary '{
    "model": "Qwen/Qwen3-0.6B",
    "max_tokens": 32,
    "messages": [
      {
        "role": "assistant",
        "content": "I am an assistant."
      }
    ]
  }'

Response (abridged):

HTTP/1.1 200 OK
content-type: application/json

{
  "model": "Qwen/Qwen3-0.6B",
  "choices": [
    {
      "message": {
        "role": "assistant",
        "content": "<think>\nOkay, the user says, \"I am an assistant.\" ..."
      },
      "finish_reason": "length"
    }
  ],
  "system_fingerprint": "vllm-0.26.0-nohash",
  "usage": {
    "prompt_tokens": 13,
    "completion_tokens": 32,
    "total_tokens": 45
  }
}

This means the proposed user-role validation is stricter than native vLLM 0.26.0 for this model/template. It would be a Dynamo API policy rather than vLLM parity.

[rmccorm4]

Native SGLang does not reject the default assistant-only request in this PR. I served Qwen/Qwen3-0.6B directly using lmsysorg/sglang:nightly-dev-cu13-20260729-16a52bff, which reports SGLang package version 0.0.0.dev1+gcb12a1547, and received HTTP 200 with generated output.

Example SGLang serve command

docker run --rm   --name sglang-qwen3   --gpus 'device=0'   --shm-size 8g   -p 127.0.0.1:18106:30000   -v "$HOME/.cache/huggingface:/root/.cache/huggingface"   lmsysorg/sglang:nightly-dev-cu13-20260729-16a52bff   sglang serve   --model-path Qwen/Qwen3-0.6B   --host 0.0.0.0   --port 30000   --dtype float16   --context-length 4096   --mem-fraction-static 0.35   --cuda-graph-backend-decode disabled   --cuda-graph-backend-prefill disabled   --disable-radix-cache

Request:

curl -i   -X POST http://127.0.0.1:18106/v1/chat/completions   -H 'Content-Type: application/json'   --data-binary '{
    "model": "Qwen/Qwen3-0.6B",
    "max_tokens": 32,
    "messages": [
      {
        "role": "assistant",
        "content": "I am an assistant."
      }
    ]
  }'

Response (abridged):

HTTP/1.1 200 OK
content-type: application/json

{
  "model": "Qwen/Qwen3-0.6B",
  "choices": [
    {
      "message": {
        "role": "assistant",
        "content": "<think>\nOkay, the user said, \"I am an assistant.\" Let me think about how to respond. ..."
      },
      "finish_reason": "length"
    }
  ],
  "usage": {
    "prompt_tokens": 13,
    "completion_tokens": 32,
    "total_tokens": 45,
    "reasoning_tokens": 0
  },
  "metadata": {
    "weight_version": "default"
  }
}

This is SGLang's default behavior with continue_final_message omitted. The proposed user-role validation is therefore also stricter than native SGLang's default path for this model/template.