Skip to content

fix(anthropic): remove non-standard 'data: [DONE]' from Anthropic streaming#37510

Merged
DarkLight1337 merged 1 commit intovllm-project:mainfrom
cdpath:fix_anthropic_api
Mar 19, 2026
Merged

fix(anthropic): remove non-standard 'data: [DONE]' from Anthropic streaming#37510
DarkLight1337 merged 1 commit intovllm-project:mainfrom
cdpath:fix_anthropic_api

Conversation

@cdpath
Copy link
Copy Markdown
Contributor

@cdpath cdpath commented Mar 19, 2026

Purpose

Fix protocol leakage in the Anthropic streaming endpoint: remove the non-standard data: [DONE]\n\n marker from all stream termination paths.

The data: [DONE] sentinel is specific to the OpenAI streaming protocol. According to the Anthropic API specification, a stream terminates with an event: message_stop block where the data payload is valid JSON (e.g., {"type": "message_stop"}).

Appending the OpenAI-style [DONE] marker to the Anthropic protocol breaks strict API gateways/middleware (such as LiteLLM) that expect all data: payloads to be JSON objects — json.loads(" [DONE]") evaluates to ['DONE'], causing an AttributeError when calling .get("type").

Test Plan

curl -s http://localhost:8000/v1/messages \
    -H "x-api-key: dummy" \
    -H "Content-Type: application/json" \
    -H "anthropic-version: 2023-06-01" \
    -d '{
      "model": "<model>",
      "messages": [{"role": "user", "content": "Say hi"}],
      "max_tokens": 20,
      "stream": true
    }'

Verify the stream ends with event: message_stop and no trailing data: [DONE].

Test Result

Before: stream ends with event: message_stop followed by data: [DONE]
After: stream ends cleanly with event: message_stop only

Standard Anthropic clients (Claude Code, official SDKs) continue to function correctly, as their state machines close upon receiving message_stop.

…eaming

Signed-off-by: cdpath <cdpath@outlook.com>
@mergify mergify bot added the frontend label Mar 19, 2026
Copy link
Copy Markdown
Contributor

@gemini-code-assist gemini-code-assist bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request correctly addresses a protocol compliance issue in the Anthropic streaming endpoint by removing the non-standard data: [DONE] marker. The change is applied to all stream termination paths, including normal completion and error scenarios, ensuring the endpoint adheres to the Anthropic API specification. This fix is well-targeted and resolves an issue for downstream consumers that expect strict protocol compliance. The changes are correct and improve the API implementation.

@DarkLight1337 DarkLight1337 enabled auto-merge (squash) March 19, 2026 05:32
@github-actions github-actions bot added the ready ONLY add when PR is ready to merge/full CI is needed label Mar 19, 2026
@DarkLight1337 DarkLight1337 merged commit 354cd58 into vllm-project:main Mar 19, 2026
49 checks passed
fxdawnn pushed a commit to fxdawnn/vllm that referenced this pull request Mar 19, 2026
chooper26 pushed a commit to intellistream/vllm-hust that referenced this pull request Mar 21, 2026
SouthWest7 pushed a commit to SouthWest7/vllm that referenced this pull request Mar 27, 2026
khairulkabir1661 pushed a commit to khairulkabir1661/vllm that referenced this pull request Mar 27, 2026
Monishver11 pushed a commit to Monishver11/vllm that referenced this pull request Mar 27, 2026
…eaming (vllm-project#37510)

Signed-off-by: cdpath <cdpath@outlook.com>
Signed-off-by: Monishver Chandrasekaran <monishverchandrasekaran@gmail.com>
JiantaoXu pushed a commit to JiantaoXu/vllm that referenced this pull request Mar 28, 2026
vrdn-23 pushed a commit to vrdn-23/vllm that referenced this pull request Mar 30, 2026
…eaming (vllm-project#37510)

Signed-off-by: cdpath <cdpath@outlook.com>
Signed-off-by: Vinay Damodaran <vrdn@hey.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

frontend ready ONLY add when PR is ready to merge/full CI is needed

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants