Skip to content

docs(api): document Messages count_tokens and forwarded Anthropic headers - #51

Merged
slin1237 merged 2 commits into
mainfrom
docs/messages-api-v1.11
Sep 24, 2026
Merged

slin1237 merged 2 commits into
mainfrom
docs/messages-api-v1.11

Conversation

@slin1237

Copy link
Copy Markdown
Member

SMG now serves POST /v1/messages/count_tokens (smg-project/smg#2638), and it forwards the anthropic-version and anthropic-beta headers to engines. Claude Code and the Anthropic SDKs depend on both. Messages also routes through HTTP PD dual dispatch (#2250).

The existing page was thin and partly wrong:

  • It said tool use works "via MCP integration". Custom tools work on every path.
  • It left out HTTP workers and HTTP PD.
  • It showed only an x-api-key example, but the gateway's own auth never reads that header.
  • It listed no streaming events.

Changes

reference/api/messages.md is rewritten for v1.11:

  • Endpoints and backend support: an endpoints table, plus a matrix of what /v1/messages and count_tokens do on gRPC, ZMQ, HTTP, HTTP PD, and the Anthropic backend, and under IGW.
  • Authentication: the client→SMG, SMG→worker, and SMG→Anthropic hops, x-api-key handling, and the CORS header list.
  • Request body: the full field table with validation rules and the accepted content blocks.
  • Examples: self-hosted and Anthropic tabs, a response example, and tool use.
  • Streaming: the event list, with an exact SSE example from the gRPC path.
  • Count Tokens: request, response, and per-backend behavior. The gRPC and Anthropic backends return 501, and requests with no router return 404 NO_ROUTER (#2659).
  • Request Headers:
    • X-SMG-MCP: enabled turns on the SMG-run MCP tool loop on the Anthropic backend.
    • The two Anthropic headers are forwarded.
    • The full forward allowlist is listed.
  • gRPC and ZMQ translation: tool handling, tool_choice, thinking, per-engine sampling fields, toolu_ tool IDs (#2111), integer cache-token counters (#2269), and 400 context_length_exceeded (#2618).
  • MCP tools: the SMG-run loop versus passthrough, with a link to the MCP concept page.
  • Errors: the real error shapes.

Verification

This is one of a series of docs PRs that bring the site up to smg v1.11.0, one per area.

🤖 Generated with Claude Code

…ders

Rewrite the Messages API reference for v1.11.0: /v1/messages/count_tokens
and its per-backend behavior (smg-project/smg#2638, smg-project/smg#2659),
request headers (forward allowlist, X-SMG-MCP), and HTTP PD dual dispatch
(smg-project/smg#2250). Document the gRPC translation (toolu_ tool ids
smg-project/smg#2111, integer cache counters smg-project/smg#2269,
context_length_exceeded smg-project/smg#2618), SMG-run MCP, and error shapes.

Signed-off-by: Simo Lin <25425177+slin1237@users.noreply.github.com>
@coderabbitai

coderabbitai Bot commented Sep 24, 2026 •

Copy link
Copy Markdown

Warning

Review limit reached

Next included review available in 23 minutes.

Check out review usage here.

View limit details

Limit details: You’ve used all 4 included reviews currently available. Your 65 included PR review attempts over the past 7 days set your current allowance at 4 reviews per hour.

Your organization has reached its usage spending cap. Adjust your spending cap in the billing tab.

Learn how review limits work.

Review configuration:

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Team

Run ID: 5fc2e35c-71a3-467a-968a-06d5c2602bcc

📥 Commits

Reviewing files that changed from the base of the PR and between 7362b40 and 0ea0ff4.

📒 Files selected for processing (1)
  • src/lib/content/reference/api/messages.md

Comment @coderabbitai help to get the list of available commands.

- Label --tenant-api-key as a Rust binary flag; pip smg launch lacks it.
- stop_sequences: sent as strings only to vLLM, TensorRT-LLM, and
  TokenSpeed gRPC workers; SGLang gRPC and ZMQ get SMG-side matching
  plus single-token stop IDs.
- tool_choice tool passes only the named tool to the chat template.
- Drop the smg-project/smg#2637 citation: it changed f32 fields only,
  and Messages has none.
- Anthropic backend example: use smg launch, not bare smg.
- MCP loop: a stream ends after the 10th round without another model
  call; non-streaming fails if the 11th reply still calls tools.
- MCP approval: the config policy is not applied, so the built-in
  default allows every tool.
- Errors: the MCP stream starts before the first upstream call, so its
  upstream errors arrive as error events; list the plain-text 501s and
  the count_tokens 400/415/422 body rejections.

Signed-off-by: Simo Lin <25425177+slin1237@users.noreply.github.com>
@slin1237
slin1237 merged commit 05cde49 into main Sep 24, 2026
8 checks passed
@slin1237
slin1237 deleted the docs/messages-api-v1.11 branch September 24, 2026 17:03
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant