Repository navigation
docs(api): document Messages count_tokens and forwarded Anthropic headers - #51
Merged
Merged
Conversation
…ders Rewrite the Messages API reference for v1.11.0: /v1/messages/count_tokens and its per-backend behavior (smg-project/smg#2638, smg-project/smg#2659), request headers (forward allowlist, X-SMG-MCP), and HTTP PD dual dispatch (smg-project/smg#2250). Document the gRPC translation (toolu_ tool ids smg-project/smg#2111, integer cache counters smg-project/smg#2269, context_length_exceeded smg-project/smg#2618), SMG-run MCP, and error shapes. Signed-off-by: Simo Lin <25425177+slin1237@users.noreply.github.com>
|
Warning Review limit reachedNext included review available in 23 minutes. View limit detailsLimit details: You’ve used all 4 included reviews currently available. Your 65 included PR review attempts over the past 7 days set your current allowance at 4 reviews per hour. Your organization has reached its usage spending cap. Adjust your spending cap in the billing tab. Review configuration: ⚙️ Run configurationConfiguration used: Organization UI Review profile: CHILL Plan: Team Run ID: 📒 Files selected for processing (1)
Comment |
- Label --tenant-api-key as a Rust binary flag; pip smg launch lacks it. - stop_sequences: sent as strings only to vLLM, TensorRT-LLM, and TokenSpeed gRPC workers; SGLang gRPC and ZMQ get SMG-side matching plus single-token stop IDs. - tool_choice tool passes only the named tool to the chat template. - Drop the smg-project/smg#2637 citation: it changed f32 fields only, and Messages has none. - Anthropic backend example: use smg launch, not bare smg. - MCP loop: a stream ends after the 10th round without another model call; non-streaming fails if the 11th reply still calls tools. - MCP approval: the config policy is not applied, so the built-in default allows every tool. - Errors: the MCP stream starts before the first upstream call, so its upstream errors arrive as error events; list the plain-text 501s and the count_tokens 400/415/422 body rejections. Signed-off-by: Simo Lin <25425177+slin1237@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
SMG now serves
POST /v1/messages/count_tokens(smg-project/smg#2638), and it forwards theanthropic-versionandanthropic-betaheaders to engines. Claude Code and the Anthropic SDKs depend on both. Messages also routes through HTTP PD dual dispatch (#2250).The existing page was thin and partly wrong:
x-api-keyexample, but the gateway's own auth never reads that header.Changes
reference/api/messages.mdis rewritten for v1.11:/v1/messagesandcount_tokensdo on gRPC, ZMQ, HTTP, HTTP PD, and the Anthropic backend, and under IGW.x-api-keyhandling, and the CORS header list.501, and requests with no router return404 NO_ROUTER(#2659).X-SMG-MCP: enabledturns on the SMG-run MCP tool loop on the Anthropic backend.tool_choice,thinking, per-engine sampling fields,toolu_tool IDs (#2111), integer cache-token counters (#2269), and400 context_length_exceeded(#2618).Verification
v1.11.0(3be823a7):crates/protocols/src/messages.rs,routers/{http/router,http/pd_router,gateway,mod}.rs,crates/external_router/src/{header_utils,anthropic/*}.rs, the gRPC messages pipeline,middleware/auth.rs, andserver.rs.pnpm lint,pnpm check(0 errors), andpnpm buildpass.zmq-workers.md(docs: add a guide for direct ZMQ engine connections #45),multimodal.md(docs: add a multimodal pipeline concept page #44), andrequest-streaming.md(docs: document request body streaming and upstream HTTP/2 #47).wrangler pages dev.This is one of a series of docs PRs that bring the site up to smg v1.11.0, one per area.
🤖 Generated with Claude Code