Skip to content

feat(session): serve Anthropic Messages through SessionCore - #2358

Merged
guapisolo merged 1 commit into
mainfrom
jiajun/anthropic-session-api
Aug 30, 2026
Merged

guapisolo merged 1 commit into
mainfrom
jiajun/anthropic-session-api

Conversation

@guapisolo

@guapisolo guapisolo commented Aug 10, 2026

Copy link
Copy Markdown
Collaborator

ci-sglang-pr: #35127

Summary

Serve Anthropic Messages through Miles sessions while preserving canonical OpenAI records.

Motivation

Claude Code uses /v1/messages, while Miles session/TITO previously handled only /v1/chat/completions. Add a strict Anthropic HTTP adapter that reuses the existing SessionCore, retains canonical OpenAI records and response identity, and returns Anthropic JSON or eager fake SSE without moving session lifecycle or TITO ownership.

Usage

curl -X POST "$SESSION_SERVER/sessions/$SESSION_ID/v1/messages" \
  -H 'content-type: application/json' \
  -d '{"model":"m","max_tokens":512,"messages":[{"role":"user","content":"hi"}]}'

Design Notes

  • Mental model: The route validates an Anthropic request, converts it to OpenAI form, delegates state and backend work to the existing SessionCore, stores the canonical record, then converts the committed response back to Anthropic JSON or eager fake SSE.
  • Adapter boundary: anthropic_adapter.py owns parsing, fixed-feature policy, reasoning replay, and wire framing.
  • Route boundary: sessions.py keeps route ordering, lifecycle, and core calls.
  • Conversion ownership: SGLang PR #35127 at 1ed0c902c28ba0c5bf3c79515c940ad8c759e100 owns Anthropic/OpenAI conversion.
  • Miles policy: The adapter preserves response IDs and rejects unsupported error tool results before core mutation.

Verification

  • tests/fast/router/test_session_anthropic.py and tests/fast/utils/test_utils/test_mock_sglang_server.py on miles-h200-dev-2: 68 passed, 1 skipped on source 46b3db8e.
  • Candidate 9d5232b3: range-diff equivalent to the tested source; upstream changed none of the five PR files.
  • git diff --check c7d81d47...9d5232b3: passed.

Review Focus

  • anthropic_adapter.py: policy and reasoning-history conversion boundaries.
  • sessions.py: route ordering, commit semantics, and response-ID identity.

Route Anthropic Messages through the existing Miles session core while
keeping canonical records OpenAI-shaped. Preserve request policy,
reasoning replay, response identity, errors, and eager fake SSE at the
HTTP adapter boundary.

Keep conversion ownership in the paired SGLang runtime.
@guapisolo guapisolo changed the title feat(session): serve anthropic messages via sglang-derived codec feat(session): serve Anthropic Messages through SessionCore Aug 22, 2026
@guapisolo
guapisolo force-pushed the jiajun/anthropic-session-api branch from 011a039 to 46b3db8 Compare August 22, 2026 06:50
@guapisolo
guapisolo force-pushed the jiajun/anthropic-session-api branch from 46b3db8 to 9d5232b Compare August 29, 2026 02:05
@guapisolo
guapisolo merged commit 1156382 into main Aug 30, 2026
24 checks passed
@guapisolo
guapisolo deleted the jiajun/anthropic-session-api branch August 30, 2026 01:34
Shi-Dong added a commit that referenced this pull request Aug 30, 2026
Shi-Dong added a commit that referenced this pull request Aug 30, 2026
Shi-Dong added a commit that referenced this pull request Aug 31, 2026
Shi-Dong added a commit that referenced this pull request Aug 31, 2026
Shi-Dong added a commit that referenced this pull request Sep 1, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants