Skip to content

fix(translator): resolve model validation response format for Claude Code CLI - #2949

Closed
ervareza wants to merge 1 commit into
decolua:masterfrom
ervareza:fix/messages-endpoint-format
Closed

ervareza wants to merge 1 commit into
decolua:masterfrom
ervareza:fix/messages-endpoint-format

Conversation

@ervareza

Copy link
Copy Markdown

Problem

When validating custom models in Claude Code CLI (e.g. /model cbai/glm-5.2), Claude Code sends a non-streaming (stream: false) POST request to /v1/messages.

Upstream OpenAI-compatible providers return standard OpenAI JSON ({"choices": [...], "usage": {"prompt_tokens": 751}}). 9router's format resolution defaulted to openai because:

  1. detectFormatByEndpoint was attempting new URL(request.url) on relative endpoints, throwing an unhandled TypeError: Invalid URL.
  2. detectFormat(body) did not inspect the endpoint or anthropic-version header, falling back to openai.

As a result, 9router returned raw OpenAI format JSON without converting it to Anthropic format ({"type": "message", "usage": {"input_tokens": 751}}), causing Claude Code CLI to crash with:
Unable to validate model: undefined is not an object (evaluating 'z.usage.input_tokens')

Fix

  • open-sse/translator/formats.js: Update detectFormatByEndpoint to match /messages endpoints and safely handle null/relative paths.
  • src/sse/handlers/chat.js: Extract endpoint safely from clientRawRequest?.endpoint before trying request?.url to avoid Invalid URL parsing errors.
  • open-sse/handlers/chatCore.js: Set fallback sourceFormat to FORMATS.CLAUDE whenever the endpoint contains /messages.
  • open-sse/services/provider.js: Add support for anthropic-version header in format detector.

Tests

  • Verified via non-streaming POST http://127.0.0.1:20128/v1/messages with cbai/glm-5.2:
    • Returns HTTP 200 OK with type: "message" and usage: { input_tokens, output_tokens }.
  • Verified model validation in Claude Code CLI passes cleanly.

Copilot AI review requested due to automatic review settings July 31, 2026 11:14

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

@ervareza ervareza closed this Aug 3, 2026
diegosouzapw added a commit to cheaper-inference/OmniRoute that referenced this pull request Aug 4, 2026
…9253)

* fix(translator): harden Claude format detection for model validation

Co-authored-by: Ervareza Naurian <rianskp644@gmail.com>
Inspired-by: decolua/9router#2949

* chore(changelog): fragment for #9253

---------

Co-authored-by: diegosouzapw <diegosouzapw@users.noreply.github.com>
Co-authored-by: Ervareza Naurian <rianskp644@gmail.com>
muhamadgalihsaputra pushed a commit to niyatna/NiyatnaRoute that referenced this pull request Sep 27, 2026
…heaper-inference#9253)

* fix(translator): harden Claude format detection for model validation

Co-authored-by: Ervareza Naurian <rianskp644@gmail.com>
Inspired-by: decolua/9router#2949

* chore(changelog): fragment for cheaper-inference#9253

---------

Co-authored-by: diegosouzapw <diegosouzapw@users.noreply.github.com>
Co-authored-by: Ervareza Naurian <rianskp644@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants