fix: strip provider_specific_fields from Anthropic count_tokens

[atharvamhaske]

Summary

Claude Code CLI serializes tool_use history with "provider_specific_fields": null. Bifrost's Anthropic /v1/messages/count_tokens path was forwarding that non-spec field to strict Anthropic-compatible backends (notably vLLM), which reject it with Extra inputs are not permitted and break Claude Code sessions after the first tool use. /v1/messages often still succeeded because that path is more lenient or converts the payload differently.

Changes

• Add StripProviderSpecificFieldsFromContentBlocks to remove provider_specific_fields from messages[].content[]
• Call it from BuildAnthropicResponsesRequestBody when IsCountTokens is true (alongside existing fallback_credit_token stripping)
• Enable IsCountTokens: true and pass Model in VLLMProvider.CountTokens so the count_tokens cleanup path actually runs for vLLM
• Add regression test covering Claude Code-shaped tool_use history on the raw-body count_tokens path

Type of change

• Bug fix
• Feature
• Refactor
• Documentation
• Chore/CI

Affected areas

• Core (Go)
• Transports (HTTP)
• Providers/Integrations
• Plugins
• UI (React)
• Docs

How to test

cd core
GOWORK=off go test ./providers/anthropic/ -run 'TestBuildAnthropicResponsesRequestBody_CountTokensStrips' -count=1
GOWORK=off go build ./providers/vllm/

Manual curl against Bifrost → vLLM:

# Should return 200 (not 400 about provider_specific_fields)
curl -s http://localhost:8080/anthropic/v1/messages/count_tokens \
  -H "Content-Type: application/json" \
  -H "x-api-key: $KEY" \
  -H "anthropic-version: 2023-06-01" \
  -d '{
    "model": "vllm/YOUR_MODEL",
    "messages": [
      {"role": "user", "content": "hi"},
      {"role": "assistant", "content": [{"type": "tool_use", "id": "toolu_01", "name": "test", "input": {}, "provider_specific_fields": null}]},
      {"role": "user", "content": [{"type": "tool_result", "tool_use_id": "toolu_01", "content": "done"}]}
    ]
  }'

Screenshots/Recordings

N/A

Breaking changes

• Yes
• No

Related issues

Related to Claude Code + OpenAI-compatible / vLLM Anthropic compatibility (e.g. #2826).
Closes : #5684

Security considerations

None. Only strips a non-spec client field from count_tokens request bodies before upstream forwarding.

Checklist

• I read docs/contributing/README.md and followed the guidelines
• I added/updated tests where appropriate
• I updated documentation where needed
• I verified builds succeed (Go and UI)
• I verified the CI pipeline passes locally if applicable

[coderabbitai]

📝 Walkthrough

Summary by CodeRabbit

• Bug Fixes
  • Improved token-counting requests for Anthropic-compatible providers by removing unsupported provider-specific fields while preserving tool information.
  • Ensured VLLM token-counting requests include the selected model and correct request context.
• Tests
  • Added regression coverage for cleaning provider-specific fields from tool-use content blocks.

Walkthrough

The change sanitizes Anthropic count-token content blocks by removing provider_specific_fields, preserves tool-use metadata, reports cleanup errors, and marks vLLM requests as count-token operations with the requested model.

Changes

Anthropic count-token request handling

Layer / File(s)	Summary
Content-block sanitization core/providers/anthropic/utils.go, core/providers/anthropic/requestbuilder.go	The request builder removes provider_specific_fields from message content blocks and returns contextual marshal errors when cleanup fails.
Count-token wiring and regression coverage core/providers/vllm/vllm.go, core/providers/anthropic/serversidefallback_test.go	The vLLM path passes the requested model and sets IsCountTokens. The regression test verifies that tool-use type and ID remain unchanged after sanitization.

Estimated code review effort: 2 (Simple) | ~10 minutes

Possibly related issues

• [Bug]: Anthropic /v1/messages/count_tokens rejects provider_specific_fields in tool_use blocks (Claude Code compatibility) #5684 — Covers stripping provider_specific_fields from tool-use blocks during Anthropic count-token processing.

Possibly related PRs

• maximhq/bifrost#5311 — Related Anthropic count-token request handling.
• maximhq/bifrost#5367 — Extends the same count-token flow with provider-specific field removal.
• maximhq/bifrost#5563 — Modifies Anthropic count-token handling and provider-specific field filtering.

Suggested reviewers: akshaydeo, tejasghatte, pratham-mishra04

🚥 Pre-merge checks | ✅ 5

✅ Passed checks (5 passed)

Check name	Status	Explanation
Docstring Coverage	✅ Passed	No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Title check	✅ Passed	The title clearly and concisely describes the main fix: removing provider_specific_fields from Anthropic count_tokens requests.
Description check	✅ Passed	The description covers the purpose, changes, testing steps, affected areas, breaking changes, security, and checklist status.

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}