feat(openai): add litellm.acount_tokens() public API + OpenAI token counting support

[Chesars]

Relevant issues

Fixes #22302

Pre-Submission checklist

Please complete all items before asking a LiteLLM maintainer to review your PR

• I have Added testing in the tests/test_litellm/ directory, Adding at least 1 test is a hard requirement - see details
• My PR passes all unit tests on make test-unit
• My PR's scope is as isolated as possible, it only solves 1 specific problem
• I have requested a Greptile review by commenting @greptileai and received a Confidence Score of at least 4/5 before requesting a maintainer review

CI (LiteLLM team)

• Branch creation CI run
  Link:

• CI run for the last commit
  Link:

• Merge / cherry-pick CI run
  Links:

Type

🆕 New Feature

Changes

1. OpenAI Token Counting Provider (litellm/llms/openai/responses/count_tokens/)

Adds OpenAITokenCounter that calls OpenAI's POST /v1/responses/input_tokens endpoint for exact token counts. Follows the same pattern as existing Anthropic, Bedrock, and Gemini token counters.

• transformation.py — Request/response transformation, including chat tools → Responses API tools format conversion
• handler.py — Async HTTP handler using httpx
• token_counter.py — BaseTokenCounter implementation that auto-detects OpenAI provider

2. Public SDK Method (litellm/main.py)

Adds litellm.acount_tokens() — an async public method that:

• Auto-detects provider from model string (e.g., openai/gpt-4o, anthropic/claude-3-5-sonnet)
• Routes to the correct provider's token counting API
• Falls back to local tiktoken when provider API is unavailable
• Returns TokenCountResponse with exact token count

result = await litellm.acount_tokens(
    model="openai/gpt-4o",
    messages=[{"role": "user", "content": "Hello"}],
    tools=[...],
    system="You are helpful",
)
print(result.total_tokens)  # 13

3. Proxy Endpoint (litellm/proxy/response_api_endpoints/endpoints.py)

Adds POST /v1/responses/input_tokens (+ /responses/input_tokens, /openai/v1/responses/input_tokens) that:

• Accepts OpenAI Responses API format (input, instructions, tools)
• Routes through the internal token counter with call_endpoint=True
• Returns {"input_tokens": N}

4. Wiring (litellm/llms/openai/chat/gpt_transformation.py)

Adds get_token_counter() to OpenAIGPTConfig so the existing proxy routing (_get_provider_token_counter) automatically picks up OpenAI.

Provider Support Status

Provider	API	Status
OpenAI	POST /v1/responses/input_tokens	✅ New in this PR
Anthropic	POST /v1/messages/count_tokens	✅ Existing
Bedrock	CountTokens API	✅ Existing
Gemini	countTokens API	✅ Existing
Vertex AI	countTokens API	✅ Existing

Tests

• tests/test_litellm/llms/openai/responses/test_openai_count_tokens_transformation.py — 15 tests for transformation logic
• tests/test_litellm/test_count_tokens_public_api.py — 7 tests for public API routing + fallback
• Verified with real OpenAI API: simple messages, tools, system prompts
• Verified proxy endpoints with live proxy server

[CLAassistant]

All committers have signed the CLA.

[vercel]

The latest updates on your projects. Learn more about Vercel for GitHub.

Project	Deployment	Actions	Updated (UTC)
litellm	Error		Mar 5, 2026 0:53am

[greptile-apps]

Greptile Summary

This PR adds OpenAI token counting support via the Responses API /v1/responses/input_tokens endpoint, a new public litellm.acount_tokens() async SDK method with provider auto-detection and local fallback, and a corresponding proxy endpoint. The implementation follows existing patterns established by the Anthropic token counter.

• New OpenAI token counter (litellm/llms/openai/responses/count_tokens/): Transformation, handler, and BaseTokenCounter implementation that converts chat-format messages to Responses API format, calls OpenAI's endpoint, and returns TokenCountResponse
• Public SDK method (litellm.acount_tokens()): Auto-detects provider from model string, routes to the correct token counting API, falls back to local tiktoken counting on failure or for unsupported providers
• Proxy endpoint (POST /v1/responses/input_tokens): Accepts Responses API format input, converts to chat format for the internal token counter, returns {"input_tokens": N}
• Wiring: OpenAIGPTConfig.get_token_counter() factory method integrates with existing _get_provider_token_counter infrastructure
• Tests: 22 unit tests covering transformation logic, SDK routing/fallback, tools passthrough, and error recovery — all properly mocked with no real network calls
• The author has addressed extensive feedback from a prior review round, including fixing tool_calls mapping, multimodal content extraction, ProxyException handling, strict field preservation, and error handling improvements

Confidence Score: 4/5

• This PR is safe to merge — it adds new functionality without modifying existing behavior, follows established patterns, and includes comprehensive tests
• The implementation closely mirrors the existing Anthropic token counter architecture. All previous review feedback has been addressed. Tests are properly mocked (no real network calls). The new code is additive — no existing functionality is modified. The proxy endpoint follows established error handling patterns. Minor remaining concerns (e.g., no dedicated proxy endpoint tests, redundant ternary in token_counter.py:78) are low-risk.
• litellm/proxy/response_api_endpoints/endpoints.py has the most complex logic (Responses API → chat format conversion) and lacks dedicated test coverage for the proxy path

Important Files Changed

Filename	Overview
litellm/llms/openai/responses/count_tokens/transformation.py	New transformation logic for OpenAI Responses API token counting. Handles chat-to-Responses format conversion including system/developer→instructions extraction, tool_calls→function_call mapping, and multimodal content extraction. Well-structured with good coverage of edge cases.
litellm/llms/openai/responses/count_tokens/handler.py	Async HTTP handler for OpenAI token counting endpoint. Uses shared httpx client, proper error handling with OpenAIError wrapping, and follows the existing Anthropic handler pattern closely.
litellm/llms/openai/responses/count_tokens/token_counter.py	BaseTokenCounter implementation for OpenAI. Handles API key resolution, message conversion, and graceful fallback to None when API is unavailable. Follows existing patterns from Anthropic token counter.
litellm/main.py	Adds acount_tokens() public async API with provider auto-detection, token counting delegation, and local tiktoken fallback. Error handling improved from earlier revisions — now logs and falls back gracefully.
litellm/proxy/response_api_endpoints/endpoints.py	Adds /v1/responses/input_tokens proxy endpoint with Responses API→chat format conversion. Handles function_call and function_call_output items correctly. ProxyException handling follows existing Anthropic endpoint pattern.
litellm/llms/openai/chat/gpt_transformation.py	Minimal change — adds get_token_counter() factory method to OpenAIGPTConfig, wiring OpenAI into the existing provider token counter infrastructure.
tests/test_litellm/llms/openai/responses/test_openai_count_tokens_transformation.py	15 unit tests for transformation logic covering basic requests, instructions, tools, message conversion, validation, and endpoint URL generation. All use mocks — no real network calls.
tests/test_litellm/test_count_tokens_public_api.py	7 tests for the public acount_tokens() API covering routing, fallback, tools passthrough, system messages, and error recovery. All provider API calls are properly mocked.

Sequence Diagram

sequenceDiagram
    participant User
    participant SDK as acount_tokens
    participant Router as ProviderConfigManager
    participant Counter as OpenAITokenCounter
    participant Transform as CountTokensConfig
    participant Handler as CountTokensHandler
    participant OpenAI as OpenAI Endpoint
    participant Fallback as Local Tiktoken

    User->>SDK: model, messages, tools, system
    SDK->>Router: get_provider_model_info(model)
    Router-->>SDK: OpenAIGPTConfig
    SDK->>Counter: count_tokens(messages, tools, system)
    Counter->>Transform: messages_to_responses_input(messages)
    Transform-->>Counter: (input_items, instructions)
    Counter->>Handler: handle_count_tokens_request(input, tools)
    Handler->>OpenAI: POST /responses/input_tokens
    OpenAI-->>Handler: input_tokens count
    Handler-->>Counter: response
    Counter-->>SDK: TokenCountResponse
    SDK-->>User: TokenCountResponse

    Note over SDK,Fallback: On failure or unsupported provider
    SDK->>Fallback: token_counter(model, messages, tools)
    Fallback-->>SDK: count (int)
    SDK-->>User: TokenCountResponse(local_tokenizer)

Loading

_{Last reviewed commit: abc381c}