feat(providers): add AI Token King (aitokenking) OpenAI-compatible provider with pricing by firekou · Pull Request #39732 · BerriAI/litellm

firekou · 2026-09-04T11:06:28Z

TLDR

Problem this solves:

AI Token King gateway users get response_cost = 0.0 from Router (unmapped models)
Only workaround (openai/<model> pricing) overwrites 13 real vendor price entries

How it solves it:

Register aitokenking as a JSON-configured OpenAI-compatible provider
Ship 48 chat-model prices under aitokenking/*, no vendor entry touched

User Flow

Before: a developer routing through the gateway sees zero spend on every call

They configure a Router deployment with model: openai/qwen3.7-max, api_base: https://api.aitokenking.com.tw/api/v1 and their gateway key
They call router.completion(...); the response comes back fine with real token usage
response_cost is 0.0, so their spend dashboard shows $0 even though the gateway billed them
If they try model: aitokenking/qwen3.7-max instead, LiteLLM rejects it with LLM Provider NOT provided

After: the same developer gets real spend with a one-line model string

They configure the deployment with model: aitokenking/qwen3.7-max and their gateway key; no api_base needed
The request goes to https://api.aitokenking.com.tw/api/v1/chat/completions with the aitokenking/ prefix stripped and the key in Authorization
response_cost is computed from aitokenking/qwen3.7-max in the price map (100k in + 100k out → $1.00)
Prices for gpt-5.5, claude-sonnet-5 etc. used elsewhere in the same process are unchanged

Relevant issues

None filed; this PR is the report.

Notes for reviewers

Follows the JSON provider path (litellm/llms/openai_like/README.md): no Python logic added. URL autodetection reuses the existing JSONProviderRegistry.get_by_base_url fallback.
Price entries carry only what the gateway publishes (input/output per-token cost, max_input_tokens, source). max_output_tokens, cache tiers and capability flags are omitted on purpose rather than guessed. The gateway's 61 image/video models are excluded because it publishes no per-token price for them; an entry with cost 0 would read as "free".
Prices measured 2026-08-23 from the gateway's read-only /models endpoint; unit (USD per 1M tokens) cross-checked against Anthropic's public list prices for two models.
Disclosure: I operate this gateway. The change is scoped to the provider namespace and does not alter any other provider's behaviour or pricing.

Pre-Submission checklist

I have added meaningful tests
The test files covering my change pass locally: tests/test_litellm/llms/openai_like/test_aitokenking_provider.py (10 passed), tests/test_litellm/llms/openai_like/test_pinstripes_provider.py, tests/test_litellm/test_model_prices_schema.py (38 passed total); tests/code_coverage_tests/check_provider_folders_documented.py passes
My PR passes all required CI/CD checks
My PR's scope is as isolated as possible; it only solves 1 specific problem
I have received a Greptile Confidence Score of at least 4/5

Screenshots / Proof of Fix

Opened as a draft because the run below uses a local OpenAI-compatible mock server (fixed 100k/100k usage) rather than the live gateway. A live-gateway run will be added before marking ready for review.

Shared setup: a local mock at http://127.0.0.1:<port>/v1 that answers /chat/completions with usage: {prompt_tokens: 100000, completion_tokens: 100000}; LITELLM_LOCAL_MODEL_COST_MAP=True so the price map is read from the checkout.

Before (`4990f06`)

litellm.completion(model="aitokenking/qwen3.7-max", api_base=<mock>, api_key="test")
→ BadRequestError: LLM Provider NOT provided
Router(model_list=[{model: "openai/qwen3.7-max", api_base: <mock>}]).completion(...) (today's workaround)
→ 200, response_cost = 0.0
Router(model_list=[{model: "aitokenking/qwen3.7-max", api_base: <mock>}]).completion(...)
→ BadRequestError: LLM Provider NOT provided

After (`cc5c94f`)

litellm.completion(model="aitokenking/qwen3.7-max", api_base=<mock>, api_key="test")
→ 200, model sent to gateway: qwen3.7-max, response_cost = 1.0
Router(model_list=[{model: "openai/qwen3.7-max", api_base: <mock>}]).completion(...) (unchanged, still the old behaviour)
→ 200, response_cost = 0.0
Router(model_list=[{model: "aitokenking/qwen3.7-max", api_base: <mock>}]).completion(...)
→ 200, model sent to gateway: qwen3.7-max, response_cost = 1.0 (100k in + 100k out @ $2.5/$7.5 per 1M)

Type

🆕 New Feature

Caveats (if any)

Low

Only /v1/chat/completions is declared; the gateway's other endpoints are not claimed until verified
Docs page docs/providers/aitokenking referenced in provider_endpoints_support.json still needs to be added in the docs repo
Prices are a snapshot (2026-08-23); the gateway's /models endpoint is the source to refresh from

Final Attestation

The tests check the right things, including the edge cases, and regressions in the respective real-world customer use-cases are not possible after this PR

🤖 Generated with Claude Code

…ovider with pricing AI Token King (api.aitokenking.com.tw) is an OpenAI-compatible LLM gateway that resells 48 chat models from Anthropic, OpenAI, Google, Qwen, DeepSeek, Zhipu, Moonshot, MiniMax and ByteDance under one API key. Users could already point LiteLLM at it via `openai/<model>` + `api_base`, but with two problems this PR fixes: 1. Cost tracking silently reported 0.0. Router registers every deployment with `register_model`, so an unmapped gateway model gets a zero-cost entry and `response_cost` reads as "free" instead of "unknown". 2. The only workaround — registering the gateway's prices under `openai/<model>` — shadows the vendor entries already in the map (13 of the 48 ids collide, e.g. `gpt-5.5`, `claude-sonnet-5`), changing the cost reported for real OpenAI/Anthropic calls in the same process. Adding `aitokenking` as a JSON-configured provider gives the gateway its own namespace, so `aitokenking/qwen3.7-max` resolves the base URL, strips the prefix before sending, and prices from `aitokenking/*` without touching any vendor entry. Changes: - providers.json: `aitokenking` (chat completions only; base URL overridable via AITOKENKING_API_BASE) - LlmProviders enum, openai_compatible_providers, openai_compatible_endpoints - provider_endpoints_support.json entry - model_prices_and_context_window.json (+ backup): 48 chat models keyed `aitokenking/<id>` — input/output cost and max_input_tokens only. Fields the gateway does not publish (max_output_tokens, cache tiers, capability flags) are deliberately omitted rather than guessed. The gateway's 61 image/video models are excluded because it publishes no per-token price for them; an entry with cost 0 would read as free. - tests: provider resolution, URL autodetection, Router config, price-map shape, no vendor-entry shadowing, completion_cost resolves to a non-zero value for a prefixed model. Prices measured 2026-08-23 from the gateway's read-only /models endpoint; the unit (USD per 1M tokens) was cross-checked against two vendors' public list prices. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01DBBj5ym47x1LmSxurSxbKH

CLAassistant · 2026-09-04T11:06:45Z

All committers have signed the CLA.

codspeed-hq · 2026-09-04T11:08:55Z

Merging this PR will not alter performance

✅ 31 untouched benchmarks

_{Comparing firekou:feat/aitokenking-provider (663a64a) with litellm_internal_staging (c8635ec)}

greptile-apps · 2026-09-04T11:08:58Z

Greptile Summary

This PR registers AI Token King as a declarative OpenAI-compatible chat provider and adds provider-scoped pricing for 48 models

Adds provider identity, base-URL detection, credential environment variables, and chat parameter mapping
Adds synchronized primary and backup pricing entries without replacing existing vendor prices
Declares chat-completions endpoint support
Adds registration and pricing tests, but does not functionally exercise completion or Router cost tracking

Confidence Score: 4/5

The provider wiring appears correct, but the repository’s explicit comment and meaningful-test requirements must be satisfied before merging

No production behavior defect was established, but the new tests do not exercise the advertised completion and Router response-cost flow, and several added comments violate repository guidance

Files Needing Attention: litellm/constants.py; tests/test_litellm/llms/openai_like/test_aitokenking_provider.py

Important Files Changed

Filename	Overview
litellm/llms/openai_like/providers.json	Adds a consistent declarative chat-provider configuration with default URL, credential variables, and token-limit parameter mapping
litellm/constants.py	Registers the provider slug and API base correctly, but adds a redundant source-code comment prohibited by repository guidance
litellm/types/utils.py	Adds the provider to the central LlmProviders enum
model_prices_and_context_window.json	Adds 48 provider-prefixed chat pricing records with positive input and output costs
litellm/model_prices_and_context_window_backup.json	Keeps the backup model-price catalog synchronized with the primary catalog
provider_endpoints_support.json	Declares chat-completions as the provider’s only supported endpoint family
tests/test_litellm/llms/openai_like/test_aitokenking_provider.py	Covers registration and isolated pricing arithmetic but omits the required functional completion and Router accounting regression path

_{Reviews (1): Last reviewed commit: "feat(providers): add AI Token King (aito..." | Re-trigger Greptile}

greptile-apps · 2026-09-04T11:09:01Z

    "docker_model_runner",
    "ragflow",
    "pinstripes",  # Pinstripes - JSON-configured provider
+    "aitokenking",  # AI Token King - JSON-configured provider


Redundant comments added

These comments restate straightforward code, violating the directive to reserve comments for complex logic, tooling, or follow-up work. Remove before merging

Context Used: CLAUDE.md (source)

Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!

greptile-apps · 2026-09-04T11:09:01Z

+    def test_aitokenking_router_config(self):
+        from litellm import Router
+
+        router = Router(
+            model_list=[
+                {
+                    "model_name": "atk-chat",
+                    "litellm_params": {
+                        "model": "aitokenking/qwen3.7-max",
+                        "api_key": "test-key",
+                    },
+                }
+            ]
+        )
+
+        assert len(router.model_list) == 1
+        assert router.model_list[0]["model_name"] == "atk-chat"


Functional coverage missing

This only checks stored Router configuration, violating the requirement for functional tests. Exercise request dispatch and response-cost integration before merging

Context Used: CLAUDE.md (source)

Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!

veria-ai · 2026-09-04T11:10:52Z

+  },
+  "aitokenking": {
+    "base_url": "https://api.aitokenking.com.tw/api/v1",
+    "api_key_env": "AITOKENKING_API_KEY",


Medium: Provider key can be sent to a caller-controlled host

The generic JSON-provider resolver accepts the request's api_base but still falls back to this environment key when api_key is omitted. An authenticated caller can submit model: "aitokenking/..." with api_base: "https://attacker.example" and capture AITOKENKING_API_KEY from the resulting Authorization header. Require an explicit caller-supplied key whenever the base URL differs from the configured provider host, or reject such overrides before resolving the environment credential.

veria-ai · 2026-09-04T11:10:58Z

PR overview

This pull request adds AI Token King as an OpenAI-compatible provider, including its provider configuration and pricing information.

One security issue remains open: an authenticated caller can override the provider base URL and cause the configured AI Token King API key to be sent to a caller-controlled host. This creates a direct credential-exfiltration path when the environment credential is configured, and no issues have yet been addressed.

Open issues (1)

Medium: Provider key can be sent to a caller-controlled host — litellm/llms/openai_like/providers.json:206

Fixed/addressed: 0 · PR risk: 7/10

codecov · 2026-09-04T11:12:48Z

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

Review feedback: the comment restated the code. The neighbouring JSON-configured entries carry the same trailing comment, but the repo guideline is to reserve comments for non-obvious logic, so the new line drops it rather than adding one more instance.

… config Review feedback: the Router test only asserted the stored model list, which proves nothing about what happens when a request is actually routed. Replaces it with two functional tests through Router.acompletion, following the pattern already used by the neighbouring cognition provider tests: - the routed request resolves to the gateway base URL and the response model is the bare id, so the `aitokenking/` prefix selects the provider without reaching the wire; - the routed response is costed from the aitokenking price-map entry and is non-zero. This is the regression the price map exists to prevent: Router registers every deployment, so an unmapped gateway model reports 0.0, which reads as "free" rather than "unknown". Verified the cost assertion actually catches that regression: with the `aitokenking/qwen3.7-max` entry temporarily renamed the test fails, and it passes again once restored.

firekou requested a review from mateo-berri as a code owner September 4, 2026 11:06

greptile-apps Bot reviewed Sep 4, 2026

View reviewed changes

veria-ai Bot reviewed Sep 4, 2026

View reviewed changes

firekou added 2 commits September 4, 2026 11:30

devin-ai-integration Bot mentioned this pull request Sep 4, 2026

fix(model_prices): verified registry audit, Databricks Sep-2026 catalog, realtime image pricing, deprecation dates #39388

Merged

7 tasks

Navigation Menu

Uh oh!

Uh oh!

feat(providers): add AI Token King (aitokenking) OpenAI-compatible provider with pricing - #39732

feat(providers): add AI Token King (aitokenking) OpenAI-compatible provider with pricing#39732
firekou wants to merge 3 commits into
BerriAI:litellm_internal_stagingfrom
firekou:feat/aitokenking-provider

firekou commented Sep 4, 2026

Uh oh!

CLAassistant commented Sep 4, 2026 •

edited

Loading

Uh oh!

codspeed-hq Bot commented Sep 4, 2026 •

edited

Loading

Uh oh!

greptile-apps Bot commented Sep 4, 2026

Important Files Changed

Uh oh!

greptile-apps Bot Sep 4, 2026

Uh oh!

greptile-apps Bot Sep 4, 2026

Uh oh!

veria-ai Bot Sep 4, 2026

Uh oh!

veria-ai Bot commented Sep 4, 2026

Uh oh!

codecov Bot commented Sep 4, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Uh oh!

Uh oh!

Conversation

firekou commented Sep 4, 2026

TLDR

User Flow

Relevant issues

Notes for reviewers

Pre-Submission checklist

Screenshots / Proof of Fix

Before (4990f06)

After (cc5c94f)

Type

Caveats (if any)

Low

Final Attestation

Uh oh!

CLAassistant commented Sep 4, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

codspeed-hq Bot commented Sep 4, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Merging this PR will not alter performance

Uh oh!

greptile-apps Bot commented Sep 4, 2026

Greptile Summary

Confidence Score: 4/5

Important Files Changed

Uh oh!

greptile-apps Bot Sep 4, 2026

Choose a reason for hiding this comment

Uh oh!

greptile-apps Bot Sep 4, 2026

Choose a reason for hiding this comment

Uh oh!

veria-ai Bot Sep 4, 2026

Choose a reason for hiding this comment

Uh oh!

veria-ai Bot commented Sep 4, 2026

PR overview

Open issues (1)

Uh oh!

codecov Bot commented Sep 4, 2026

Codecov Report

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Before (`4990f06`)

After (`cc5c94f`)

CLAassistant commented Sep 4, 2026 •

edited

Loading

codspeed-hq Bot commented Sep 4, 2026 •

edited

Loading