Skip to content

feat(providers): add Tokenify as an OpenAI-compatible provider - #44163

Open
BeEdward wants to merge 4 commits into
BerriAI:mainfrom
BeEdward:feat/add-tokenify-provider
Open

BeEdward wants to merge 4 commits into
BerriAI:mainfrom
BeEdward:feat/add-tokenify-provider

Conversation

@BeEdward

@BeEdward BeEdward commented Oct 2, 2026

Copy link
Copy Markdown

TLDR

Problem this solves:

  • Tokenify users can only reach it through openai/ plus a manual api_base
  • No cost tracking or Admin UI entry for Tokenify

How it solves it:

User Flow

Before: a developer has to know the base URL and route through the generic OpenAI provider

  1. They call litellm.completion(model="tokenify/deepseek/deepseek-v4-pro", ...)
  2. LiteLLM raises "LLM Provider NOT provided" because tokenify is not a known prefix
  3. They fall back to model="openai/deepseek/deepseek-v4-pro" with api_base="https://api.tokenify.dev/v1"

After: the provider prefix works on its own

  1. They set TOKENIFY_API_KEY and call litellm.completion(model="tokenify/deepseek/deepseek-v4-pro", ...)
  2. The request goes to https://api.tokenify.dev/v1/chat/completions with their key and returns a normal completion
  3. litellm.responses(...) and litellm.anthropic.messages.acreate(...) with the same model reach /v1/responses and /v1/messages
  4. In the Admin UI, "Tokenify" appears in the provider list with API Base and API Key fields

Pre-Submission checklist

  • I have added meaningful tests (tests/unit/llms/openai_like/test_tokenify_provider.py, mirroring the Cortecs tests)
  • The test files covering my change pass locally: pytest tests/unit/llms/openai_like/test_tokenify_provider.py tests/unit/llms/openai_like/test_cortecs_provider.py → 14 passed; ruff check and ruff format --check clean
  • My PR passes all required CI/CD checks
  • My PR's scope is as isolated as possible; it only adds one provider
  • Greptile Confidence Score of at least 4/5

Type

🆕 New Feature

Docs: https://www.tokenify.dev/docs/ — models: GET https://api.tokenify.dev/v1/models. A docs page for docs.litellm.ai can follow in BerriAI/litellm-docs once this lands.

🤖 Generated with Claude Code

Tokenify serves DeepSeek and GLM models on /v1/chat/completions,
/v1/responses and /v1/messages at https://api.tokenify.dev/v1. Registered
through the JSON provider config, following the Cortecs provider.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
@BeEdward
BeEdward requested a review from a team October 2, 2026 09:07
@CLAassistant

CLAassistant commented Oct 2, 2026 •

Copy link
Copy Markdown

CLA assistant check
All committers have signed the CLA.

@greptile-apps

greptile-apps Bot commented Oct 2, 2026 •

Copy link
Copy Markdown
Contributor

RetriggerConfidence Score: 5/5

[Medium risk] Adds a new OpenAI-compatible provider to the routing system.

The PR appears safe to merge; no outstanding findings remain.

Summary

Adds Tokenify as an OpenAI-compatible provider, with model pricing, endpoint declarations, Admin UI fields, and request tests.

  • The latest change makes the five expected model IDs explicit so the tests detect missing price entries.

Reviews (4) · Last reviewed commit: "test(providers): pin the expected Tokeni..."

Comment thread litellm/llms/openai_like/providers.json
Comment thread litellm/proxy/public_endpoints/provider_create_fields.json
@codspeed

codspeed Bot commented Oct 2, 2026 •

Copy link
Copy Markdown
Contributor

Merging this PR will not alter performance

✅ 31 untouched benchmarks


Comparing BeEdward:feat/add-tokenify-provider (f28f328) with main (fe9b6fd)1

Open in CodSpeed

Footnotes

  1. No successful run was found on main (615ed79) during the generation of this report, so fe9b6fd was used instead as the comparison base. There might be some changes unrelated to this pull request in this report. ↩

@codecov

codecov Bot commented Oct 2, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

Adds cost-map entries for the five models Tokenify serves (identical in the
root and backup maps), with capabilities taken from its GET /v1/models, and
tests that the costs match the published per-1M prices and that the Admin UI
placeholder model is priced.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
@BeEdward

BeEdward commented Oct 2, 2026

Copy link
Copy Markdown
Author

@greptileai

Comment thread tests/unit/llms/openai_like/test_tokenify_provider.py Outdated
…or price literals

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
@BeEdward

BeEdward commented Oct 2, 2026

Copy link
Copy Markdown
Author

@greptileai

Comment thread tests/unit/llms/openai_like/test_tokenify_provider.py Outdated
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
@BeEdward

BeEdward commented Oct 2, 2026

Copy link
Copy Markdown
Author

@greptileai

@BeEdward

BeEdward commented Oct 9, 2026

Copy link
Copy Markdown
Author

Hi maintainers — this is ready for review whenever you have a moment: all required checks pass, Greptile is at 5/5, and the CLA is signed. It follows the Cortecs provider (#43872): a JSON openai_like entry, cost-map entries for the five models, and unit tests. Happy to adjust anything, or for you to land it as an internal copy if that is easier. Thank you!

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants