Repository navigation
Conversation
Add llmgateway.io as a built-in OpenAI-completions provider, mirroring the OpenRouter integration: - catalogue fetcher sources https://api.llmgateway.io/v1/models, keeping models with at least one tool-capable provider mapping (routing rejects tool calls to mappings without tools support) and deriving thinking levels from the mappings' declared reasoning efforts when they agree - compat: the gateway only honors max_tokens (max_completion_tokens is silently dropped), strips store and function.strict, and accepts anthropic cache_control on claude-routed models - auth: LLMGATEWAY_API_KEY plus a browser sign-in flow that mints an API key via the gateway's loopback callback (state-checked; minted keys live 90 days and cannot be refreshed) - overflow: detect the gateway's pre-flight context-size error Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
LLM Gateway attributes traffic per coding agent via the x-source request header (pi-agent is registered on their side); stamp it on every generated model like the Copilot and Kimi static headers. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
DevPass is LLM Gateway's flat-rate coding subscription. It shares the endpoint, request shape and x-source attribution of the pay-as-you-go llmgateway provider; only the account behind the key differs, so it ships as a sibling provider rather than a mode of the existing one: - auth: the browser sign-in flow is now parameterised by organisation. org=devpass mints the key in the user's personal DevPass org (usage bills the subscription), org=default keeps minting in their dashboard org as before. Credentials and LLMGATEWAY_DEVPASS_API_KEY are scoped per provider so one key cannot silently bill the wrong plan. - catalogue: DevPass keys are answered with 403 for models a coding plan does not cover, so the fetcher narrows the DevPass catalogue with the gateway's own predicate (paid and stable, with a mapping that has tools, streaming and cached-input pricing). Prompt caching is a hard requirement of the flat-rate plan, hence the price check. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Both behaviors ship in this branch without direct coverage: - the gateway pins sticky-session routing on x-session-id, so it takes the openrouter session-affinity format rather than the openai one - it normalizes reasoning_tokens to the top level of usage and omits completion_tokens_details, so reasoning falls back to that field Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Contributor
|
This PR was auto-closed. Only contributors approved with Maintainers review auto-closed issues daily. Issues that do not meet the quality bar in CONTRIBUTING.md will not be reopened or receive a reply. If a maintainer replies See CONTRIBUTING.md. |
Adds LLM Gateway to the cross-provider e2e batteries every built-in provider participates in (abort, empty messages, token stats, tool calls without results, unicode surrogates, tool result images, total tokens, cross-provider handoff), plus an offline catalog test covering the x-source attribution header, the anthropic cache-control routing for claude-backed models, and the DevPass catalog narrowing. Live blocks are gated on LLMGATEWAY_API_KEY and skip without it.
# Conflicts: # packages/ai/README.md # packages/ai/scripts/generate-models.ts # packages/ai/test/cross-provider-handoff.test.ts
# Conflicts: # packages/ai/src/api/openai-completions.ts # packages/ai/test/oauth-auth.test.ts
# Conflicts: # packages/ai/scripts/generate-models.ts
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Resolve the session affinity conflict in openai-completions.ts by enabling the headers for OpenRouter (now the upstream default) and LLM Gateway. Adapt tests to upstream changes: stub the LLM Gateway catalog URL in the Fireworks model generation harness, and pass a normalized transcript context in the usage parsing test. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
- generate-models: keep LLM Gateway fetcher alongside main's Radius and classifier catalogs; exclude LLM Gateway from strict mode next to Cerebras - openai-completions: take main's runtime supportsStrictMode default (false), which already covers LLM Gateway - llmgateway oauth: import page helpers from utils/oauth-page.ts (moved upstream) - regenerate models.generated.ts and llmgateway catalogs for the new chat/image/classifier catalog split - providers.md: re-add LLM Gateway env vars and login notes to main's rewrite - tests: stub LLM Gateway fetch in fireworks generation test; generated compat no longer carries supportsStrictMode: false since it matches the runtime default Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Adds LLM Gateway (an OpenRouter-style router) as built-in
openai-completionsproviders, contributed on behalf of the LLM Gateway team.Replaces #7480, which was auto-closed seconds after opening and stopped tracking the branch, so its diff never showed anything past the first commit. This one carries the complete work.
Two providers ship here. They share an endpoint, a request shape and an attribution header; only the account behind the key differs, which is what decides how usage is billed:
llmgatewayllmgateway-devpassWhat's included
fetchLlmGatewayModels()sources the publichttps://api.llmgateway.io/v1/models(~152 tool-capable models). Models are kept only when at least one provider mapping declarestools: true— the gateway's routing rejects tool calls to mappings without it, and itssupported_parametersfield is unreliable in both directions. Thinking levels are derived from the mappings' declaredreasoning_effortswhen they agree (e.g. gpt-5.4-pro exposes only medium/high/xhigh; claude models gain xhigh/max). One fetch builds both catalogues.max_tokens(max_completion_tokensis silently dropped), stripsstoreandfunction.strict(sosupportsStrictMode: false), and honors anthropiccache_controlonclaude-*routed models.LLMGATEWAY_API_KEY/LLMGATEWAY_DEVPASS_API_KEY, plus a browser sign-in flow modeled on the OpenRouter one — LLM Gateway has no OAuth token endpoint; its dashboard mints an API key delivered to a state-checked loopback callback (source=pi-agentis whitelisted on their side). Minted keys live 90 days and can't be refreshed; on expiry the API 401s and the user re-runs/login.x-session-id(the OpenRouter format rather than the OpenAI one), and normalizesreasoning_tokensto the top level ofusagewhile omittingcompletion_tokens_details, so reasoning falls back to that field.x-source: pi-agentheader (same mechanism as the Copilot/Kimi static headers), so LLM Gateway attributes the traffic to Pi like it does for other coding agents.LLMGATEWAY_API_KEY-gated e2e blocks instreamandcontext-overflow.DevPass specifics
org=devpassmints the key in the user's personal DevPass org so usage bills the subscription,org=defaultkeeps minting in their dashboard org. Everything else about the request is identical.auth.jsonentry, separate env var) so a pay-as-you-go key can't silently bill through the provider labelled as the subscription, or the reverse.Notes for review
*.models.tsshards makeshydrate:model-data --strictdepend onapi.llmgateway.iobeing reachable, like the other live-fetched catalogues. The endpoint is public/unauthenticated./v1/modelshas no coding-plan filter, so the DevPass gate mirrorsisCodingModelfrom the gateway's own source (theopenco/llmgateway,packages/shared/src/coding-models.ts). If those rules change, the catalogue drifts until Pi regenerates. Adding a filter parameter upstream would remove the duplication; happy to follow up with that.max_output(added in feat: expose max_output in /v1/models theopenco/llmgateway#3379, now deployed), falling back to 4096 — the gateway 400s on an over-limitmax_tokens, so the fallback stays conservative.🤖 Generated with Claude Code