Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 4 additions & 2 deletions docs/configuration/models.md
Original file line number Diff line number Diff line change
Expand Up @@ -156,8 +156,9 @@ curl -sS -X POST http://127.0.0.1:3001/admin/v1/models \

## Field Notes

- `display_name` is the alias clients send in proxy requests.
- `provider` currently supports `openai`, `anthropic`, `google`, `deepseek`, `cohere`, and `jina`.
- `display_name` is the alias clients send in proxy requests, and the value `response.model` echoes back. It is **not** the upstream model id.
- `model_name` is the upstream model id — the literal string the upstream provider expects in its own `model` field (for example `gpt-4o`, `claude-sonnet-4-5`, an Azure deployment name, or a Bedrock model id). Despite the name, this field holds the upstream id, not a caller alias; the caller alias is `display_name`.
- `provider` is a free-form vendor label, not a closed enum. The value must match the pattern `^[a-z0-9][a-z0-9._-]*$` (lowercase alphanumerics plus `.`, `-`, `_`, and no leading separator) and be at most 64 characters. In AISIX Cloud it is the catalog provider id (for example `openai`, `anthropic`, `deepseek`, `amazon-bedrock`); in the self-hosted gateway it can be any label you choose for a vendor or endpoint (for example `vllm`, `openrouter`, `xai`). Dispatch reads the referenced provider key's `adapter` and `provider`; this field also serves as a metrics and access-log label and gates a few vendor-specific endpoints. See [Adapter protocol families](../reference/adapters.md#how-a-model-resolves-to-a-bridge).
- `provider_key_id` must reference an existing `ProviderKey` resource.
- `timeout` is in milliseconds. `0` or omission means no timeout.
- `cost` stores pricing metadata that AISIX Cloud's cp-api consumes when emitting usage events. The standalone OSS proxy does not consult this field at request time and always emits `cost_usd=0.0`; pricing-aware budget enforcement requires the AISIX Cloud control plane.
Expand Down Expand Up @@ -221,6 +222,7 @@ That is expected with the current discovery boundary.
## Related Pages

- [Provider Keys](provider-keys.md)
- [Adapter protocol families](../reference/adapters.md) — how `provider` and the provider key's `adapter` select an upstream bridge.
- [API Keys](api-keys.md)
- [Routing And Failover](routing-and-failover.md)
- [Configuration Propagation](configuration-propagation.md)
21 changes: 19 additions & 2 deletions docs/configuration/provider-keys.md
Original file line number Diff line number Diff line change
Expand Up @@ -19,12 +19,20 @@ Think of a provider key as the upstream credential container, not the client-fac
- `display_name`
- `secret`
- optional `api_base`
- optional `provider`
- optional `adapter`
- optional `telemetry_tags`

In practice:

- `display_name` is for operator readability
- `secret` is the actual upstream credential used at dispatch time
- `api_base` is how you override the provider's default endpoint root
- `provider` is the upstream vendor identity (a free-form lowercase label such as `openai`, `anthropic`, `deepseek`, `vllm`); it is the first-tier [dispatch key](../reference/adapters.md#how-a-model-resolves-to-a-bridge) and a metrics label
- `adapter` pins the upstream wire shape to one of `openai`, `anthropic`, `bedrock`, `vertex`, `azure-openai`; it is the second-tier dispatch key that routes Bedrock, Vertex, Azure OpenAI, and long-tail OpenAI-compatible vendors to the right bridge
- `telemetry_tags` carries attribution metadata (`kind` of `catalog` or `byo`, plus optional labels); it is populated by AISIX Cloud and is not required for self-hosted use

For the complete field reference — every type, validation rule, and the `request`/`response` runtime-config overrides — see the [Provider key schema](../reference/runtime-config-schema.md).

Example:

Expand All @@ -38,6 +46,8 @@ curl -sS -X POST http://127.0.0.1:3001/admin/v1/provider_keys \
-H "Content-Type: application/json" \
-d '{
"display_name": "openai-prod",
"provider": "openai",
"adapter": "openai",
"secret": "YOUR_PROVIDER_API_KEY",
"api_base": "https://api.openai.com/v1"
}'
Expand All @@ -47,15 +57,17 @@ curl -sS -X POST http://127.0.0.1:3001/admin/v1/provider_keys \

`api_base` overrides the provider's default upstream base URL. Each provider bridge appends a different path at request time, so the canonical form `api_base` should take depends on which `provider` your model selects.

Each provider has its own convention — the four current bridges do **not** share one. Use the table below; do not generalize from one row to another.
Each provider has its own convention — the bridges do **not** share one. Use the table below; do not generalize from one row to another. The first rows are keyed on the vendor `provider`; the last two rows are keyed on the `adapter` family (Bedrock and Azure OpenAI dispatch by adapter, not by a fixed vendor string). For how `provider` and `adapter` select a bridge, see [Adapter protocol families](../reference/adapters.md).

| `provider` | Canonical `api_base` form | Bridge appends | Default if `api_base` is omitted |
| `provider` / `adapter` | Canonical `api_base` form | Bridge appends | Default if `api_base` is omitted |
|---|---|---|---|
| `openai` | include `/v1` | `/chat/completions`, `/embeddings`, `/completions`, `/images/generations`, `/audio/*` | `https://api.openai.com/v1` |
| `deepseek` | bare host (DeepSeek serves OpenAI-compatible paths at the host root) | `/chat/completions` | `https://api.deepseek.com` |
| `google` | host plus the OpenAI-compat prefix `/v1beta/openai` | `/chat/completions` | `https://generativelanguage.googleapis.com/v1beta/openai` |
| `anthropic` | bare host | `/v1/messages` | `https://api.anthropic.com` |
| `google-vertex` | bare host, no path | `/v1/projects/<project>/locations/<region>/publishers/google/models/<model>:generateContent` (non-streaming) or `:streamGenerateContent?alt=sse` (streaming). `<project>` and `<region>` come from the SA JSON inside `secret`. | `https://<region>-aiplatform.googleapis.com` |
| `bedrock` (adapter) | `api_base` usually **unset** | `/model/<model>/converse` or the Anthropic `/invoke` route, SigV4-signed | Region-keyed `bedrock-runtime.<region>.amazonaws.com`; the region comes from the `region` field in the credential JSON inside `secret`, not from `api_base`. Set `api_base` only for a private (VPC) Bedrock endpoint. |
| `azure-openai` (adapter) | the resource host `https://<resource>.openai.azure.com` (a bare resource name is also accepted) | `/openai/deployments/<deployment>/chat/completions?api-version=<version>` | No default — `api_base` is required and supplies the resource host. A verbatim override host that does not end in `.openai.azure.com` is trusted as-is for a corporate proxy or mock. |

The OpenAI and Anthropic conventions match each upstream's official SDK — `openai-python` initialises `base_url = "https://api.openai.com/v1"`, while `anthropic-sdk-python` initialises `base_url = "https://api.anthropic.com"` and appends `/v1/messages` itself. DeepSeek is OpenAI-compatible but exposes `/chat/completions` directly at the host root, and Google's Gemini OpenAI-compatible surface lives under a fixed `/v1beta/openai` prefix that the bridge does not synthesize. The Vertex bridge appends a parameterized URL of the form `/v1/projects/<project>/locations/<region>/publishers/google/models/<model>:generateContent`, so the canonical `api_base` form for `google-vertex` is the bare host root — operators behind a corporate proxy or air-gapped network point `api_base` at their proxy host, and the bridge tacks on the rest. Note that OAuth token minting still hits `secret.token_uri` (controlled by the SA JSON, not `api_base`); operators behind a fully air-gapped network must additionally point `token_uri` at their internal token endpoint.

Expand Down Expand Up @@ -118,5 +130,10 @@ That is expected if they all share the same provider key. The shared key is the
## Related Pages

- [Models](models.md)
- [Provider key schema](../reference/runtime-config-schema.md) — the complete field reference, including `request`/`response` overrides.
- [Adapter protocol families](../reference/adapters.md) — how `provider` and `adapter` select a bridge.
- [Bring your own endpoint](byo-endpoint.md) — point the `openai` adapter at a private or self-hosted endpoint.
- [OpenAI-compatible vendor upstream](../integration/upstream-openai-compat.md) — onboard a public OpenAI-compatible vendor (DeepSeek, Groq, Mistral).
- [AWS Bedrock upstream](../integration/upstream-bedrock.md), [Google Vertex AI upstream](../integration/upstream-vertex.md), [Azure OpenAI upstream](../integration/upstream-azure-openai.md) — the specialized-family guides.
- [OpenAI-Compatible API](../integration/openai-compatible-api.md)
- [Configuration Propagation](configuration-propagation.md)
10 changes: 10 additions & 0 deletions docs/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -37,6 +37,14 @@ This documentation set is organized for two primary audiences:
- Use [Errors And Retries](integration/errors-and-retries.md) for shared failure handling.
- Use the quickstarts to configure a working model and caller key first.

### I want to connect an upstream provider

- Read [Adapter protocol families](reference/adapters.md) to see which of the five wire shapes your provider uses.
- Onboard a public OpenAI-compatible vendor (DeepSeek, Groq, Mistral) with [OpenAI-compatible vendor upstream](integration/upstream-openai-compat.md).
- Point the gateway at a private or self-hosted endpoint with [Bring your own endpoint](configuration/byo-endpoint.md).
- Connect a specialized provider with [AWS Bedrock](integration/upstream-bedrock.md), [Google Vertex AI](integration/upstream-vertex.md), or [Azure OpenAI](integration/upstream-azure-openai.md).
- Look up the credential resource fields in the [Provider key schema](reference/runtime-config-schema.md).

### I want to operate the gateway in production

- Start with the [Self-Hosted Quickstart](quickstart/self-hosted.md).
Expand All @@ -59,6 +67,8 @@ This documentation set is organized for two primary audiences:
- [AISIX Cloud](cloud/overview.md)
- [Operations](operations/production-deployment.md)
- [Reference](reference/proxy-api-reference.md)
- [Adapter protocol families](reference/adapters.md)
- [Provider key schema](reference/runtime-config-schema.md)
- [Tutorials](tutorials/build-a-virtual-model-with-failover.md)
- [Roadmap](roadmap.md)

Expand Down
Loading
Loading