diff --git a/docs/configuration/bootstrap-config.md b/docs/configuration/bootstrap-config.md index 4771ee83..a79533a2 100644 --- a/docs/configuration/bootstrap-config.md +++ b/docs/configuration/bootstrap-config.md @@ -215,7 +215,7 @@ Do not try to mix the two mental models in one deployment. ## `bedrock_endpoint_url` -Use `bedrock_endpoint_url` only when you need a deployment-wide override for Bedrock guardrail traffic. +Use `bedrock_endpoint_url` only when you need a deployment-wide override for Bedrock guardrail traffic. Skip this field unless you actively use the AWS Bedrock guardrail integration (`kind: bedrock` on a [Guardrail](../overview/glossary.md#guardrail) row); it overrides the default Bedrock endpoint for all such traffic in this deployment. This is a deployment concern, not a per-guardrail-row field. diff --git a/docs/index.md b/docs/index.md index 4d3984df..b790ad1c 100644 --- a/docs/index.md +++ b/docs/index.md @@ -4,7 +4,7 @@ description: Official documentation for AISIX AI Gateway and AISIX Cloud, includ sidebar_position: 1 --- -AISIX AI Gateway is an AI gateway for platform engineers and AI agent developers who need a consistent way to route, govern, and observe LLM traffic across multiple providers. AISIX Cloud extends that gateway with a managed control plane and managed data-plane workflows. +AISIX AI Gateway is an AI [gateway](overview/glossary.md#gateway) for platform engineers and AI agent developers who need a consistent way to route, govern, and observe LLM traffic across multiple providers. [AISIX Cloud](overview/glossary.md#aisix-cloud) extends that gateway with a managed [control plane](overview/glossary.md#control-plane) and managed [data plane](overview/glossary.md#data-plane) workflows. This documentation set is organized for two primary audiences: @@ -25,7 +25,7 @@ This documentation set is organized for two primary audiences: - Continue with [First Model, First Key, First Request](quickstart/first-model-first-key-first-request.md). - Use [OpenAI SDK Quickstart](quickstart/openai-sdk.md) if you already have an OpenAI client. - Use [Anthropic SDK Quickstart](quickstart/anthropic-sdk.md) if you need the Anthropic-style `messages` API. -- If you are evaluating the managed control plane, use the [Deployment Modes](overview/deployment-modes.md) and [Roadmap](roadmap.md) pages first. +- If you signed up for AISIX Cloud and want to point a hosted control plane at your own gateway, follow the [Deployment Modes](overview/deployment-modes.md) and [Roadmap](roadmap.md) pages. If you want to run everything locally on your own machine, start with the [Self-Hosted Quickstart](quickstart/self-hosted.md) instead. - For current Cloud bootstrap behavior, review [AISIX Cloud Managed Data Plane Quickstart](quickstart/aisix-cloud-managed-dp.md). ### I want to integrate an SDK or client @@ -87,7 +87,7 @@ The gateway currently exposes these client-facing routes: - `POST /v1/audio/translations` - `POST /v1/audio/speech` - `POST /v1/images/generations` -- `ANY /passthrough/:provider/*rest` +- `ANY /passthrough/:provider/*rest` (`ANY` here means the route accepts every HTTP verb: GET, POST, PUT, DELETE, etc.) Start with [OpenAI-Compatible API](integration/openai-compatible-api.md) and the [Reference](reference/proxy-api-reference.md) section. diff --git a/docs/overview/core-concepts.md b/docs/overview/core-concepts.md index 4e9c16a8..caf553e7 100644 --- a/docs/overview/core-concepts.md +++ b/docs/overview/core-concepts.md @@ -8,7 +8,7 @@ This page defines the core objects and terms used across AISIX AI Gateway and AI ## Model -A `Model` is the resource clients target through the gateway. +A `Model` is the resource clients target through the [gateway](glossary.md#gateway). For direct models, a model includes: @@ -59,7 +59,7 @@ An empty `allowed_models` list denies access to every model. A wildcard entry `" ## Rate Limit Policy -A `RateLimitPolicy` is a standalone rate-limit rule stored in etcd. Each policy targets a single subject through `(scope, scope_ref)`: +A `RateLimitPolicy` is a standalone rate-limit rule stored in [etcd](glossary.md#etcd). Each policy targets a single subject through `(scope, scope_ref)`: - `api_key` — match by `ApiKey` entry id - `model` — match by `Model` entry id @@ -87,7 +87,7 @@ A `Guardrail` is a request or response policy object applied by the gateway. Current schema supports: - `keyword` -- `bedrock` +- `bedrock` (the codename for the AWS Bedrock guardrails integration — see the Guardrails reference for setup) Important current boundary: @@ -123,7 +123,7 @@ An `Observability Exporter` ships per-request span telemetry — derived from ga ## Environment -An `Environment` is a first-class AISIX Cloud control-plane concept. +An `Environment` is a first-class [AISIX Cloud](glossary.md#aisix-cloud) [control-plane](glossary.md#control-plane) concept. The managed data plane watches configuration scoped to its environment. In Cloud mode, projection rules ensure the data plane only sees the resources intended for that environment. @@ -139,6 +139,8 @@ In this mode: ## Playground +The playground is an in-process proxy endpoint mounted on the admin listener that forwards requests to the proxy router so model rows can be smoke-tested without a network hop. Auth uses a proxy API key (not the admin key); the full proxy middleware stack — auth, rate limit, bridge, guardrails — still runs. + There are two different playground concepts: - the standalone gateway has an in-process playground endpoint on the admin listener diff --git a/docs/overview/glossary.md b/docs/overview/glossary.md new file mode 100644 index 00000000..a72d639f --- /dev/null +++ b/docs/overview/glossary.md @@ -0,0 +1,52 @@ +--- +title: Glossary +description: Operational definitions of the cross-cutting AISIX AI Gateway and AISIX Cloud terms used throughout the documentation. +sidebar_position: 5 +--- + +This page collects the cross-cutting terms used across AISIX AI Gateway and AISIX Cloud documentation. Each entry is a 1–3 sentence operational definition — what the term *does at runtime*, not marketing prose. + +Page-local codenames and one-off identifiers (for example, the `bedrock` guardrail kind or the `YOUR_ADMIN_KEY` placeholder format) are explained inline at first use on the page where they appear. + +## gateway + +The AISIX runtime binary that accepts caller traffic on the proxy listener and forwards it to upstream model providers. Synonym: data plane. + +## data plane + +The request-handling tier — the gateway itself. Receives caller traffic, applies caching, guardrails, budgets, and routing, forwards to upstream providers, and returns responses. + +## control plane + +The management tier. Stores model, API-key, provider-key, guardrail, and cache-policy rows in [etcd](#etcd); the data plane reads from etcd. In standalone deployments the control plane is the gateway's own admin listener. In AISIX Cloud the control plane is a separate hosted service that projects state down to the gateway via etcd-over-TLS. + +## AISIX Cloud + +The managed control-plane service operated separately from the OSS gateway. Provides multi-tenant team, project, and budget concepts that don't exist in standalone-only mode. See [Deployment Modes](deployment-modes.md) for the comparison. + +## API key + +Also called the **caller key**. The bearer token your clients send in the `Authorization` header on the proxy listener. Created via the admin API's `POST /admin/v1/apikeys`. The data plane stores `key_hash`, not plaintext — the caller chooses (or generates) the plaintext bearer and SHA-256-hashes it locally before submission, so the gateway never sees or returns the plaintext at create time. The only endpoint that emits a server-generated plaintext is `POST /admin/v1/apikeys/:id/rotate`, which returns the new plaintext exactly once. + +## provider key + +The upstream provider's credential (for example an OpenAI `sk-...` key) the gateway uses to authenticate to the provider on outbound requests. Created via the admin API's `POST /admin/v1/provider_keys`. Distinct from the [API key](#api-key) your callers send to the gateway. + +## guardrail + +A request- or response-policy object applied by the gateway. Configured via the admin API's `/admin/v1/guardrails`. Current schema supports the in-process `keyword` backend and an AWS Bedrock backend behind a feature flag. See [Core Concepts § Guardrail](core-concepts.md#guardrail) for the full kind list. + +## Observability Exporter + +A per-row admin resource that ships per-request span telemetry — derived from gateway `UsageEvent` records — over OTLP/HTTP to an external backend such as Grafana Tempo, Honeycomb, or Langfuse via OTLP. Configure one when you want a per-request trace of gateway proxy activity forwarded to your existing tracing backend. Distinct from process-wide bootstrap observability (service name, log level, Prometheus scrape endpoint) configured in [Bootstrap Configuration](../configuration/bootstrap-config.md). + +## etcd + +The key-value store the gateway uses for control-plane state. The admin listener writes dynamic resources (models, API keys, provider keys, guardrails, cache policies, observability exporters) into etcd; the data plane watches etcd for live config updates, so restart-free changes ride this path. The admin-side etcd client lives in `crates/aisix-admin/src/etcd_store.rs`. + +## Related pages + +- [What Is AISIX AI Gateway](what-is-aisix-ai-gateway.md) +- [Core Concepts](core-concepts.md) +- [Deployment Modes](deployment-modes.md) +- [Bootstrap Configuration](../configuration/bootstrap-config.md) diff --git a/docs/overview/what-is-aisix-ai-gateway.md b/docs/overview/what-is-aisix-ai-gateway.md index 67afc345..157d0573 100644 --- a/docs/overview/what-is-aisix-ai-gateway.md +++ b/docs/overview/what-is-aisix-ai-gateway.md @@ -4,7 +4,7 @@ description: Learn what AISIX AI Gateway is, what problems it solves, and how it sidebar_position: 1 --- -AISIX AI Gateway is an AI gateway that sits between your applications and upstream LLM providers. It gives platform teams a single operational layer for routing, governing, and observing model traffic without forcing application teams to manage every provider integration directly. +AISIX AI Gateway is an AI [gateway](glossary.md#gateway) that sits between your applications and upstream LLM providers. It gives platform teams a single operational layer for routing, governing, and observing model traffic without forcing application teams to manage every provider integration directly. ## What Problems It Solves @@ -95,7 +95,7 @@ You run the gateway directly and manage bootstrap configuration, dynamic resourc ### AISIX Cloud Managed Data Plane -AISIX Cloud adds a managed control plane for environments, certificates, and Cloud workflows while the data plane still runs as AISIX AI Gateway. +[AISIX Cloud](glossary.md#aisix-cloud) adds a managed [control plane](glossary.md#control-plane) for environments, certificates, and Cloud workflows while the [data plane](glossary.md#data-plane) still runs as AISIX AI Gateway. See [Deployment Modes](deployment-modes.md) for the comparison. diff --git a/docs/quickstart/first-model-first-key-first-request.md b/docs/quickstart/first-model-first-key-first-request.md index 416cebee..edddc985 100644 --- a/docs/quickstart/first-model-first-key-first-request.md +++ b/docs/quickstart/first-model-first-key-first-request.md @@ -4,7 +4,7 @@ description: Create a provider key, model, and API key through the AISIX AI Gate sidebar_position: 11 --- -This guide shows how to move from a running self-hosted gateway to a working end-to-end request. You will create: +This guide shows how to move from a running self-hosted [gateway](../overview/glossary.md#gateway) to a working end-to-end request. You will create: - one `ProviderKey` - one `Model` @@ -22,16 +22,16 @@ Then you will verify that the new configuration is visible on the proxy surface. The standalone gateway uses: -- **provider keys** to store upstream credentials and optional base URLs +- **[provider keys](../overview/glossary.md#provider-key)** to store upstream credentials and optional base URLs - **models** to expose operator-defined model aliases on the proxy surface -- **API keys** to control which callers can access which models +- **[API keys](../overview/glossary.md#api-key)** to control which callers can access which models ## Step 1: Create a Provider Key Create a provider key that points at your upstream provider. :::warning Production credentials -The standalone gateway stores `secret` as plaintext under the etcd `prefix` you configured in [`config.yaml`](self-hosted.md#step-2-create-a-bootstrap-config). For production, front etcd with encryption-at-rest, or use AISIX Cloud's managed [Provider Key Rotation](../cloud/provider-key-rotation.md), which holds the secret in the control plane and projects only what each environment needs. +The standalone gateway stores `secret` as plaintext under the [etcd](../overview/glossary.md#etcd) `prefix` you configured in [`config.yaml`](self-hosted.md#step-2-create-a-bootstrap-config). For production, front etcd with encryption-at-rest, or use [AISIX Cloud](../overview/glossary.md#aisix-cloud)'s managed [Provider Key Rotation](../cloud/provider-key-rotation.md), which holds the secret in the [control plane](../overview/glossary.md#control-plane) and projects only what each environment needs. ::: ```bash title="Create a provider key" @@ -63,7 +63,17 @@ The admin envelope returns a `ResourceEntry` shape: } ``` -Capture the returned `id`. You will use it as `provider_key_id` when creating the model. +Capture the returned `id` for use as `provider_key_id` in the next step — copy it by eye from the response above, or use this jq-capturing form **instead of** the curl above (running both creates a duplicate `display_name` and the second POST returns 409): + +```bash title="Create and capture the id in one shot" +PROVIDER_KEY_ID=$(curl -sS -X POST http://127.0.0.1:3001/admin/v1/provider_keys \ + -H "Authorization: Bearer YOUR_ADMIN_KEY" \ + -H "Content-Type: application/json" \ + -d '{"display_name":"openai-upstream","secret":"YOUR_PROVIDER_API_KEY","api_base":"https://api.openai.com/v1"}' \ + | jq -r .id) +``` + +The same pattern applies to the model `id` and API-key `id` captures in the next two steps. :::warning The `secret` field is returned as plaintext in this response. Treat the command output as sensitive — avoid pasting it into shared documents, issue trackers, or chat. The admin API also returns the plaintext on subsequent `GET /admin/v1/provider_keys/:id` calls, so the same handling applies any time you read this resource. @@ -73,6 +83,10 @@ The `secret` field is returned as plaintext in this response. Treat the command Create a model alias that the proxy will expose to callers. +:::tip +This guide uses `gpt-4o-mini` to minimize evaluation costs. Swap in any chat-completions-compatible model supported by your upstream provider. +::: + ```bash title="Create a model" curl -sS -X POST http://127.0.0.1:3001/admin/v1/models \ -H "Authorization: Bearer YOUR_ADMIN_KEY" \ @@ -80,7 +94,7 @@ curl -sS -X POST http://127.0.0.1:3001/admin/v1/models \ -d '{ "display_name": "gpt-4o-prod", "provider": "openai", - "model_name": "gpt-4o", + "model_name": "gpt-4o-mini", "provider_key_id": "YOUR_PROVIDER_KEY_ID" }' ``` diff --git a/docs/quickstart/openai-sdk.md b/docs/quickstart/openai-sdk.md index b32480f2..c7a07fab 100644 --- a/docs/quickstart/openai-sdk.md +++ b/docs/quickstart/openai-sdk.md @@ -14,9 +14,9 @@ Use it when: Use this page after you have already created: -- a provider key +- a [provider key](../overview/glossary.md#provider-key) - a model alias -- a caller-facing API key +- a caller-facing [API key](../overview/glossary.md#api-key) If you have not done that yet, start with [First Model, First Key, First Request](first-model-first-key-first-request.md). @@ -41,6 +41,12 @@ What does **not** change: ## Install The SDK +All commands below run from the same project directory. Create one first: + +```bash title="Create the demo project directory" +mkdir aisix-openai-demo && cd aisix-openai-demo +``` + ```bash title="Install openai" npm install openai ``` @@ -111,7 +117,9 @@ for await (const chunk of stream) { } ``` -Run with `AISIX_API_KEY=sk-demo-caller node openai-sdk-streaming.mjs`. +```bash title="Run the streaming example" +AISIX_API_KEY=sk-demo-caller node openai-sdk-streaming.mjs +``` ## When To Use This Quickstart diff --git a/docs/quickstart/self-hosted.md b/docs/quickstart/self-hosted.md index 3ae601e4..9a99644d 100644 --- a/docs/quickstart/self-hosted.md +++ b/docs/quickstart/self-hosted.md @@ -10,7 +10,7 @@ This guide shows how to start a self-hosted AISIX AI Gateway instance with the l - **Rust 1.93 or newer with `cargo`.** Install via [rustup](https://rustup.rs) (for example, `curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh`) and verify with `cargo --version`. The repo pins this version through `rust-toolchain.toml`, so `rustup` will install the right channel automatically the first time you run `cargo` in the working tree. - Docker -- A reachable etcd instance +- A reachable [etcd](../overview/glossary.md#etcd) instance ## Step 1: Start etcd @@ -29,7 +29,7 @@ docker run -d \ ## Step 2: Create a bootstrap config -Create a local `config.yaml` based on the example config. +Create a local `config.yaml` based on the example config. Place this file in the repo root — Step 3's `cargo run` looks for `config.yaml` relative to your current working directory. ```yaml title="config.yaml" {2-7,9-14} etcd: @@ -65,7 +65,7 @@ cargo run -p aisix-server --bin aisix -- --config config.yaml The first time you run this, `cargo` will compile several hundred dependencies before the gateway starts, which typically takes 3–5 minutes on common hardware. Subsequent runs are incremental and much faster. -In another terminal, you should now have: +Keep this terminal running. In a new terminal, you should now have: - proxy listener on `http://127.0.0.1:3000` - admin listener on `http://127.0.0.1:3001`