Skip to content
2 changes: 1 addition & 1 deletion docs/configuration/bootstrap-config.md
Original file line number Diff line number Diff line change
Expand Up @@ -215,7 +215,7 @@ Do not try to mix the two mental models in one deployment.

## `bedrock_endpoint_url`

Use `bedrock_endpoint_url` only when you need a deployment-wide override for Bedrock guardrail traffic.
Use `bedrock_endpoint_url` only when you need a deployment-wide override for Bedrock guardrail traffic. Skip this field unless you actively use the AWS Bedrock guardrail integration (`kind: bedrock` on a [Guardrail](../overview/glossary.md#guardrail) row); it overrides the default Bedrock endpoint for all such traffic in this deployment.

This is a deployment concern, not a per-guardrail-row field.

Expand Down
6 changes: 3 additions & 3 deletions docs/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ description: Official documentation for AISIX AI Gateway and AISIX Cloud, includ
sidebar_position: 1
---

AISIX AI Gateway is an AI gateway for platform engineers and AI agent developers who need a consistent way to route, govern, and observe LLM traffic across multiple providers. AISIX Cloud extends that gateway with a managed control plane and managed data-plane workflows.
AISIX AI Gateway is an AI [gateway](overview/glossary.md#gateway) for platform engineers and AI agent developers who need a consistent way to route, govern, and observe LLM traffic across multiple providers. [AISIX Cloud](overview/glossary.md#aisix-cloud) extends that gateway with a managed [control plane](overview/glossary.md#control-plane) and managed [data plane](overview/glossary.md#data-plane) workflows.

This documentation set is organized for two primary audiences:

Expand All @@ -25,7 +25,7 @@ This documentation set is organized for two primary audiences:
- Continue with [First Model, First Key, First Request](quickstart/first-model-first-key-first-request.md).
- Use [OpenAI SDK Quickstart](quickstart/openai-sdk.md) if you already have an OpenAI client.
- Use [Anthropic SDK Quickstart](quickstart/anthropic-sdk.md) if you need the Anthropic-style `messages` API.
- If you are evaluating the managed control plane, use the [Deployment Modes](overview/deployment-modes.md) and [Roadmap](roadmap.md) pages first.
- If you signed up for AISIX Cloud and want to point a hosted control plane at your own gateway, follow the [Deployment Modes](overview/deployment-modes.md) and [Roadmap](roadmap.md) pages. If you want to run everything locally on your own machine, start with the [Self-Hosted Quickstart](quickstart/self-hosted.md) instead.
- For current Cloud bootstrap behavior, review [AISIX Cloud Managed Data Plane Quickstart](quickstart/aisix-cloud-managed-dp.md).

### I want to integrate an SDK or client
Expand Down Expand Up @@ -87,7 +87,7 @@ The gateway currently exposes these client-facing routes:
- `POST /v1/audio/translations`
- `POST /v1/audio/speech`
- `POST /v1/images/generations`
- `ANY /passthrough/:provider/*rest`
- `ANY /passthrough/:provider/*rest` (`ANY` here means the route accepts every HTTP verb: GET, POST, PUT, DELETE, etc.)

Start with [OpenAI-Compatible API](integration/openai-compatible-api.md) and the [Reference](reference/proxy-api-reference.md) section.

Expand Down
10 changes: 6 additions & 4 deletions docs/overview/core-concepts.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ This page defines the core objects and terms used across AISIX AI Gateway and AI

## Model

A `Model` is the resource clients target through the gateway.
A `Model` is the resource clients target through the [gateway](glossary.md#gateway).

For direct models, a model includes:

Expand Down Expand Up @@ -59,7 +59,7 @@ An empty `allowed_models` list denies access to every model. A wildcard entry `"

## Rate Limit Policy

A `RateLimitPolicy` is a standalone rate-limit rule stored in etcd. Each policy targets a single subject through `(scope, scope_ref)`:
A `RateLimitPolicy` is a standalone rate-limit rule stored in [etcd](glossary.md#etcd). Each policy targets a single subject through `(scope, scope_ref)`:

- `api_key` — match by `ApiKey` entry id
- `model` — match by `Model` entry id
Expand Down Expand Up @@ -87,7 +87,7 @@ A `Guardrail` is a request or response policy object applied by the gateway.
Current schema supports:

- `keyword`
- `bedrock`
- `bedrock` (the codename for the AWS Bedrock guardrails integration — see the Guardrails reference for setup)

Important current boundary:

Expand Down Expand Up @@ -123,7 +123,7 @@ An `Observability Exporter` ships per-request span telemetry — derived from ga

## Environment

An `Environment` is a first-class AISIX Cloud control-plane concept.
An `Environment` is a first-class [AISIX Cloud](glossary.md#aisix-cloud) [control-plane](glossary.md#control-plane) concept.

The managed data plane watches configuration scoped to its environment. In Cloud mode, projection rules ensure the data plane only sees the resources intended for that environment.

Expand All @@ -139,6 +139,8 @@ In this mode:

## Playground

The playground is an in-process proxy endpoint mounted on the admin listener that forwards requests to the proxy router so model rows can be smoke-tested without a network hop. Auth uses a proxy API key (not the admin key); the full proxy middleware stack — auth, rate limit, bridge, guardrails — still runs.

There are two different playground concepts:

- the standalone gateway has an in-process playground endpoint on the admin listener
Expand Down
52 changes: 52 additions & 0 deletions docs/overview/glossary.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,52 @@
---
title: Glossary
description: Operational definitions of the cross-cutting AISIX AI Gateway and AISIX Cloud terms used throughout the documentation.
sidebar_position: 5
---

This page collects the cross-cutting terms used across AISIX AI Gateway and AISIX Cloud documentation. Each entry is a 1–3 sentence operational definition — what the term *does at runtime*, not marketing prose.

Page-local codenames and one-off identifiers (for example, the `bedrock` guardrail kind or the `YOUR_ADMIN_KEY` placeholder format) are explained inline at first use on the page where they appear.

## gateway

The AISIX runtime binary that accepts caller traffic on the proxy listener and forwards it to upstream model providers. Synonym: data plane.

## data plane

The request-handling tier — the gateway itself. Receives caller traffic, applies caching, guardrails, budgets, and routing, forwards to upstream providers, and returns responses.

## control plane

The management tier. Stores model, API-key, provider-key, guardrail, and cache-policy rows in [etcd](#etcd); the data plane reads from etcd. In standalone deployments the control plane is the gateway's own admin listener. In AISIX Cloud the control plane is a separate hosted service that projects state down to the gateway via etcd-over-TLS.

## AISIX Cloud

The managed control-plane service operated separately from the OSS gateway. Provides multi-tenant team, project, and budget concepts that don't exist in standalone-only mode. See [Deployment Modes](deployment-modes.md) for the comparison.

## API key

Also called the **caller key**. The bearer token your clients send in the `Authorization` header on the proxy listener. Created via the admin API's `POST /admin/v1/apikeys`. The data plane stores `key_hash`, not plaintext — the caller chooses (or generates) the plaintext bearer and SHA-256-hashes it locally before submission, so the gateway never sees or returns the plaintext at create time. The only endpoint that emits a server-generated plaintext is `POST /admin/v1/apikeys/:id/rotate`, which returns the new plaintext exactly once.

## provider key

The upstream provider's credential (for example an OpenAI `sk-...` key) the gateway uses to authenticate to the provider on outbound requests. Created via the admin API's `POST /admin/v1/provider_keys`. Distinct from the [API key](#api-key) your callers send to the gateway.

## guardrail

A request- or response-policy object applied by the gateway. Configured via the admin API's `/admin/v1/guardrails`. Current schema supports the in-process `keyword` backend and an AWS Bedrock backend behind a feature flag. See [Core Concepts § Guardrail](core-concepts.md#guardrail) for the full kind list.

## Observability Exporter

A per-row admin resource that ships per-request span telemetry — derived from gateway `UsageEvent` records — over OTLP/HTTP to an external backend such as Grafana Tempo, Honeycomb, or Langfuse via OTLP. Configure one when you want a per-request trace of gateway proxy activity forwarded to your existing tracing backend. Distinct from process-wide bootstrap observability (service name, log level, Prometheus scrape endpoint) configured in [Bootstrap Configuration](../configuration/bootstrap-config.md).

## etcd

The key-value store the gateway uses for control-plane state. The admin listener writes dynamic resources (models, API keys, provider keys, guardrails, cache policies, observability exporters) into etcd; the data plane watches etcd for live config updates, so restart-free changes ride this path. The admin-side etcd client lives in `crates/aisix-admin/src/etcd_store.rs`.

## Related pages

- [What Is AISIX AI Gateway](what-is-aisix-ai-gateway.md)
- [Core Concepts](core-concepts.md)
- [Deployment Modes](deployment-modes.md)
- [Bootstrap Configuration](../configuration/bootstrap-config.md)
4 changes: 2 additions & 2 deletions docs/overview/what-is-aisix-ai-gateway.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ description: Learn what AISIX AI Gateway is, what problems it solves, and how it
sidebar_position: 1
---

AISIX AI Gateway is an AI gateway that sits between your applications and upstream LLM providers. It gives platform teams a single operational layer for routing, governing, and observing model traffic without forcing application teams to manage every provider integration directly.
AISIX AI Gateway is an AI [gateway](glossary.md#gateway) that sits between your applications and upstream LLM providers. It gives platform teams a single operational layer for routing, governing, and observing model traffic without forcing application teams to manage every provider integration directly.

## What Problems It Solves

Expand Down Expand Up @@ -95,7 +95,7 @@ You run the gateway directly and manage bootstrap configuration, dynamic resourc

### AISIX Cloud Managed Data Plane

AISIX Cloud adds a managed control plane for environments, certificates, and Cloud workflows while the data plane still runs as AISIX AI Gateway.
[AISIX Cloud](glossary.md#aisix-cloud) adds a managed [control plane](glossary.md#control-plane) for environments, certificates, and Cloud workflows while the [data plane](glossary.md#data-plane) still runs as AISIX AI Gateway.

See [Deployment Modes](deployment-modes.md) for the comparison.

Expand Down
26 changes: 20 additions & 6 deletions docs/quickstart/first-model-first-key-first-request.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ description: Create a provider key, model, and API key through the AISIX AI Gate
sidebar_position: 11
---

This guide shows how to move from a running self-hosted gateway to a working end-to-end request. You will create:
This guide shows how to move from a running self-hosted [gateway](../overview/glossary.md#gateway) to a working end-to-end request. You will create:

- one `ProviderKey`
- one `Model`
Expand All @@ -22,16 +22,16 @@ Then you will verify that the new configuration is visible on the proxy surface.

The standalone gateway uses:

- **provider keys** to store upstream credentials and optional base URLs
- **[provider keys](../overview/glossary.md#provider-key)** to store upstream credentials and optional base URLs
- **models** to expose operator-defined model aliases on the proxy surface
- **API keys** to control which callers can access which models
- **[API keys](../overview/glossary.md#api-key)** to control which callers can access which models

## Step 1: Create a Provider Key

Create a provider key that points at your upstream provider.

:::warning Production credentials
The standalone gateway stores `secret` as plaintext under the etcd `prefix` you configured in [`config.yaml`](self-hosted.md#step-2-create-a-bootstrap-config). For production, front etcd with encryption-at-rest, or use AISIX Cloud's managed [Provider Key Rotation](../cloud/provider-key-rotation.md), which holds the secret in the control plane and projects only what each environment needs.
The standalone gateway stores `secret` as plaintext under the [etcd](../overview/glossary.md#etcd) `prefix` you configured in [`config.yaml`](self-hosted.md#step-2-create-a-bootstrap-config). For production, front etcd with encryption-at-rest, or use [AISIX Cloud](../overview/glossary.md#aisix-cloud)'s managed [Provider Key Rotation](../cloud/provider-key-rotation.md), which holds the secret in the [control plane](../overview/glossary.md#control-plane) and projects only what each environment needs.
:::

```bash title="Create a provider key"
Expand Down Expand Up @@ -63,7 +63,17 @@ The admin envelope returns a `ResourceEntry` shape:
}
```

Capture the returned `id`. You will use it as `provider_key_id` when creating the model.
Capture the returned `id` for use as `provider_key_id` in the next step — copy it by eye from the response above, or use this jq-capturing form **instead of** the curl above (running both creates a duplicate `display_name` and the second POST returns 409):

```bash title="Create and capture the id in one shot"
PROVIDER_KEY_ID=$(curl -sS -X POST http://127.0.0.1:3001/admin/v1/provider_keys \
-H "Authorization: Bearer YOUR_ADMIN_KEY" \
-H "Content-Type: application/json" \
-d '{"display_name":"openai-upstream","secret":"YOUR_PROVIDER_API_KEY","api_base":"https://api.openai.com/v1"}' \
| jq -r .id)
```

The same pattern applies to the model `id` and API-key `id` captures in the next two steps.

:::warning
The `secret` field is returned as plaintext in this response. Treat the command output as sensitive — avoid pasting it into shared documents, issue trackers, or chat. The admin API also returns the plaintext on subsequent `GET /admin/v1/provider_keys/:id` calls, so the same handling applies any time you read this resource.
Expand All @@ -73,14 +83,18 @@ The `secret` field is returned as plaintext in this response. Treat the command

Create a model alias that the proxy will expose to callers.

:::tip
This guide uses `gpt-4o-mini` to minimize evaluation costs. Swap in any chat-completions-compatible model supported by your upstream provider.
:::

```bash title="Create a model"
curl -sS -X POST http://127.0.0.1:3001/admin/v1/models \
-H "Authorization: Bearer YOUR_ADMIN_KEY" \
-H "Content-Type: application/json" \
-d '{
"display_name": "gpt-4o-prod",
"provider": "openai",
"model_name": "gpt-4o",
"model_name": "gpt-4o-mini",
"provider_key_id": "YOUR_PROVIDER_KEY_ID"
}'
```
Expand Down
14 changes: 11 additions & 3 deletions docs/quickstart/openai-sdk.md
Original file line number Diff line number Diff line change
Expand Up @@ -14,9 +14,9 @@ Use it when:

Use this page after you have already created:

- a provider key
- a [provider key](../overview/glossary.md#provider-key)
- a model alias
- a caller-facing API key
- a caller-facing [API key](../overview/glossary.md#api-key)

If you have not done that yet, start with [First Model, First Key, First Request](first-model-first-key-first-request.md).

Expand All @@ -41,6 +41,12 @@ What does **not** change:

## Install The SDK

All commands below run from the same project directory. Create one first:

```bash title="Create the demo project directory"
mkdir aisix-openai-demo && cd aisix-openai-demo
```

```bash title="Install openai"
npm install openai
```
Expand Down Expand Up @@ -111,7 +117,9 @@ for await (const chunk of stream) {
}
```

Run with `AISIX_API_KEY=sk-demo-caller node openai-sdk-streaming.mjs`.
```bash title="Run the streaming example"
AISIX_API_KEY=sk-demo-caller node openai-sdk-streaming.mjs
```

## When To Use This Quickstart

Expand Down
6 changes: 3 additions & 3 deletions docs/quickstart/self-hosted.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@ This guide shows how to start a self-hosted AISIX AI Gateway instance with the l

- **Rust 1.93 or newer with `cargo`.** Install via [rustup](https://rustup.rs) (for example, `curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh`) and verify with `cargo --version`. The repo pins this version through `rust-toolchain.toml`, so `rustup` will install the right channel automatically the first time you run `cargo` in the working tree.
- Docker
- A reachable etcd instance
- A reachable [etcd](../overview/glossary.md#etcd) instance

## Step 1: Start etcd

Expand All @@ -29,7 +29,7 @@ docker run -d \

## Step 2: Create a bootstrap config

Create a local `config.yaml` based on the example config.
Create a local `config.yaml` based on the example config. Place this file in the repo root — Step 3's `cargo run` looks for `config.yaml` relative to your current working directory.

```yaml title="config.yaml" {2-7,9-14}
etcd:
Expand Down Expand Up @@ -65,7 +65,7 @@ cargo run -p aisix-server --bin aisix -- --config config.yaml

The first time you run this, `cargo` will compile several hundred dependencies before the gateway starts, which typically takes 3–5 minutes on common hardware. Subsequent runs are incremental and much faster.

In another terminal, you should now have:
Keep this terminal running. In a new terminal, you should now have:

- proxy listener on `http://127.0.0.1:3000`
- admin listener on `http://127.0.0.1:3001`
Expand Down
Loading