Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
12 changes: 5 additions & 7 deletions docs/configuration/bootstrap-config.md
Original file line number Diff line number Diff line change
Expand Up @@ -151,7 +151,7 @@ Recommended pattern:

## `observability`

Use `observability` to set process-wide telemetry knobs: service name, log level, Prometheus exporter control, and (in future releases) access-log gating and OTLP exporters. Today `service_name`, `log_level`, and the `metrics.prometheus.*` block are consulted at runtime; the remaining keys are recognized in the schema and reserved for upcoming releases — setting them is harmless but currently has no effect.
Use `observability` to set process-wide telemetry knobs. Today `service_name`, `log_level`, and the `metrics.prometheus.*` block are consulted at runtime; the other fields have varying current behavior — see the `Status` column below.

Important fields:

Expand All @@ -161,17 +161,15 @@ Important fields:
| `log_level` | fallback `EnvFilter` directive when `RUST_LOG` is not set in the environment | `"info"` | wired |
| `access_log` | reserved field; access logs are currently emitted by every proxy handler regardless of this setting | `true` | reserved (not yet consulted) |
| `metrics.prometheus.enabled` | controls whether the admin listener mounts the Prometheus scrape endpoint; when `false`, no `/metrics` route is registered | `true` | wired |
| `metrics.prometheus.path` | mount path for the Prometheus scrape endpoint when `metrics.prometheus.enabled` is `true`; values without a leading slash are normalised by prepending one, and an empty value falls back to `/metrics` | `"/metrics"` | wired |
| `metrics.prometheus.path` | mount path for the Prometheus scrape endpoint | `"/metrics"` | wired |
| `metrics.otlp.enabled` | reserved field; no OTLP metrics export pipeline is installed in the current release | `false` | reserved (not yet wired) |
| `metrics.otlp.endpoint` | reserved field; see `metrics.otlp.enabled` | none | reserved (not yet wired) |
| `tracing.otlp.enabled` | enabling this validates the endpoint at boot and emits a startup log line; the OTLP traces pipeline itself is deferred to a future release | `false` | partial (validation only) |
| `tracing.otlp.endpoint` | OTLP/gRPC collector endpoint for traces; validated at boot when `tracing.otlp.enabled` is `true` | none | partial (validation only) |
| `metrics.otlp.endpoint` | OTLP/gRPC metrics endpoint | none | reserved (not yet wired) |
| `tracing.otlp.enabled` | boot-time endpoint validation; OTLP traces pipeline deferred | `false` | partial (validation only) |
| `tracing.otlp.endpoint` | OTLP/gRPC collector endpoint for traces | none | partial (validation only) |
| `tracing.otlp.sample_ratio` | head-based sampling ratio reserved for the future OTLP traces pipeline | `1.0` | reserved (not yet wired) |

Bootstrap observability settings are process-wide. They are different from dynamic `ObservabilityExporter` rows, which control per-request span fan-out via OTLP/HTTP at runtime. For per-row dynamic exporters added at runtime via the admin API, see [Observability Exporters](observability-exporters.md).

`observability.metrics.prometheus.enabled` controls whether the admin listener mounts the Prometheus scrape endpoint. `observability.metrics.prometheus.path` controls the mounted path and defaults to `/metrics`.

## `cache`

Use `cache` to choose the bootstrap cache backend.
Expand Down
2 changes: 1 addition & 1 deletion docs/configuration/caching.md
Original file line number Diff line number Diff line change
Expand Up @@ -82,7 +82,7 @@ Current runtime boundary:
- bootstrap config can wire a Redis backend at process start
- the dynamic `CachePolicy.backend` field should still be treated conservatively because broader Redis support boundaries are still being expanded

Note: the per-policy `backend` field is currently parsed and stored on the `CachePolicy` row but is not consulted by the runtime proxy. The proxy uses the cache backend selected via bootstrap-config (`cache.backend`) regardless of what each policy specifies. The field is preserved for forward compatibility; do not depend on it to override the runtime backend.
Note: the per-policy `backend` field is parsed and stored on the `CachePolicy` row but is not consulted by the runtime proxy — the proxy always uses the cache backend selected by bootstrap-config (`cache.backend`) regardless of what each policy specifies. The field is preserved for forward compatibility; do not depend on it to override the runtime backend.

## Operator Guidance

Expand Down
4 changes: 2 additions & 2 deletions docs/configuration/models.md
Original file line number Diff line number Diff line change
Expand Up @@ -77,7 +77,7 @@ Current semantics:

- only direct models may carry `background_model_check`
- routing models reject `background_model_check`
- `ignore_statuses` records the last probe result without marking the model unhealthy. If omitted, **no** probe statuses are ignored — a 408 or 429 probe response would mark the model unhealthy. Set this field explicitly to skip those statuses. For most deployments, `ignore_statuses: [408, 429]` is a reasonable starting point — it tolerates transient upstream timeouts (408) and rate-limit responses (429) during probes without flapping the model unhealthy.
- `ignore_statuses` records the last probe result without marking the model unhealthy. If omitted, **no** statuses are ignored — a 408 or 429 probe response would mark the model unhealthy. For most deployments, `[408, 429]` is a reasonable starting point.
- `stale_after_seconds` is a safety valve for old unhealthy probe state when the checker stops refreshing
- `interval_seconds` has a minimum of `5`; `timeout_seconds`, `max_tokens`, and `stale_after_seconds` have a minimum of `1`

Expand Down Expand Up @@ -160,7 +160,7 @@ curl -sS -X POST http://127.0.0.1:3001/admin/v1/models \
- `provider` currently supports `openai`, `anthropic`, `google`, `deepseek`, `cohere`, and `jina`.
- `provider_key_id` must reference an existing `ProviderKey` resource.
- `timeout` is in milliseconds. `0` or omission means no timeout.
- `cost` stores pricing metadata. AISIX Cloud's cp-api recomputes cost server-side from its pricing catalog when emitting usage events and consumes this field at that layer. The standalone OSS proxy does not consult this field at request time and always emits `cost_usd=0.0` in its own usage events; pricing-aware budget enforcement requires the AISIX Cloud control plane.
- `cost` stores pricing metadata that AISIX Cloud's cp-api consumes when emitting usage events. The standalone OSS proxy does not consult this field at request time and always emits `cost_usd=0.0`; pricing-aware budget enforcement requires the AISIX Cloud control plane.
- `background_model_check` drives direct-model runtime unhealthy state and the `/admin/v1/models/status` view.
- `cooldown` drives direct-model request-path cooldown and is also surfaced through `/admin/v1/models/status`.

Expand Down
2 changes: 1 addition & 1 deletion docs/quickstart/first-model-first-key-first-request.md
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@ Then you will verify that the new configuration is visible on the proxy surface.
## Prerequisites

- A running gateway from the [Self-Hosted Quickstart](self-hosted.md)
- An API key from an upstream provider (OpenAI, Anthropic, Google, DeepSeek, or another supported provider). If you do not have one, sign up at the provider (for example, <https://platform.openai.com/api-keys> — paid; pay-as-you-go starts around $5). The key looks like `sk-...` (or the provider equivalent) and is what you will paste into the `YOUR_PROVIDER_API_KEY` placeholder in Step 1.
- An API key from an upstream provider (OpenAI, Anthropic, Google, DeepSeek, or another supported provider). The key looks like `sk-...` and is what you will paste into the `YOUR_PROVIDER_API_KEY` placeholder in Step 1. If you do not have one, sign up at the provider (for example, <https://platform.openai.com/api-keys>).
Comment thread
janiussyafiq marked this conversation as resolved.
- Your admin key from the bootstrap config

## What This Quickstart Configures
Expand Down
2 changes: 1 addition & 1 deletion docs/quickstart/self-hosted.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ This guide shows how to start a self-hosted AISIX AI Gateway instance with the l

## Prerequisites

- **Rust 1.93 or newer with `cargo`.** Install via [rustup](https://rustup.rs) (for example, `curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh`) and verify with `cargo --version`. The repo pins this version through `rust-toolchain.toml`, so `rustup` will install the right channel automatically the first time you run `cargo` in the working tree.
- **Rust 1.93 or newer with `cargo`.** Install via [rustup](https://rustup.rs) and verify with `cargo --version`. The repo pins this version through `rust-toolchain.toml`, so `rustup` selects the right channel automatically.
- Docker
- A reachable etcd instance

Expand Down
Loading