[whybeyoung]

Motivation

Goal: let an operator declare a vendor's whole API contract in one YAML file, instead of assembling it from flags, and reuse that contract across deployments.

Serving a model "the way vendor X serves it" takes more than sampling defaults. A vendor's acceptance checks also cover:

• the largest output it allows, and whether a larger request is rejected or clamped;
• how big a request may be, and how many images it may carry;
• which model names it answers to;
• the type its 400 errors carry.

Today only the sampling part is configurable (--override-sampling-params, #39000–#39002). Everything else is either hard-coded (the 32 MiB body limit) or not possible at all. Each new vendor contract would otherwise mean another set of flags, repeated in every deployment manifest.

An API profile is that contract as a YAML file:

• Presets compiled in. Vendor presets ship in the binary, so --api-profile moonshot-kimi is a complete contract.
• Deployment overrides. A deployment file can extends a preset and change only what differs.
• Opt-in. Without a profile the router behaves exactly as before.

Modifications

Profile sections (every key is optional; an unknown key fails startup):

Section	Effect
sampling	The existing --override-sampling-params contract as a section, parsed and enforced by the same code (parse_sampling_overrides, resolve_sampling_defaults). The flag still works and, when given, replaces the section.
output.max_tokens	cap, with on_exceed: reject (400) or clamp, and a default injected when a request sets no budget. Read with the engine's precedence (max_completion_tokens, then max_tokens).
limits	max_body_bytes replaces the 32 MiB default (413 above it); max_images caps image_url parts per request (400).
models.aliases	Model names accepted besides --model-id and routed as it.
errors.bad_request_type	Replaces type in every 400 body: router rejections and engine errors.

Selection: the first source that is set wins.

1. --api-profile-file
2. --api-profile
3. SGLANG_ROUTER_API_PROFILE
4. /etc/sgl-router/profile.yaml, which lets a vendor-specific image bake one in
5. the built-in openai-compatible, which enforces nothing

extends merges a file onto a preset or another file: maps merge, scalars replace, and null deletes an inherited key. --print-profile prints the merged result and exits.

Presets:

• openai-compatible: the no-op default.
• moonshot-kimi: Moonshot's immutable sampling, the case [Router] Fleet-wide sampling contract 1/3: the config surface #39000 was built for, now one flag.

Request path:

• The budget and image checks run once in PreparedRequest::chat.
• A default budget is spliced into the body with the existing append_fields. Only a clamp re-serializes the body.
• output, limits.max_images, models and errors apply to /v1/chat/completions. The body limit applies to every inference route.

Code:

• New src/profile/ module: mod.rs holds the types and validation, load.rs handles presets, files and extends, and enforce.rs holds the request-time checks.
• New profiles/*.yaml presets.
• A README section "API profiles".
• Every hand-built ModelConfig in the tests gains one line, profile: Default::default().

Not in this PR:

• Profile switches for /v1/responses and /v1/messages ([Feat] sgl-router: Serve the OpenAI Responses API on /v1/responses #42314, [sgl-router] Serve the Anthropic Messages API on /v1/messages #42316). They follow once those land.
• Presets for other vendors.

Accuracy Tests

N/A: no model or kernel change.

• Unit tests: byte sizes, sampling compiled through the flag parser, startup validation, every preset loads, extends merge and null-delete, cycle detection, selection order, output budget (inject / reject / clamp), image counting, and 400 retyping for both error shapes.
• Proxy tests (tests/proxy/api_profile.rs): budget injected, rejected and clamped on the wire; aliases; image limit; body limit; error type; the sampling section; and flag precedence.
• Checks: cargo test (lib and proxy) and cargo clippy --all-targets -D warnings pass.

Speed Tests and Profiling

Without a profile there is no new work on the request path. With one:

• the budget check reads fields already parsed;
• injecting a default is a splice;
• only max_images parses messages, and only when it is set.

Checklist

• Format your code according to the Format code with pre-commit.
• Add unit tests according to the Run and add unit tests.
• Update documentation according to Write documentations.
• Provide accuracy and speed benchmark results according to Test the accuracy and Benchmark the speed.
• Follow the SGLang code style guidance.

🤖 Generated with Claude Code

---

CI States

Latest PR Test (Base): ✅ Run #37096145028
Latest PR Test (Extra): ❌ Run #37096144904
Latest PR Test (AMD ROCm 10): ➖ No AMD PR run found for this commit.