docs: Responses API background mode is live: background: true queues, runs behind its id, and can be cancelled

[github-actions]

What this PR does

When I send background: true to POST /v1/responses, does SMG answer a queued Response object at once and run the generation behind its id — polled with GET /v1/responses/{id} and stopped with POST /v1/responses/{id}/cancel — and what are the storage requirement, the streaming shape, the in-flight cap, and the cancel answers?

Why we need it

Source change: smg-project/smg@24dea1b

Commit 2 (#3095, 24dea1b6, 2026-10-11; supplied diff read, every claim re-verified on the current default-branch snapshot). Behavior: model_gateway/src/routers/grpc/common/responses/background.rs (module doc, :1-24) — a background response is answered before it is generated: the queued Response object (status queued, empty output, null usage, background: true, the request's parameters echoed; queued_response :187-194, shape pinned by test the_queued_object_has_the_public_shape) is stored under a fresh resp_ id and returned 200 at once (grpc/router.rs route_background_responses :511-618), the work runs in its own task, and the stored object moves queued -> in_progress -> completed | incomplete | failed | cancelled (drive/drive_watching :439-528; a work that ends without storing a result, or panics, leaves failed). Polling: GET /v1/responses/{id} reads the stored record, and GET /v1/responses/{id}/input_items answers from the first poll (store_record stores the input items with the queued record, :199-219). Streaming: with stream: true the stream carries a response.queued event right after response.created (event_types.rs ResponseEvent::Queued, :103-117; ResponseStatus::Queued in crates/protocols/src/responses.rs:2791), the work keeps running and is stored when the client leaves early (BackgroundStream::detach :574-607), and a cancelled stream ends without a terminal event (:609-614). Cancel: POST /v1/responses/{id}/cancel (background::cancel :626-720, wired for both pipelines via common/responses/handlers.rs:13-15) stops a response this gateway runs and answers the cancelled object; a queued/in_progress background record no local task runs (another gateway sharing the store, or lost to a restart) is closed as cancelled and the owner drops the work within its 2 s watch (STATUS_WATCH, tests at :1073-1123); a second cancel answers the same cancelled object; completed/incomplete answer 400 response_already_completed, failed 400 response_already_failed, and a non-background response 400 cancellation_not_supported (answer_for_closed :707-720). Validation: background: true with store: false is refused 400 invalid_value — an unstored response could not be polled (crates/protocols/src/responses.rs:3597-3610, test background_without_storage_is_rejected). Cap: at most --max-background-responses in flight per gateway (model_gateway/src/main.rs:1040-1044 and bindings/python/src/smg/router_args.py:343-351, default 1024); one beyond answers 429 background_responses_limit_reached (background.rs limit_reached :172-179); 0 switches the mode off and such requests run in the foreground (router.rs:420-451). Scope the page already frames: both the regular gRPC and the Harmony (gpt-oss) Responses paths serve it (router.rs:424-448); HTTP workers keep forwarding the body to the worker's own /v1/responses (routers/http/router.rs:2162-2171), matching responses.md:35-48's existing Backend Support framing. Stale claims (repo-wide grep for background: responses.md is the only canonical page): reference/api/responses.md:99-100 "background is accepted, but SMG does not run responses in the background: every response is returned synchronously or streamed" — false on main; :682 "gRPC workers: Background mode is not supported, so there is nothing to cancel..." — false; :708's example error message "Background mode is not supported. Synchronous and streaming responses cannot be cancelled." — no longer the message (background.rs:717-719 reads "Only responses created with 'background': true can be cancelled."). What stays: the cancellation_not_supported/response_already_completed/response_already_failed/response_not_found codes themselves, and the HTTP-workers cancel row (:683). Version framing: #3095 (2026-10-11) postdates the v1.12.0 bump (commit 219, #2888, 2026-10-08); the page already uses "new on main after v1.12.0 / in v1.12.0 and earlier" framing (:102-107, from merged PR #141) — present background mode the same way and keep #141's unknown-parameter and v1.12.0 caveats intact. Boundary: the --max-background-responses flag's reference-syntax row belongs to the deploy-config scan (configuration.md is not edited); this item documents the API behavior and names the flag only as the cap's control. One independent concern: one commit made one request parameter do what the public API does, corrected on the one page that documents that parameter and the cancel endpoint. Dedup: no open PRs exist; no closed PR mentions #3095 or background mode (merged #141 wrote the now-stale sentence before #3095 landed; merged #102 covered a different Responses input rule). Edit well under 1,000 lines.

Scope: responses-api / background-mode-queued-response. Other concerns are deferred.

How to test

• Passed the documentation path and size guard (under 1000 added plus deleted lines; no file-count limit).
• Passed an independent accuracy and single-concern review.
• Passed git diff --check and pnpm check and the production pnpm build.

Checklist

• Tests added/updated (if applicable)
• Docs updated (if applicable)
• Commit includes a DCO sign-off

[coderabbitai]

Important

Review skipped

Bot user detected.

To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

• Configuration used: Organization UI
• Review profile: CHILL
• Plan: Advanced
• Run ID: d58a609b-f491-4355-b9d0-6eba9b5f4520

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

• 🔍 Trigger review

• Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

_{Comment @coderabbitai help to get the list of available commands.}

[github-actions]

Documentation maintenance: retry-infrastructure. Unsuccessful content rounds: 0/3. Operational attempts: 1/3.

Head: 604d2d1236e71a769d7295067326ac64200b2659; reviewed main: 8df4f5b4b0429633d2a22f92c67a2c93640477d5.
Reviewed SMG source: unavailable.

A worker or publication step failed before the round completed. The content-repair budget was not charged. Inspect the linked workflow; no success verdict was recorded.

Workflow evidence

Human review threads and CODEOWNER approval remain under repository policy.

[XinyueZhang369]

This source change is not included in the latest stable release, v1.12.0. Public documentation will now target published stable releases only, without unreleased/current-main sections. Closing at the maintainer’s request. The docs-awaiting-release label marks this as deferred rather than declined, so the nightly workflow can rediscover the gap against a stable release and propose a focused, release-correct update.