[RGaonkar26]

Adds request_gate to client configs. When it is set, every model request through that endpoint waits for a slot of the host-wide gate from #2968. Also adds a vf-gate CLI to inspect and resize gates live.

Why
Several runs or many workers on one host can push a provider past its per-key concurrency limit (for example Prime Inference's 429 Too many concurrent requests for this model (limit: 32)). Main has no way to cap requests per model across processes, or to change a cap during a run.

Change

• BaseClientConfig.request_gate: RequestGateConfig | None = None. Because it is on the base config, it applies to [client], per-agent clients and judges, and to both eval and train clients.
• When it is set, build_async_httpx returns a GatedAsyncClient. A request whose JSON body names a model takes a permit before it is sent, and returns it once the response has been read. A stream holds its permit until it is closed or garbage-collected. Requests that name no model pass straight through. A process's first use of a gate, which may start the daemon, runs in a thread.
• Gate identity is a hash of the endpoint origin, the credential and the model. Anthropic's bare key and OpenAI's Bearer key share one gate.
• vf-gate (a new script) shows a table of model, cap, in flight, waiting, admitted and wait. It also takes --resize N --model M, --dir and --output json. A resize with no live gate exits non-zero.
• RequestGateConfig is exported from verifiers.v1.clients. There is a short section in docs/evaluation.md.

Behaviour

• Off by default. With request_gate unset, build_async_httpx returns the same plain httpx.AsyncClient as before, and verifiers.v1.request_gate is not imported.
• An unset gate is left out of dumps, so saved configs and traces still load in older verifiers.
• With a gate, a provider 429 still reaches the harness as it does today (feat(v1): adapt the request gate's limit to 429/529 #2970 adds absorption). clients/eval.py is unchanged.
• Prime Inference requires X-Prime-Team-ID. Verifiers' clients add it for the Prime Inference host; a gated client whose base_url points elsewhere at the same gateway must send it itself.

Tests

• Unit: 13 tests in tests/unit/request_gate/test_request_gate.py, against a local fake provider that answers 429 above its limit. They cover one cap shared by 4 processes, a live resize reaching running processes, vf-gate, streams, and a GC-collected stream not deadlocking. Unit suite: 253 passed, 18 skipped.
• Live (Prime Inference, 2026-10-23):
  • Gate unset: identical to main (plain client, no daemon, gate module not imported, nothing added to the config dump); 8/8 requests succeeded.
  • vf-gate --resize 6 mid-run from cap 2: in flight stayed at 2 or below before the resize and reached 6 after it; 48/48 requests succeeded.
  • A gated agent run (4 rollouts × 3 turns, cap 2) peaked at 2 upstream requests.
• Not tested live: real gateway 429s (deliberately not provoked on a shared gateway).

Notes

• Stacked on feat(v1): a host-wide request gate daemon #2968; merge after it. feat(v1): adapt the request gate's limit to 429/529 #2970 and feat(v1): model pools: bind each session to a deployment with room #2971 are stacked on this PR.
• Adds no new conflicts with open client PRs (Simplify dialects and reuse SDK inference transport #2675, feat(v1): share client transport timeout and retry settings #2698). With feat(v1): share client transport timeout and retry settings #2698's retry settings, each SDK retry takes its own permit.
• Ported from Parker's feat/pipelines (627154bf1, 9af804eac, dad67e0b4, and The request gate admits through one daemon per host #2946). Differences: no 429 absorption yet (feat(v1): adapt the request gate's limit to 429/529 #2970), and no ProviderRetryConfig. There is no MODEL_EXTENSION: the gate reads the body's model, so clients/eval.py stays untouched. An unset gate is left out of dumps, and vf-gate is new.
• On reconciling with The request gate admits through one daemon per host #2946: see the question to Parker in feat(v1): a host-wide request gate daemon #2968.

🤖 Generated with Claude Code

[RGaonkar26]

Closing in favour of Parker's #2946, which is the same request-gate / model-pool work and where it should land. Nothing on verifiers main or our pipeline needs it right now. The branch stays up: it has shared-host hardening (private 0700 dir, peer-uid checks, single-daemon lock, restart window), live tests against the Prime Inference gateway, and the pool poll backoff, all of which apply to #2946 and can be cherry-picked.