[Model] GR00T-N1.7: support request-level batching

[hyunnnchoi]

Purpose

GR00T serving handles exactly one request at a time. Gr00tN1d7Pipeline never implemented the request-batch contract, so max_num_seqs is pinned to 1 and N connected robots just queue behind each other: throughput stays at ~6.6 req/s while latency grows linearly with N. The policy underneath doesn't have this limitation — Gr00tPolicy.get_action was written for batched (B, ...) observations from the start (unbatch, per-sample preprocess, collate, one model forward). This PR connects the two.

forward() now takes the whole scheduler wave, concatenates the observations along the batch axis, runs one get_action() call, and splits the actions back per request. Declaring supports_request_batch = True lets the engine accept max_num_seqs > 1. The bundled YAML keeps max_num_seqs: 1, so the default deployment is unchanged; opting in is an overlay:

base_config: vllm_omni/deploy/Gr00tN1d7.yaml
stages:
  - stage_id: 0
    max_num_seqs: 8

A malformed request in a wave fails alone instead of failing its neighbors, observations that cannot be concatenated (mismatched shapes) fall back to per-request calls, and a single-request wave takes the exact old code path.

The first commit fixes a plumbing problem the overlay would otherwise hit: since #5676, stage init overwrites the YAML's max_num_seqs with diffusion_batch_size (default 1), and vllm serve --omni has no way to set diffusion_batch_size, so the overlay reaches the engine as max_num_seqs=1. The fix is two guarded lines plus a regression test. I'm aware #6405 and #6484 touch the same area — happy to drop that commit and rebase on whichever lands first; the GR00T commit stands on its own.

Test Plan

vLLM Version: 0.26.0+cu129

vLLM-Omni Commit: based on 0e4b28f4

1xA30, DP=1, eager, closed-loop OpenPI WebSocket clients, 30 requests/client after 20 warmup, GR00T_NOISE_SEED=42. Actions were checked against the reference values in tests/e2e/online_serving/test_gr00t_openpi_expansion.py (atol 1e-2) before each sweep. Unit tests: tests/diffusion/models/gr00t/test_pipeline.py, tests/engine/test_async_omni_engine_stage_init.py.

Because batching changes what actually runs on the GPU, I also checked at the policy level (real checkpoint) that a request's actions do not depend on its batch neighbors: swap one observation's neighbor for a completely different one (images, state, prompt length) and the fixed row's actions come back bit-identical. Solo vs batched differs by up to 5.5e-2 in bf16 and by 1.0e-5 when the same comparison is re-run in fp32 — batch-size-dependent kernel rounding amplified by the 4 denoise steps, not a masking bug. For scale, one noise-seed step (42 to 43) moves the same actions by 7.9e-1.

Test Result

Bundled YAML on this branch: reference check passes (eef_9d 3.9e-3, gripper 3.9e-3, joint 7.5e-3) and the sweep reproduces main's serial behavior:

clients:      1      2      4      8
req/s:      6.56   6.68   6.71   6.77
mean ms:  152.3  296.9  589.2 1164.5

With the overlay (max_num_seqs: 8), reference check still passes on a single session, and:

clients:      1      2      4      8
req/s:      6.76   6.94  10.89  13.84
mean ms:  148.0  285.9  365.5  575.8

At 8 clients that is 2.0x the throughput at half the mean latency. A separate debug-logging run under the same 8-client load shows waves of 7 observations dominating (10 of 14 batched waves). Policy-level ceiling for reference: 6.97 obs/s at B=1, 16.74 obs/s at B=8.

Notes

• Waves only form from requests that queued while a forward was running (request_batch_max_wait_ms defaults to 0), so a single robot's latency is untouched — and two closed-loop clients still ping-pong at wave size 1, which is why the gain only shows from 4 clients up.
• With a fixed GR00T_NOISE_SEED, CUDA's seeded randn((B, ...)) does not reproduce the (1, ...) draw row by row, so bit-exact reproducibility holds per wave shape, not across load levels. Production serving runs unseeded. The CI golden test runs one session (wave size 1) and is unaffected.
• If this and [Model] GR00T-N1.7: add setup_compile() for the action head #6450 both land, the compile path's dynamic=False would recompile per distinct wave size; whichever lands second should switch batched deployments to dynamic=None. The recipe keeps the two opt-ins separate for now.

[chatgpt-codex-connector]

Codex usage limits have been reached for code reviews. Please check with the admins of this repo to increase the limits by adding credits.
Credits must be used to enable repository wide code reviews.

[vllm-omni-review-bot]

This PR appears to belong to: docs/design/module/diffusion/diffusion_model_integration.md.

Module owners: @fhfuih @Bounty-hunter @wtomin @RuixiangMa

@hyunnnchoi, please review your own changes and leave a short self-review comment describing what you checked. PRs without author self-review may not be assigned a reviewer.

Please take a look when you have a chance. If you would like an automated review, mention @vllm-omni-review-bot in a comment.

[hsliuustc0106]

@hyunnnchoi Are you planning to continue work on this PR? Please reply by 2026-10-19 10:47:09 UTC.

A yes or a brief plan is enough; code changes do not need to be finished within these three days. If you are waiting on a review or other maintainer action, please let us know.

If there is no reply, new commit, or maintainer takeover by then, we will recheck and close this PR as inactive. It can be reopened when work resumes.

[hyunnnchoi]

@hsliuustc0106 I won't be able to get back to this for a while, so closing it is fine. I'll reopen when I have time to work on it again.