[mickqian]

Summary

This PR reduces LingBot realtime serving latency in two places:

• Cache LingBot camera conditioner scale/shift for denoise timesteps and prefetch the per-block cached values once per model forward. Context-cache update forwards recompute instead of reusing the denoise cache, so the causal state update stays bitwise consistent.
• Send default realtime raw RGB as lossless raw bytes instead of delta-gzip, and split large payloads into a small msgpack header followed by raw bytes. The realtime collector supports both combined frame batches and header + payload messages.

Benchmark

Measured on a 4xH200 LingBot realtime run at 832x480, 6 chunks, using chunks 3/4/5 as steady-state.

Variant	Steady chunk total	Steady scheduler forward	raw payload build
main baseline	2360 ms	2252 ms	~20-22 ms/chunk
this PR	2114 ms	2092 ms	~1 ms/chunk

This is a 10.3% steady chunk total improvement and a 7.1% scheduler-forward improvement in the measured run.

Consistency

4xH200 realtime consistency matched the baseline bitwise:

• frames: [45, 480, 832, 3]
• sha256: e7692f77377df12aebed86b4193b33f0007bc8489757d32e41f9989a471c1d8d

Tests

python -m pytest \
  python/sglang/multimodal_gen/test/unit/realtime/test_realtime_output_transport.py \
  python/sglang/multimodal_gen/test/unit/realtime/test_lingbot_causal_denoising.py \
  -q

Result: 27 passed in latest CI-fix validation.

Additional CI OOM validation:

• Single-GPU LingBot realtime server case lingbot_world_realtime_plastic_beach: 1 passed in 115.69s.
• Fix scope: disable LingBot realtime request warmup in CI and wait for server_warmup_done on realtime websocket before accepting generation work.

---

CI States

Latest PR Test (Base): ✅ Run #26999422139
Latest PR Test (Extra): ✅ Run #26999422012

[gemini-code-assist]

Warning

You have reached your daily quota limit. Please wait up to 24 hours and I will start processing your requests again!

[mickqian]

/tag-and-rerun-ci

[mickqian]

/tag-and-rerun-ci

[mickqian]

/tag-and-rerun-ci

[mickqian]

/tag-and-rerun-ci extra