[WIP][Diffusion] LingBot-VA world-action model support

[KevinZeng08]

Purpose

This PR enables LingBot-VA world-action model inference in vllm-omni, currently work in progress.

LingBot-VA is an autoregressive world-action model that jointly predicts future video latents and robot actions.

Related issue: #1987 #2162

Roadmap

• I2VA (Image to Video-Action) mode for offline inference
• Server mode for online inference
  • Integrate Robotwin and LIBERO simulation environments
  • Integrate API server support (reference: [Diffusion] DreamZero world model integration with CFG parallel + OpenPI serving #2162)
  • Add interface to receive simulation-environment outputs
• Performance
  • Support CFG parallel
  • Support TP
• Refactor internal state management
  • Expose KV cache and related inference-state data to upper layers
  • Explore integration with vLLM KV cache management

Test Plan

• I2VA mode offline inference video-action output
• Server mode output

Test Result

Text prompt: Grab the medium-sized white mug, rotate it, place it on the table, and hook it onto the smooth dark gray rack.

vllm-omni (I2VA offline inference)	LingBot-VA official repo (I2VA)
vllm-omni-lingbot-va.mp4	lingbot-va-official.mp4

Future Work

• Support for more world-action models (e.g. FastWAM)
• Explore to unify the framework of autoregressive world-action models, which share design similarities
  • (A) includes DreamZero, (B) includes LingBot-VA, (C) includes FastWAM

Source: FastWAM

[hsliuustc0106]

This PR is marked as WIP. Ready for full review when draft status is removed.

Note: DCO check is currently failing. Please resolve before requesting review.