Unify launch cmd by gtong-nv · Pull Request #438 · NVIDIA/flashdreams

gtong-nv · 2026-08-11T06:55:08Z

Summary

unify LingBot and OmniDreams demo launches behind flashdreams-run with manifest-driven modes
add omnidreams and omnidreams-perf runner aliases
organize launch manifests under configs/launch_manifest
fail immediately when a requested manifest path is missing or not a file

examples

flashdreams-run lingbot-world-fast webrtc --manifest lingbot-live.yaml
flashdreams-run lingbot-world-fast mp4 --manifest lingbot-replay.yaml
flashdreams-run omnidreams-sv-2steps-chunk2-loc6-lightvae-lighttae webrtc \
  --manifest omnidreams-drive.yaml
flashdreams-run omnidreams-sv-2steps-chunk2-loc6-lightvae-lighttae local-window \
  --manifest example_world_model_perf.yaml
flashdreams-run wan21-t2v-1.3b-480p run --prompt "..."

Signed-off-by: Gangzheng Tong <gtong@nvidia.com>

copy-pr-bot · 2026-08-11T06:55:12Z

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

ArielG-NV

Review; questions, suggestions, one note.

ArielG-NV · 2026-08-11T15:09:07Z

+  pixel_height: 464
+
+output:
+  path: ../../outputs/lingbot-replay.mp4


should be outputting to somewhere in `/artifacts/...>

ArielG-NV · 2026-08-11T15:10:02Z

@@ -0,0 +1,14 @@
+schema_version: 1


why are our "launch configs" in docs. docs is for documentation?

Those are just example configs. We can move that to configs

ArielG-NV · 2026-08-11T15:15:10Z

+schema_version: 1
+runner: lingbot-world-fast
+mode: mp4
+
+scenario:
+  example_data: true
+  example_idx: 0
+  total_blocks: 20
+  pixel_width: 832
+  pixel_height: 464
+
+output:
+  path: ../../outputs/lingbot-replay.mp4
+  fps: 16


Where do we specify the target checkpoint we want to use? (and model-arch - although this requires the input/output API to be complete)

If lingbot releases a new hypothetical checkpoint that is 0.1b, and its compatible with their existing model-arch, we should be-able to select the checkpoint (either: hugging-face link, or local-path?)

This is not the scope of this PR. If a checkpoint can be easily swapped, it should be the demo's parsing logic to recognize that and construct the pipeline based on the input yaml

ArielG-NV · 2026-08-11T15:19:02Z

+Set ``output.profile_world_model: true`` to enable FlashDreams CUDA-event
+profiling for the world-model runtime. Set ``output.sync_gpu_timing: true`` only
+when you need raster compute
 timings; it synchronizes GPU work and is not a throughput setting.


Suggested change

Set ``output.profile_world_model: true`` to enable FlashDreams CUDA-event

profiling for the world-model runtime. Set ``output.sync_gpu_timing: true`` only

when you need raster compute

timings; it synchronizes GPU work and is not a throughput setting.

Set ``output.profile_world_model: true`` to enable FlashDreams CUDA-event

profiling for the world-model runtime. Set ``output.sync_gpu_timing: true`` to synchronize GPU events to accurately compute timings for raster operations.

What is only when you need raster compute timings?

there is some async/overlap in raster and world model generation, which makes the raster profile unreliable. Set output.sync_gpu_timing: true will make things in order, but will hurt the overall performance. this is only needed for debugging raster

ArielG-NV · 2026-08-11T15:25:44Z

+`webrtc` launch mode. See the
 **[interactive demo guide](https://nvidia.github.io/flashdreams/main/models/omnidreams.html#launch-the-interactive-demo)**.


Note: we will need to update the website accordingly.

Will do another round of updating

Signed-off-by: Gangzheng Tong <gtong@nvidia.com>

greptile-apps · 2026-08-11T20:40:07Z

Greptile Summary

The PR unifies LingBot and OmniDreams demo startup behind flashdreams-run, adding manifest-driven launch modes and stable OmniDreams runner aliases.

Adds shared launch-manifest parsing, validation, path resolution, and mode dispatch.
Migrates integration launch adapters, CI workflows, benchmarks, tests, and documentation to the unified interface.
Moves checked-in manifests under configs/launch_manifest; their ../../ paths now resolve from that directory to repository-root locations.

Confidence Score: 5/5

The PR appears safe to merge.

No blocking failure remains; the previously reported path traversal was corrected by relocating the manifests under configs/launch_manifest, where the existing ../../ paths resolve to the intended repository-root targets.

Important Files Changed

Filename	Overview
flashdreams/flashdreams/scripts/cli.py	Adds positional launch-mode parsing, manifest integration, mode-aware help, and dispatch through the shared launch API.
flashdreams/flashdreams/serving/launch_manifest.py	Introduces strict manifest loading and resolves relative path-valued fields from the manifest directory.
flashdreams/flashdreams/serving/launch.py	Defines the common launch capability and resolved-launch contracts used by integration-owned modes.
integrations/omnidreams/omnidreams/launch.py	Adapts OmniDreams MP4, null, WebRTC, and local-window modes to the unified launcher.
integrations/lingbot/lingbot/launch.py	Adapts LingBot MP4 and WebRTC launches to the shared capability interface.
configs/launch_manifest/omnidreams_local_window.yaml	References the world-model manifest with a path that now correctly resolves to the repository’s integration directory.
configs/launch_manifest/omnidreams_mp4.yaml	Defines the default OmniDreams replay settings with an output path resolving to the repository-root outputs directory.
configs/launch_manifest/lingbot_mp4.yaml	Defines the LingBot replay settings with an output path resolving to the repository-root outputs directory.

Flowchart

%%{init: {'theme': 'neutral'}}%%
flowchart LR
  User["flashdreams-run runner mode"] --> CLI["CLI argument normalization"]
  CLI --> Manifest["Load and validate launch manifest"]
  Manifest --> Config["Apply runner and scenario overrides"]
  Config --> Capability["Resolve integration launch capability"]
  Capability --> Target{"Launch mode"}
  Target --> Run["Standard runner"]
  Target --> MP4["MP4 / null output"]
  Target --> WebRTC["WebRTC server"]
  Target --> Window["OmniDreams local window"]

_{Reviews (3): Last reviewed commit: "Fail loudly for missing launch manifests" | Re-trigger Greptile}

greptile-apps · 2026-08-11T20:40:10Z

+  preload_scenes: false
+
+output:
+  world_model_manifest_path: ../../integrations/omnidreams/omnidreams/interactive_drive/configs/example_world_model_perf.yaml


Manifest paths escape repository

When the checked-in local-window or MP4 manifests are loaded, their ../../ paths resolve from the configs directory to locations outside the repository, causing local-window startup to miss its required world-model manifest and MP4 output to be written outside the documented output directory. The same extra parent traversal occurs in configs/lingbot_mp4.yaml and configs/omnidreams_mp4.yaml.

Signed-off-by: Gangzheng Tong <gtong@nvidia.com>

gtong-nv · 2026-08-11T21:04:38Z

/ok to test 8b76cd8

Signed-off-by: Gangzheng Tong <gtong@nvidia.com>

Unify demo launch commands

7672a23

Signed-off-by: Gangzheng Tong <gtong@nvidia.com>

gtong-nv changed the base branch from main to dev/jarcher/inference-runtime-api August 11, 2026 06:55

ArielG-NV reviewed Aug 11, 2026

View reviewed changes

gtong-nv changed the title ~~Dev/gtong/unify launch cmd~~ Unify launch cmd Aug 11, 2026

Move launch manifests to configs

454ac03

Signed-off-by: Gangzheng Tong <gtong@nvidia.com>

greptile-apps Bot reviewed Aug 11, 2026

View reviewed changes

Add OmniDreams launch aliases

8b76cd8

Signed-off-by: Gangzheng Tong <gtong@nvidia.com>

Fail loudly for missing launch manifests

7c01e70

Signed-off-by: Gangzheng Tong <gtong@nvidia.com>

gtong-nv merged commit 895922d into dev/jarcher/inference-runtime-api Aug 11, 2026
3 checks passed

gtong-nv deleted the dev/gtong/unify-launch-cmd branch August 11, 2026 21:31

		`webrtc` launch mode. See the
		[interactive demo guide](https://nvidia.github.io/flashdreams/main/models/omnidreams.html#launch-the-interactive-demo).

Uh oh!

Conversation

gtong-nv commented Aug 11, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Uh oh!

copy-pr-bot Bot commented Aug 11, 2026

Uh oh!

ArielG-NV left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

greptile-apps Bot commented Aug 11, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Greptile Summary

Confidence Score: 5/5

Important Files Changed

Flowchart

Uh oh!

greptile-apps Bot Aug 11, 2026

Choose a reason for hiding this comment

Uh oh!

gtong-nv commented Aug 11, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

gtong-nv commented Aug 11, 2026 •

edited

Loading

ArielG-NV left a comment •

edited

Loading

greptile-apps Bot commented Aug 11, 2026 •

edited

Loading