Add modular FlashDreams application runner

[gtong-nv]

Architecture

application module              flashdreams-runner                 I/O mode
+------------------+       +--------------------------+       +---------------+
| create_runtime() | ----> | initialize Runtime       | ----> | Replay / MP4  |
|                  |       | create Session           |       | WebRTC        |
| Runtime          |       |                          |       | None          |
|  model weights   |       | loop:                    |       +-------+-------+
|  global state    |       |   input = mode.read()    |               |
|                  |       |   output = Session.      |<--------------+
| RunnerInference  |<------|       step(input)        |
| Session          |       |                          |
|  prompt/cache    |       |   mode.write(output)     |-------------->+
|  game state      |       | destroy Session/Runtime  |
+------------------+       +--------------------------+

[copy-pr-bot]

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

[csyonghe]

I feel like flashdreams_app shouldn't be called app, it is the runner that can run other apps. and it shouldn't be located in a place parallel to t2v_app. The runner is the "host", "t2v" is the thing that gets run by the host.

[gtong-nv]

yea, once the basic logic is implemented, this is easy to change.

[csyonghe]

so the main loop is sort of duplicated and implemented differently in the webrtc path and mp4 path. This is fine for now but could be cleaned up by abstracting native window/web rtc into objects, so you make the main loop implemented just once.

[gtong-nv]

with the current runtime API, looks like we have to separate it into branches

[csyonghe]

I don't think you can possibly express all forms of applications in a declarative way. An application should be defined by an object that provides init() and generate(), not a metadata/manifest.