Add production runtime standard runner - #422
Merged
jarcherNV merged 14 commits intoAug 6, 2026
Merged
Conversation
Contributor
Greptile SummaryThe PR promotes the synchronous inference loop into the production runtime API and adds shared replay, MP4 output, and WebRTC demo infrastructure for OmniDreams.
Confidence Score: 5/5The PR appears safe to merge. No blocking failure remains; the previously reported loss of caller-provided per-step inputs is addressed by preserving Important Files Changed
Sequence DiagramsequenceDiagram
participant Demo as Demo/Caller
participant Runner as run_inference_session
participant Mapping as InputMapping
participant Runtime as InferenceRuntime
participant Session as InferenceSession
participant Output as OutputTarget
participant Metrics as MetricsRecorder
Demo->>Runner: adapter, config, inputs, mapping
Runner->>Mapping: validate compatibility
Runner->>Mapping: map global conditioning
Runner->>Runtime: create_runtime(config)
Runtime->>Session: start_session(mapped inputs)
Runner->>Output: open()
loop each StepRequest
Session-->>Runner: next_step_request()
Runner->>Mapping: map_step_inputs(canonical inputs, preserved step inputs)
Runner->>Session: step(step inputs)
Session-->>Runner: StepResult
Runner->>Output: write(result)
Runner->>Metrics: record timing
end
Runner->>Output: close()
Runner->>Session: close()
Runner->>Runtime: close()
Runner->>Metrics: close()
Reviews (18): Last reviewed commit: "Update runtime API migration plan" | Re-trigger Greptile |
Promote the test-only inference loop into flashdreams.runtime as run_inference_session. The runner validates mapping compatibility before runtime creation, maps global conditioning, drives a synchronous sequential session, writes step outputs, records timing metrics, and reliably closes output/session/runtime/metrics. Add focused production runner tests for success, validation ordering, cleanup, and declared mapping compatibility failures. Update runtime docs to mark T4 complete and point at the new runner entry point.
Pass the caller-provided initial step payload into per-step input mapping instead of always supplying an empty InferenceInput. This keeps IdentityInputMapping and other pass-through mappings compatible with sessions that require fixed per-step inputs. Add a regression test covering that path while still avoiding global conditioning on steady-state step calls.
Apply the import sorting changes required by the CPU pre-commit job for the runtime runner and its focused tests.
Apply the formatting changes produced by the CPU pre-commit job. This updates runtime mapping code and input-mapping tests to match Ruff format, resolving the ruff-format hook failure from PR CI.
Add the Phase 1 shared demo skeleton under flashdreams.runtime.demo with DemoSpec, output specs, PreparedScenario, and DemoAdapter shapes. Route replay demos through run_inference_session(), add shared output target construction, and shape WebRTC demo construction around the existing BaseWebRTCSessionManager. Cover the new boundary with fake-model tests, and update the demo API plan to port OmniDreams before LingBot.
jarcherNV
force-pushed
the
dev/jarcher/api-runtime
branch
from
August 6, 2026 05:45
3499fe0 to
5632e12
Compare
PyNvVideoCodec 2.1 only publishes wheels through CPython 3.12, so uv could select Python 3.13 and fail before the OmniDreams demo starts. Narrow the OmniDreams package Python range to <3.13 and refresh uv.lock.
jarcherNV
force-pushed
the
dev/jarcher/api-runtime
branch
from
August 6, 2026 07:33
9e6180e to
fa8a38f
Compare
Switch the shared OmniDreams demo default away from the perf preset so replay and WebRTC use the same stable non-perf preset as benchmark-style runs. Document perf as an explicit opt-in pending follow-up, and add a regression test for the CLI default.
Add a manual benchmark comparison for the legacy OmniDreams single-view runner and the experimental shared demo replay path. Document the copy-paste command and cover the shipped scenario file in benchmark harness tests.
jarcherNV
added a commit
that referenced
this pull request
Aug 8, 2026
Introduce the experimental runtime/session/input envelopes and a shared demo-level API for replay and WebRTC flows. Add the shared runner, output target plumbing, fake-model coverage, and benchmark hooks. Port OmniDreams replay and WebRTC onto the shared demo path via a thin model-owned adapter, add local/remote validation docs, and update the migration plan to track remaining output/stat work and legacy demo cleanup.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Promote the test-only inference loop into flashdreams.runtime as run_inference_session.
The runner validates mapping compatibility before runtime creation, maps global conditioning, drives a synchronous sequential session, writes step outputs, records timing metrics, and reliably closes output/session/runtime/metrics.
Add focused production runner tests for success, validation ordering, cleanup, and declared mapping compatibility failures. Update runtime docs to mark T4 complete and point at the new runner entry point.