Skip to content

Add production runtime standard runner - #422

Merged
jarcherNV merged 14 commits into
dev/jarcher/inference-runtime-apifrom
dev/jarcher/api-runtime
Aug 6, 2026
Merged

Add production runtime standard runner#422
jarcherNV merged 14 commits into
dev/jarcher/inference-runtime-apifrom
dev/jarcher/api-runtime

Conversation

@jarcherNV

Copy link
Copy Markdown
Collaborator

Promote the test-only inference loop into flashdreams.runtime as run_inference_session.

The runner validates mapping compatibility before runtime creation, maps global conditioning, drives a synchronous sequential session, writes step outputs, records timing metrics, and reliably closes output/session/runtime/metrics.

Add focused production runner tests for success, validation ordering, cleanup, and declared mapping compatibility failures. Update runtime docs to mark T4 complete and point at the new runner entry point.

@greptile-apps

greptile-apps Bot commented Aug 6, 2026

Copy link
Copy Markdown
Contributor

Greptile Summary

The PR promotes the synchronous inference loop into the production runtime API and adds shared replay, MP4 output, and WebRTC demo infrastructure for OmniDreams.

  • Validates input-mapping compatibility before runtime creation and guarantees ordered resource cleanup.
  • Preserves caller-provided per-step inputs across sequential session steps.
  • Adds shared demo specifications, replay and WebRTC launchers, video output support, focused tests, benchmark scenarios, and migration documentation.
  • Introduces the experimental omnidreams-demo CLI and updates the workspace to support Python versions below 3.13.

Confidence Score: 5/5

The PR appears safe to merge.

No blocking failure remains; the previously reported loss of caller-provided per-step inputs is addressed by preserving initial_inputs.step and metadata for each mapping call.

Important Files Changed

Filename Overview
flashdreams/flashdreams/runtime/runner.py Adds the validated sequential production runner, preserves initial per-step inputs, records timing metrics, and closes resources in deterministic order.
flashdreams/tests/test_runtime_runner.py Covers successful execution, per-step input preservation, validation ordering, compatibility rejection, metrics, and cleanup failures.
flashdreams/flashdreams/runtime/video_output.py Adds an MP4 output target that collects typed video step results and emits a video artifact with output metadata.
flashdreams/flashdreams/runtime/demo/replay.py Adds shared replay orchestration that prepares an adapter scenario and delegates execution to the production runner.
flashdreams/flashdreams/runtime/demo/webrtc.py Adds shared WebRTC runtime, manager, application, and server construction around model-specific adapters.
integrations/omnidreams/omnidreams/demo/adapter.py Adapts OmniDreams scenario preparation and runtime creation to the shared demo API.
integrations/omnidreams/omnidreams/demo/cli.py Introduces the experimental OmniDreams replay and WebRTC command-line entry point.
configs/omnidreams_demo_replay_benchmarks.json Adds manual one-minute benchmark scenarios comparing the legacy and shared OmniDreams replay paths.
uv.lock Regenerates the workspace lockfile after narrowing supported Python versions to below 3.13.

Sequence Diagram

sequenceDiagram
  participant Demo as Demo/Caller
  participant Runner as run_inference_session
  participant Mapping as InputMapping
  participant Runtime as InferenceRuntime
  participant Session as InferenceSession
  participant Output as OutputTarget
  participant Metrics as MetricsRecorder
  Demo->>Runner: adapter, config, inputs, mapping
  Runner->>Mapping: validate compatibility
  Runner->>Mapping: map global conditioning
  Runner->>Runtime: create_runtime(config)
  Runtime->>Session: start_session(mapped inputs)
  Runner->>Output: open()
  loop each StepRequest
    Session-->>Runner: next_step_request()
    Runner->>Mapping: map_step_inputs(canonical inputs, preserved step inputs)
    Runner->>Session: step(step inputs)
    Session-->>Runner: StepResult
    Runner->>Output: write(result)
    Runner->>Metrics: record timing
  end
  Runner->>Output: close()
  Runner->>Session: close()
  Runner->>Runtime: close()
  Runner->>Metrics: close()
Loading

Reviews (18): Last reviewed commit: "Update runtime API migration plan" | Re-trigger Greptile

Comment thread flashdreams/flashdreams/runtime/runner.py Outdated
Promote the test-only inference loop into flashdreams.runtime as
run_inference_session.

The runner validates mapping compatibility before runtime creation, maps global
conditioning, drives a synchronous sequential session, writes step outputs,
records timing metrics, and reliably closes output/session/runtime/metrics.

Add focused production runner tests for success, validation ordering, cleanup,
and declared mapping compatibility failures. Update runtime docs to mark T4
complete and point at the new runner entry point.
Pass the caller-provided initial step payload into per-step input mapping
instead of always supplying an empty InferenceInput.

This keeps IdentityInputMapping and other pass-through mappings compatible with
sessions that require fixed per-step inputs. Add a regression test covering that
path while still avoiding global conditioning on steady-state step calls.
Apply the import sorting changes required by the CPU pre-commit job for the
runtime runner and its focused tests.
Apply the formatting changes produced by the CPU pre-commit job.

This updates runtime mapping code and input-mapping tests to match Ruff format,
resolving the ruff-format hook failure from PR CI.
Add the Phase 1 shared demo skeleton under flashdreams.runtime.demo with
DemoSpec, output specs, PreparedScenario, and DemoAdapter shapes. Route replay
demos through run_inference_session(), add shared output target construction,
and shape WebRTC demo construction around the existing BaseWebRTCSessionManager.

Cover the new boundary with fake-model tests, and update the demo API plan to
port OmniDreams before LingBot.
@jarcherNV
jarcherNV force-pushed the dev/jarcher/api-runtime branch from 3499fe0 to 5632e12 Compare August 6, 2026 05:45
PyNvVideoCodec 2.1 only publishes wheels through CPython 3.12, so uv could
select Python 3.13 and fail before the OmniDreams demo starts. Narrow the
OmniDreams package Python range to <3.13 and refresh uv.lock.
@jarcherNV
jarcherNV force-pushed the dev/jarcher/api-runtime branch from 9e6180e to fa8a38f Compare August 6, 2026 07:33
Switch the shared OmniDreams demo default away from the perf preset so replay
and WebRTC use the same stable non-perf preset as benchmark-style runs. Document
perf as an explicit opt-in pending follow-up, and add a regression test for the
CLI default.
Add a manual benchmark comparison for the legacy OmniDreams single-view runner
and the experimental shared demo replay path. Document the copy-paste command
and cover the shipped scenario file in benchmark harness tests.
@jarcherNV
jarcherNV merged commit 8610c0f into dev/jarcher/inference-runtime-api Aug 6, 2026
6 checks passed
@jarcherNV
jarcherNV deleted the dev/jarcher/api-runtime branch August 6, 2026 09:40
jarcherNV added a commit that referenced this pull request Aug 8, 2026
Introduce the experimental runtime/session/input envelopes and a shared
demo-level API for replay and WebRTC flows. Add the shared runner, output
target plumbing, fake-model coverage, and benchmark hooks.

Port OmniDreams replay and WebRTC onto the shared demo path via a thin
model-owned adapter, add local/remote validation docs, and update the migration
plan to track remaining output/stat work and legacy demo cleanup.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant