Add "Crazy Robotaxi" standalone game demo without using new API - #463
Draft
aidanfnv wants to merge 52 commits into
Draft
Add "Crazy Robotaxi" standalone game demo without using new API#463aidanfnv wants to merge 52 commits into
aidanfnv wants to merge 52 commits into
Conversation
This was referenced Aug 18, 2026
This was referenced Aug 19, 2026
Signed-off-by: Aidan Foster <aidanf@nvidia.com>
Signed-off-by: Aidan Foster <aidanf@nvidia.com>
Signed-off-by: Aidan Foster <aidanf@nvidia.com>
Signed-off-by: Aidan Foster <aidanf@nvidia.com>
Signed-off-by: Aidan Foster <aidanf@nvidia.com>
Signed-off-by: Aidan Foster <aidanf@nvidia.com>
Signed-off-by: Aidan Foster <aidanf@nvidia.com>
Signed-off-by: Aidan Foster <aidanf@nvidia.com>
Signed-off-by: Aidan Foster <aidanf@nvidia.com>
Signed-off-by: Aidan Foster <aidanf@nvidia.com>
Signed-off-by: Aidan Foster <aidanf@nvidia.com>
Signed-off-by: Aidan Foster <aidanf@nvidia.com>
Signed-off-by: Aidan Foster <aidanf@nvidia.com>
Signed-off-by: Aidan Foster <aidanf@nvidia.com>
Signed-off-by: Aidan Foster <aidanf@nvidia.com>
Signed-off-by: Aidan Foster <aidanf@nvidia.com>
Signed-off-by: Aidan Foster <aidanf@nvidia.com>
Signed-off-by: Aidan Foster <aidanf@nvidia.com>
Signed-off-by: Aidan Foster <aidanf@nvidia.com>
Signed-off-by: Aidan Foster <aidanf@nvidia.com>
Signed-off-by: Aidan Foster <aidanf@nvidia.com>
Signed-off-by: Aidan Foster <aidanf@nvidia.com>
Signed-off-by: Aidan Foster <aidanf@nvidia.com>
Signed-off-by: Aidan Foster <aidanf@nvidia.com>
Signed-off-by: Aidan Foster <aidanf@nvidia.com>
Signed-off-by: Aidan Foster <aidanf@nvidia.com>
Signed-off-by: Aidan Foster <aidanf@nvidia.com>
Signed-off-by: Aidan Foster <aidanf@nvidia.com>
Signed-off-by: Aidan Foster <aidanf@nvidia.com>
Signed-off-by: Aidan Foster <aidanf@nvidia.com>
Signed-off-by: Aidan Foster <aidanf@nvidia.com>
Signed-off-by: Aidan Foster <aidanf@nvidia.com>
Signed-off-by: Aidan Foster <aidanf@nvidia.com>
Signed-off-by: Aidan Foster <aidanf@nvidia.com>
Signed-off-by: Aidan Foster <aidanf@nvidia.com>
Signed-off-by: Aidan Foster <aidanf@nvidia.com>
Signed-off-by: Aidan Foster <aidanf@nvidia.com>
Signed-off-by: Aidan Foster <aidanf@nvidia.com>
Signed-off-by: Aidan Foster <aidanf@nvidia.com>
Signed-off-by: Aidan Foster <aidanf@nvidia.com>
Signed-off-by: Aidan Foster <aidanf@nvidia.com>
Signed-off-by: Aidan Foster <aidanf@nvidia.com>
Signed-off-by: Aidan Foster <aidanf@nvidia.com>
Signed-off-by: Aidan Foster <aidanf@nvidia.com>
Signed-off-by: Aidan Foster <aidanf@nvidia.com>
Signed-off-by: Aidan Foster <aidanf@nvidia.com>
Signed-off-by: Aidan Foster <aidanf@nvidia.com>
Signed-off-by: Aidan Foster <aidanf@nvidia.com>
Signed-off-by: Aidan Foster <aidanf@nvidia.com>
Signed-off-by: Aidan Foster <aidanf@nvidia.com>
Signed-off-by: Aidan Foster <aidanf@nvidia.com>
Signed-off-by: Aidan Foster <aidanf@nvidia.com>
aidanfnv
force-pushed
the
dev/aidanf/game/crazy-robotaxi-standalone
branch
from
August 21, 2026 01:07
be12871 to
8fd5362
Compare
aidanfnv
added a commit
that referenced
this pull request
Aug 26, 2026
…cles, weather (stacked on #463) (#494) * Initial impl Signed-off-by: Aidan Foster <aidanf@nvidia.com> * More overlays Signed-off-by: Aidan Foster <aidanf@nvidia.com> * Add handbrake and pedal brake->reverse behaviour Signed-off-by: Aidan Foster <aidanf@nvidia.com> * Improve pickup dropoff placements Signed-off-by: Aidan Foster <aidanf@nvidia.com> * add global timer and scoreboard Signed-off-by: Aidan Foster <aidanf@nvidia.com> * minor improvements Signed-off-by: Aidan Foster <aidanf@nvidia.com> * Fix idling speed Signed-off-by: Aidan Foster <aidanf@nvidia.com> * Show highscore in HUD, remove pickup bonus time Signed-off-by: Aidan Foster <aidanf@nvidia.com> * Faster acceleration Signed-off-by: Aidan Foster <aidanf@nvidia.com> * Make pickup/dropoffs more random Signed-off-by: Aidan Foster <aidanf@nvidia.com> * Fix taxi integration on PhysX baseline Signed-off-by: Aidan Foster <aidanf@nvidia.com> * Keep taxi driveable after collisions Signed-off-by: Aidan Foster <aidanf@nvidia.com> * Strengthen curb steering and reverse transition Signed-off-by: Aidan Foster <aidanf@nvidia.com> * Make taxi handling more arcade-like Signed-off-by: Aidan Foster <aidanf@nvidia.com> * Place initial taxi pickup ahead of player Signed-off-by: Aidan Foster <aidanf@nvidia.com> * Make braking and handbrake turns arcade-sharp Signed-off-by: Aidan Foster <aidanf@nvidia.com> * Make curb recovery follow arcade steering Signed-off-by: Aidan Foster <aidanf@nvidia.com> * Keep collision physics aligned with world rendering Signed-off-by: Aidan Foster <aidanf@nvidia.com> * Keep vehicle attitude and steering responsive Signed-off-by: Aidan Foster <aidanf@nvidia.com> * Reduce traffic and bevel vehicle collision shapes Signed-off-by: Aidan Foster <aidanf@nvidia.com> * Synchronize native taxi markers with displayed frames Signed-off-by: Aidan Foster <aidanf@nvidia.com> * Make arcade steering responsive and consistent Signed-off-by: Aidan Foster <aidanf@nvidia.com> * Unify presented state with authoritative physics frames Signed-off-by: Aidan Foster <aidanf@nvidia.com> * Keep physics yaw within world model conditioning Signed-off-by: Aidan Foster <aidanf@nvidia.com> * Support Python 3.10 high-score timestamps Signed-off-by: Aidan Foster <aidanf@nvidia.com> * Keep ego heading on the conditioning trajectory Signed-off-by: Aidan Foster <aidanf@nvidia.com> * Restore progressive keyboard steering Signed-off-by: Aidan Foster <aidanf@nvidia.com> * Vary taxi pickups after the first fare Signed-off-by: Aidan Foster <aidanf@nvidia.com> * Cap initial pickup distance Signed-off-by: Aidan Foster <aidanf@nvidia.com> * Isolate taxi physics from Ludus renderer Signed-off-by: Aidan Foster <aidanf@nvidia.com> * Move taxi game into crazy_robotaxi package Signed-off-by: Aidan Foster <aidanf@nvidia.com> * Restore Crazy Robotaxi physics policy Signed-off-by: Aidan Foster <aidanf@nvidia.com> * Isolate Crazy Robotaxi from Interactive Drive Signed-off-by: Aidan Foster <aidanf@nvidia.com> * Add routed turn guidance to Crazy Robotaxi Signed-off-by: Aidan Foster <aidanf@nvidia.com> * Improve Crazy Robotaxi intersection guidance Signed-off-by: Aidan Foster <aidanf@nvidia.com> * Synchronize Crazy Robotaxi BEV frames Signed-off-by: Aidan Foster <aidanf@nvidia.com> * Expand Crazy Robotaxi pickup choice Signed-off-by: Aidan Foster <aidanf@nvidia.com> * Remove Crazy Robotaxi turn guidance Signed-off-by: Aidan Foster <aidanf@nvidia.com> * Limit visible Crazy Robotaxi pickups Signed-off-by: Aidan Foster <aidanf@nvidia.com> * Keep Crazy Robotaxi fares inside map bounds Signed-off-by: Aidan Foster <aidanf@nvidia.com> * Align Crazy Robotaxi generated frames with world state Signed-off-by: Aidan Foster <aidanf@nvidia.com> * Fix Crazy Robotaxi native BEV targets Signed-off-by: Aidan Foster <aidanf@nvidia.com> * Point pickup compass at nearest target Signed-off-by: Aidan Foster <aidanf@nvidia.com> * Prefer longer Crazy Robotaxi fares Signed-off-by: Aidan Foster <aidanf@nvidia.com> * Keep taxi targets farther from map edges Signed-off-by: Aidan Foster <aidanf@nvidia.com> * Enclose the Crazy Robotaxi play area Signed-off-by: Aidan Foster <aidanf@nvidia.com> * Fix Crazy Robotaxi play-area enclosure Signed-off-by: Aidan Foster <aidanf@nvidia.com> * Enclose interior road-network boundaries Signed-off-by: Aidan Foster <aidanf@nvidia.com> * Add pedestrians to Crazy Robotaxi pickups Signed-off-by: Aidan Foster <aidanf@nvidia.com> * Place Robotaxi stops along road edges Signed-off-by: Aidan Foster <aidanf@nvidia.com> * Extract Crazy Robotaxi as a standalone app Signed-off-by: Aidan Foster <aidanf@nvidia.com> * Fix Crazy Robotaxi runner manifest options Signed-off-by: Aidan Foster <aidanf@nvidia.com> * Add playable live-edit abilities to Crazy Robotaxi (skins + coins) Wire the flag-gated live_edit package into the composition root: - CLI: --live-edit-* flags registered in runtime_cli.build_parser and plumbed through cli._build_application / runtime_cli.run into CrazyRobotaxiApp(live_edit_config=...). - Session: install_style_ability_on_backend swaps in a CUDA-graph-free flashdreams session when the drift corrector is configured and attaches the text-edit LoRA + corrector after model warmup. - Presenter: CausalFrameAlignmentPresenter(LiveEditPresenter(inner)) so coin compositing sees frame-aligned poses; pixel-drawn SKIN / COINS chips; per-rollout coin ability bound via set_coin_ability. - Input: K cycles the world skin, C toggles coins, in both the native HUD and the MJPEG /control path; rising-edge requests live on CrazyRobotaxiKeyboardState.live_edit and are drained each tick by CrazyRobotaxiRuntime.process_events. - Coins: course laid along ALL driving-lane centerlines (previously only allows_taxi_stops lanes, which kept coins outside the pickup radius); pickups run on the authoritative per-frame trajectory states. - Style: duty-cycled re-swap (--live-edit-style-reswap-chunks, default 8) re-issues the active skin's replace_text so long holds stay crisp, respecting the finalize -> replace_text ordering. Validated end-to-end on GPU via the headless MJPEG app: photoreal start, mid-run key-cycle to cyberpunk at chunk 9, re-swaps at 17/25 keeping the style crisp through chunk 32, 57 coin pickups on-route. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * Add obstacle and weather live-edit events to Crazy Robotaxi Obstacle events (O key): clone a real moving vehicle track from the scene's perception tracks, retime it to now, and rigidly shift it ahead of the ego through the advance_frames dynamic-actor seam (synthetic boxes render in the conditioning but never materialize; clones do). Merging same-direction templates at city speed are preferred. Optional box-axis guidance (validated operating point s=2.0: in-box |diff| ~18 vs ~7 unguided, out-box clean) runs a shadow encoder cache over obstacle-free conditioning and doubles predict_flow during events. Weather events (V key): clear -> rain -> snow -> clear via plain two-prompt guided swaps (2.5/20, the PR #431 mechanism); the style LoRA is detached around weather-only swaps and the drift corrector is gated off during weather (its gate profile was calibrated on style v6). Skin + weather compose into one compositional prompt through the LoRA. Presenter gains WEATHER/OBSTACLE/HITS chips and an optional projected 3D-box annotation for the obstacle clone (evidence aid). 29 new CPU tests (weather cycle/LoRA-bypass/corrector policy, template extraction/placement/despawn/collision, guidance seams, key plumbing). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * Improve live-edit coin compositing: aspect-true sizing, contact shadow, edge clipping Sizing keeps the FTheta vertical-extent projection for height, preserves the sprite's native aspect ratio, and carries the spin squash on width. Adds a light elliptical contact shadow (kept subtle so the coin pops) and edge-clipped compositing so large near-edge sprites can't raise. The default coin stays procedural; a custom RGBA sprite can be supplied with --live-edit-coin-sprite. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * Pad the coin contact shadow past its blur radius The 2 px padding clipped the Gaussian falloff into a faint rectangle on flat road textures (visible in the cyberpunk demo stills). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * Make weather a base-world-only ability with a corrector-gain dispatch Skin+weather combo prompts produced rain that read as no rain under the neon skins, and the corrector hard-off rule left over half the demo run drifting. Weather now only runs over the base world: the V key is rejected while a skin is active or pending, activating a skin clears an active weather, and compose_swap_target rejects the combo outright. WeatherPreset drops the combo clause and front-loads visible-rain cues (streaks in the air, droplets on the windshield/lens, tire spray) in the standalone prompt. The corrector's boolean dispatch becomes a per-state gain dispatch: the style gain rides the validated gated predict_flow, gain 0 short-circuits to the bit-clean base forward, and any other gain re-derives the alpha*(t) x gain LoRA scale before the base forward. New flags: --live-edit-weather-guidance (2.5 default; snow needed 3.0 in sweeps) and --live-edit-weather-corrector-gain (0 default = corrector off during weather, the calibrated-safe behavior). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * Add storm preset, stronger snow prompt, and direct weather select - Rewrite the snow preset with front-loaded falling-precipitation cues (heavy snowfall, flakes streaking past the windshield, accumulation on the hood/roadsides) after the 2.5-guidance capture read as a dusting; A/B probes picked guidance 3.0 over 3.5 (3.5 added sky smear without more snow). - Add an experimental hurricane-force 'storm' preset (appearance cues expected to land; dynamic wind effects documented as unlikely). - Add --live-edit-weather-first to rotate the V-key cycle so any preset is one confirmed press away (no pass-through transition weather). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * Port fused drift-corrector mode and per-state dispatch from omnidreams-live-edit Brings 671d9f99 (CUDA-graph-safe fused mode) and 39fe3af1 (DriftCorrectorDispatch + TextEditLoRA.release_targets) into the game branch; library tests live on omnidreams-live-edit. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * Wire the fused per-state drift corrector into the live-edit abilities corrector_mode=fused (default; LIVE_EDIT_CORRECTOR_MODE / --live-edit-corrector-mode fall back to unfused) deploys DriftCorrectorDispatch with per-(base|skin|weather) pre-merged sets: base keeps an optional photoreal corrector, the skin state composes the released self-attn LoRA delta with the style corrector in one copy_ source, weather reuses the style (or a dedicated) corrector at its own gain. compile_network + use_cuda_graph stay ON in fused mode; the graph-free session rebuild now happens only for unfused. The K/V state machine selects the dispatch state at chunk boundaries. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * Gate live-edit native-DIT rejection on real hooks; add corrector mode off and guidance-window knobs Fixes the three perf problems reported against PR #494 + #496: - --live-edit-corrector-mode off disables every drift corrector even when checkpoints are configured: no DriftCorrectorDispatch is built, no weight sets are snapshotted, and the scheduler/finalize gate drivers are never installed (weights verified bit-identical in tests). - The native-DIT guard is now precise and actionable: it fires only for the prompt-swap abilities that genuinely need the Python transformer forward (replace_text_embeddings raises NotImplementedError under the native executor, and TextEditLoRA's copy_ toggles never reach the native fp8 weight snapshot), names the exact flags to drop, and only mentions corrector flags when a corrector is actually enabled. Coins and other pixel-only abilities never construct the ability and run under native_dit_acceleration untouched. - New knobs --live-edit-skin-guidance-chunks (default 6) and --live-edit-weather-guidance-chunks (default 20). The skin window is LoRA-realized single-branch (length is not a per-chunk cost); a GPU A/B on a cyberpunk swap showed 6 lands the style as fast and as strong as 20. Weather has no LoRA: its guided window costs ~2x per chunk while open, and the 8-chunk re-swap refresh re-opens it - now documented in the README with the knob to shorten it. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * Composite live-edit pixels on the GPU when the model frame is CUDA-resident The presenter used to materialize the model frame to host numpy for PIL compositing, which on the native Vulkan HUD fast path (lazy CUDA uint8 HWC frames) forced a GPU->CPU->GPU round trip per frame (~10 fps observed). LiveEditFrameCompositor keeps the frame on device: coin sprites, contact shadows, and HUD chips are pre-rendered once (PIL) and cached as premultiplied per-size textures; the per-frame work is one float32 conversion, a fused lerp ROI blend or two per coin, and a single round/clamp/uint8 cast (plus an optional separable-Gaussian unsharp for skins). Measured on GB300 at 1280x704: 0.09 ms/frame with 0 coins, 0.59 ms with 9 coins (~4.7 ms per 8-frame chunk), issued from the presenter thread so it overlaps model compute. The composited result is re-wrapped as a LazyCudaFrame whose CUDA event is recorded after the blends, so both consumers stay ordered (Vulkan interop copy stream, MJPEG host prefetch) and the single-materialization numpy-XOR-cuda contract holds. Host/numpy sources keep the PIL path, as does the obstacle box-outline annotation (debug aid). All compositor math is device-agnostic and unit-tested on CPU tensors. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * Fix ruff findings in the live-edit branch files obstacle_ability/test_runner import-block ordering (repo ruff 0.12.7 --select I), C408 dict literals, PLR0402/RUF059/RUF046/UP037 from the current ruff default set on the live-edit files. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * Add --stream-token gate to the MJPEG streaming presenter Every HTTP endpoint (page, /stream, /bev_stream, /control, /state, /scenes, /thumbnail, /scene/select, /taxi/name) now requires the shared token via ?token= query param or X-Stream-Token header when the flag is set; requests without a valid token get 403. The served page reads the token from its own URL and appends it to all of its requests. Omitting the flag keeps the historical open behavior. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * Add demo_kit: hosted-demo launch scripts and runbook Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * Add hurricane weather preset (kill-gated vs shipped storm) Escalates storm along the cues that materialize (2026-08-21 A/B at guidance 3.5/4.0 vs the shipped storm frames): visibility collapse, walls of spray/mist, static debris lying on the flooded road, and a black-green emergency-gloom sky. Dynamic wind wording (bending trees, flying debris) is deliberately absent — it never materializes in the history-anchored model. Shipped at weather guidance 4.0. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * Bound the coin ability's per-frame cost at course scale Aidan reported coins still hurting native-path perf after the CUDA compositor fix. Profiling at real course scale (4395 coins on the shipped suburb scene) found two costs the 9-coin measurement missed: - visible_sprites/advance_frames scanned all coins every frame (0.60 ms/frame CPU at 4400 coins). A static 32 m spatial hash with per-cell cached windows plus a vectorized projection filter makes both O(nearby): 0.20 ms/frame, pickup 0.10 -> 0.015 ms. - The compositor is CPU-launch-bound, not bandwidth-bound (GPU drain adds ~0 in every measurement), and dense areas put 211 sprites inside the 120 m render radius -> 11-14 ms/frame enqueue. Fixes: one blend per coin (shadow+coin+quantized fade pre-composited into one cached texture, 2.6x fewer kernels: 1.14 -> 0.44 ms at 9 sprites, 5.20 -> 1.84 ms at 48) and a nearest-N sprite cap (--live-edit-coin-max-visible, default 64) so dense courses cannot blow the budget. Also adds the ROI-only uint8 blend path (LIVE_EDIT_COMPOSITOR=roi) that never converts the full frame to float32 - exact parity with the float path (max diff 0) and minimal GPU traffic, but slower wall-clock on launch-bound machines, so the float canvas stays the default - and a perf self-report (--live-edit-perf-log N / LIVE_EDIT_PERF_LOG) that logs p50/p95 coin CPU ms, compositor enqueue CPU ms, and compositor GPU ms (lazy CUDA event pairs, no added sync) every N composited frames so remote users can send numbers instead of guesses. Validated headless MJPEG at course scale: chunk cadence unchanged (219-242 ms, matching the no-coins baseline), coin rendering parity vs the previous compositor within 1.5 LSB mean on sprite pixels (fade quantization + <=1 px shadow anchor). New tests cover the windowed culling, the sprite cap, ROI/float parity, and the perf log. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * Add traffic events: multi-clone obstacle bursts One O press now spawns --live-edit-obstacle-count clones (default 1 = unchanged single obstacle): distinct crossing/oncoming template tracks (never pace-matched lead cars, which render at ghost strength), staggered across a 16-40 m band ahead of the ego (spacing_m per slot) and --live-edit-obstacle-stagger-chunks apart in time, each retimed+rigidly shifted with ground-z correction and despawning after its own pass. Guidance already strips every OBSTACLE_ENTITY_PREFIX actor from the shadow branch, so the box-axis path handles simultaneous boxes unchanged. Mask-anchored validation (RNG-matched baseline, per-clone camera-projected boxes, 26-chunk headless runs): s=2.0 lands 2/3 clones at N=3 (52/61/18% of real-car in-box benchmark) and 4/5 at N=5 (52/60/15/69/83%), out-box clean (median 5.7-7.5); s=2.5 is stronger still at N=3 (63/79/22%) with no breakup, so the traffic demo ships at s=2.5. Simultaneous spawns (stagger 0) are stable; the 1-chunk default is for on-screen pacing. The one weak clone is the fastest template (~11 m/s) which holds distance until late. The presenter annotates every active clone and shows a TRAFFIC xN chip. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * Land weather then release: unguided holds persist, steady-state 2x -> 1x The always-guided weather deployment (guidance_chunks 20, re-opened by the 8-chunk skin re-swap) paid a second forward per denoise step for the whole hold — and, worse, every re-swap snapshotted its old KV from buffers that already held the weather text, so the re-opened windows guided along a ZERO direction (2x cost, no edit pressure). RNG-matched A/B on the real model (scratch weather_hold probe, rain, 27-chunk hold): a 6-chunk guided landing followed by a plain unguided hold matches the always-guided policy both in divergence-vs-clear (peak 35.1 vs 38.4 uint8, identical late-run decay) and eyes-on frames; a rebased 2-chunk maintenance pulse every 16 adds ~1 uint8 (noise), so the default hold has no maintenance. - weather guidance_chunks default 20 -> 6 (landing window only) - weather no longer rides the skin re-swap; optional maintenance knobs --live-edit-weather-maintain-interval / -maintain-chunks (default 0/2), and a maintenance pulse rebases (plain base swap first) so its guidance direction is weather-minus-base, not zero Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * Pre-encode swap prompts at session start; swaps inject cached embeddings Every skin/weather swap paid a text-encoder forward inside replace_text at the chunk boundary (450-930 ms model_ms on the boundary chunk). All swap prompts are known up front, so StyleAbility.attach() now encodes the configured skin + weather prompts once (and start() encodes the scene's base prompt once), and _replace_text injects the cached [1,1,L,D] embeddings through the pipeline's existing replace_text_from_embeddings — no main-repo change needed. Uncached prompts (or pipelines without the embedding API / a resident text encoder, e.g. the offload path) fall back to the encode-per-swap replace_text. Each swap logs cached_embeddings=<bool> swap_ms=<ms> for measurement. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * Make obstacle box-axis guidance CUDA-graph safe; drop the graph-free rebuild The guidance previously forced use_cuda_graph=False session-wide (~545 ms per chunk for the WHOLE session, events or not). It never needed to: the transformer's CUDAGraphWrapper stages every top-level tensor kwarg — including hdmap_condition — into static buffers per call, so the two forwards of a guided step are two REPLAYS of the same captured cond graph with box/no-box conditioning staged in (the mechanism two-prompt text-edit guidance already rides), and the predict_flow dispatch runs eagerly outside any capture. The one genuinely graph-unsafe seam was the shadow ENCODER: the Wan VAE's graph wrapper passes its streaming cache dict through verbatim, binding captured kernels to one cache's buffer addresses — feeding it the shadow cache would silently read/write the real cache (and alternating real/shadow calls through one wrapper would capture against whichever cache was live). Shadow encodes now run eagerly via _eager_vae_scope, which also keeps the wrapper's warmup/capture stream fed by the real cache only. install_obstacle_guidance_on_backend keeps the accelerated session; the transformer guard now rejects only the native optimized-DiT executor. GPU validation (graphs ON, s=2.5, 3-clone burst, RNG-matched baseline, scratch obstacle_graphs probe): one graph captured at chunk 4 and never re-captured through spawn/despawn; pre-spawn |diff| exactly 0.0; event chunks ~340 ms model vs ~230 non-event (non-event unchanged vs baseline); per-clone in-box |diff| 44/60/44% of the real-car benchmark with out-box clean (6.1 median) — materialization matches the graph-free reference. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * Add timed power-up skin mode with HUD countdown --live-edit-skin-duration-chunks N auto-reverts an activated skin to the base world after N generated chunks through the existing plain-swap revert path (guidance 1.0/0; corrector dispatch returns to base). 0 keeps the hold-until-cycled behavior. K during an active timed skin keeps its cycle semantics: next skin with a fresh timer (same key meaning in both modes; base reachable by cycling; a user press at the expiry boundary wins over the auto-revert). Duty-cycle re-swaps are skipped when the duration fits inside the re-swap interval; re-swaps never reset the timer. HUD skin chip shows the remaining seconds (chunk-granular, manifest-derived chunk length). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * Add --live-edit-skin-first rotation; document timed skin mode Mirrors --live-edit-weather-first: rotate the K cycle so a named skin comes first for direct one-press selection — timed power-up demos would otherwise burn transitional chunks cycling through the skins ahead of it. README documents the power-up mode and its K semantics. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * Add MJPEG drop-to-latest backpressure fix and stream bandwidth knobs Slow clients (VPN / SSH tunnel) were watching seconds-old frames: the LatestFrameBus already drops to the newest frame, but the default kernel SO_SNDBUF (several MB) let TCP queue seconds of encoded JPEGs anyway. Cap the per-connection send buffer to ~2 frames so writes block as soon as the link saturates and the next bus read skips to the newest frame. Per-connection sent/dropped/bandwidth stats are logged every 10 s and on disconnect. New knobs: --stream-jpeg-quality (default 85, unchanged) and --stream-scale (e.g. 0.5 = 640x352) downscale before JPEG encode, off the render thread as before. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * Fix fp32 base-weight aliasing in TextEditLoRA pre-merge (backport) Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * Add pickup-driven effect items and timed weather Effect items: sparse pickups along the lanes (rain/snow icons -> that weather; mystery box -> seeded-random timed skin burst) reusing the coin course/projection/culling/compositing machinery via per-sprite bank keys. Pickups dispatch through the existing ability state machines at the next chunk boundary — the same path as the K/V keys, which stay fully live. Weather stays base-world-only: a pickup during a skin is ignored with a HUD hint (not queued). Item sprites are local-only config paths; defaults render procedural placeholder icons. Timed weather: --live-edit-weather-duration-chunks (default 90 ~ 24 s) auto-reverts any weather activation to clear via a GUIDED clear landing (clear is itself a weather transition; plain swaps leave precipitation running on KV momentum). HUD weather chip gains a countdown. Re-picking the active weather refreshes the timer without a zero-direction re-swap. Weather-state corrector stays default-off per policy (clean-forcing corrector only for game-skin states); knob kept for A/B. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * Make item-course sparsity global across the lane network Real maps chop lanes into segments shorter than the item spacing (the shipped suburb map laid out 4 items instead of ~300): walk candidates every ~spacing/4 and accept one only when no accepted item lies within spacing_m (spacing-sized spatial hash), so rarity is a property of the whole network rather than each polyline. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * Document effect items and timed weather in the README Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * Keep item sparsity independent per driving direction Global min-distance suppression let whichever directed lane was walked first claim every spacing-disc along a road, so drivers of the opposite lane never passed within pickup radius of an item (0 pickups over a 30 s GPU drive that collected 60 coins). Suppress a candidate only against accepted items with a similar heading (dot > 0.5): each direction of a road carries its own items while overlapping/duplicated lanes still dedup. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * Add static-roadblock obstacle mode (parked clones from chunk 0) --live-edit-obstacle-static-count N lays out N parked-track clones midroad ahead of the spawn pose, in the conditioning from the session's first chunk, alternating laterals so the ego weaves between them; they persist until a rollout reset re-anchors them, and O-key bursts keep working alongside. Probed 2026-08-23 (RNG-matched, mask-anchored): static clones stay at ghost strength unguided even from chunk 0 (the initial camera frame anchors an empty road), materialize solid stopped cars in the 5-25 m band with box-axis guidance s=2.0, and break up when spawned mid-stream at s=2.5 - so the mode pairs with guide_scale 2.0 and nearer slots are expected to lag. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * Expose the static-roadblock lateral offset on the CLI --live-edit-obstacle-static-lateral-m widens the alternating slots so the ego has a slalom line with clearance beyond the 3 m collision radius; the first capture at the 2.8 m default clipped the clone boxes, and a camera-through-box overlap sends the box-axis guidance into frame-wide breakup. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * Add the nitro pickup item: an instant timed speed boost A nitro item (GPU icon) joins the effect-item course mix at the same rarity as the other kinds. Picking one up applies a temporary boost inside the app-authoritative taxi physics tick — the integrate_fn seam in RolloutSpec — multiplying max speed AND max acceleration by --live-edit-nitro-boost (1.6) for --live-edit-nitro-duration-s (4 s game time). Unlike the weather/skin items there is no chunk-boundary handshake: nitro is physics-only, so the effect lands on the very next sampled physics tick and composes with every skin/weather/obstacle state. A second pickup while boosted resets the timer (no stacking). The boosted max speed is hard-capped at --live-edit-nitro-max-speed (16 m/s default) so the conditioned ego never outruns the world model's manifold on the suburb map. The HUD shows the NITRO! pickup flash plus a boost chip with a game-time countdown. --live-edit-item-types restricts the course mix (single-effect capture courses); --live-edit-item-nitro-sprite supplies the local-only icon, with a procedural placeholder as the shipped default. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * Route prompt swaps through session.replace_prompt StyleAbility._replace_text's encode-per-swap fallback now calls the public FlashdreamsWorldModelSession.replace_prompt instead of reaching into session.pipeline/_cache, dropping the TODO(upstream) note. Since replace_prompt flushes the adapter's deferred chunk finalize itself, the manual flush in the continue_generation wrapper is gone; the embeddings fast path (no upstream equivalent yet) keeps an explicit _flush_pending_finalize so finalize still runs under the old text. The fake sessions in the live-edit tests grow a matching replace_prompt that mirrors the real flush-then-swap semantics. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * Add SPDX headers to demo_kit scripts Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --------- Signed-off-by: Aidan Foster <aidanf@nvidia.com> Signed-off-by: aidanfnv <aidanf@nvidia.com> Co-authored-by: Aidan Foster <aidanf@nvidia.com> Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #477 if this we want the demo in before the API work is complete, otherwise development will move to #460