R23: Add examples/libero_isaac_fleet.py — IsaacLab fleet-scale LIBERO (Stage 3)

[yinsong1986]

Summary

Add examples/libero_isaac_fleet.py — same LIBERO task as R5 / R8 / R12, but at fleet scale on Isaac Sim via IsaacLab. Fills the missing isaac | 4096 row in the backend comparison matrix and the R4 README table.

Stage 3 of #8. Companion to R8 / #15 (single-env Isaac).

Why this exists

The original LIBERO matrix scope (R8 + R12) had Isaac at single-env-RTX and Newton at both single and fleet. That implied 4096+ parallelism was a Newton-unique capability — but Isaac Sim supports 4096+ parallel envs natively via IsaacLab on PhysX GPU; the differentiator between Isaac and Newton at fleet scale is photorealism vs. solver flexibility / differentiability, not n_envs.

R23 makes the matrix honest: both heavy backends carry single-env and fleet rows, with different cost profiles per env (Isaac pays Omniverse overhead even with RTX off; Newton runs raw OpenGL / null-render).

Discussion thread: comments on #8 — both Isaac and Newton support 4096+ envs.

Scope

• examples/libero_isaac_fleet.py — ~50 LOC, runs libero-spatial-pick_up_the_red_cube via sim.evaluate_benchmark(...) on create_simulation("isaac", num_envs=4096, headless=True, rtx_mode="real_time") (or RTX off entirely for max throughput)
• Replicates R12's fleet pattern: aggregate metrics (success rate per env, mean wall-time, total throughput in steps/sec)
• Header docstring frames this as "the photoreal-stack alternative to Newton fleet" — i.e. fleet RL and USD scenes / IsaacLab tooling
• examples/README.md and R4's README LIBERO matrix table updated with the isaac | 4096 row + measured numbers

Sketch (design subject to R7's IsaacLab integration)

# examples/libero_isaac_fleet.py
"""LIBERO at fleet scale on Isaac Sim via IsaacLab — 4096-env RL-style throughput.

Use this when you want fleet-scale rollouts on USD scenes with optional photoreal
observations on a subset. For pure throughput without rendering, prefer Newton
(libero_newton_fleet.py) — Isaac Sim still pays some Omniverse overhead even with
RTX off.

Requires:
    pip install 'strands-robots-sim[isaac]'
    pip install 'strands-robots[benchmark-libero]'
"""
import time
import strands_robots_sim  # noqa: F401 — registers "isaac" backend
from strands_robots.simulation import create_simulation
from strands_robots.benchmarks.libero import load_libero_suite

sim = create_simulation(
    "isaac",
    num_envs=4096,
    headless=True,
    rtx_mode="real_time",        # or "off" for max throughput
)
sim.create_world()
load_libero_suite("libero_spatial")

t0 = time.time()
result = sim.evaluate_benchmark(
    benchmark_name="libero-spatial-pick_up_the_red_cube",
    policy_provider="mock", n_episodes=1, seed=42,   # 1 ep × 4096 envs = 4096 episodes
)
elapsed = time.time() - t0
steps_per_sec = result["total_steps"] / elapsed
print(f"success_rate={result['success_rate']:.2f} "
      f"wall_time={elapsed:.1f}s "
      f"throughput={steps_per_sec:,.0f} steps/sec")
sim.destroy()

Definition of Done

• examples/libero_isaac_fleet.py runs end-to-end on an RTX GPU with [isaac] + [benchmark-libero] installed
• Same output format as R5 / R8 / R12 so R15 (matrix flagship) can parse all rows uniformly
• Header docstring contrasts explicitly with libero_newton_fleet.py (R12) — "when to pick which fleet backend"
• examples/README.md updated: isaac | 4096 row present with measured numbers
• R4 README LIBERO matrix table updated: 5-row matrix with this row inserted between isaac | 1 and newton | 1 (so Isaac single + fleet sit together)
• Throughput recorded; if it hits a meaningful fraction of Newton's number, flagged in the header docstring

Dependencies

• Blocked on: R7.4 (Isaac obs/action/render) — needs IsaacLab vectorized-env support. May bump scope of R7 if IsaacLab fleet integration is heavier than originally scoped.
• Parallel with: R9 / #16 (Replicator synth-data — same Isaac scaffolding)
• Unblocks: R15 / #22 (fills the 5th row of the flagship matrix)

References

• Umbrella: #8
• Backend epic: R7 / #14 (especially R7.4)
• Single-env Isaac counterpart: R8 / #15
• Newton fleet counterpart: R12 / #19
• Matrix flagship: R15 / #22

[yinsong1986]

Closing per maintainer decision: dropping R23 (examples/libero_isaac_fleet.py) from the v0.2.0 plan. Removed from the #8 roadmap as well.