[Newton] Lots of hackery to get kit-optional paths working with and without OV

[kellyguo11]

Description

A lot of hacking to decouple from OV - so far only tested on Cartpole.

Changes that need to be considered:

• Includes new cloner changes from Octi
• Need a nice way to deal with assets without nucleus and omni.client APIs
• Removed isaac sim SimulationContext and SimulationManager references
• All omni.ui dependencies are lazy loaded
• A lot of hacks around prim and stage utils
• Alternative way for carb settings
• XFormPrim class is re-implemented in Isaac Lab
• omni.kit.commands, semantics are replaced
• omni.timeline callbacks are removed

Known Issues:

• OV visualizer with ground plane doesn't show up properly - material files are not being downloaded with the asset
• Not sure if training is still working

[greptile-apps]

Greptile Overview

Greptile Summary

This PR introduces significant architectural changes to decouple Isaac Lab from Omniverse/Kit dependencies, enabling standalone operation with Newton and Rerun visualizers.

Key Changes:

• New SettingsManager class provides unified settings interface for both Omniverse and standalone modes
• AppLauncher now conditionally launches SimulationApp only when Omniverse visualizer, livestreaming, or cameras are needed
• SimulationContext reimplemented as standalone class with custom singleton pattern (no longer inherits from Isaac Sim)
• New scene/cloner.py module with newton_replicate, usd_replicate, and grid_transforms functions for environment cloning
• Asset loading refactored to use boto3/requests for S3 and HTTP file retrieval instead of omni.client
• Most omni.timeline callbacks and SimulationManager references commented out
• Semantic labeling now uses USD core attributes instead of deprecated Semantics API
• omni.kit.commands replaced with direct USD core API calls throughout

Known Issues (per PR description):

• OV visualizer with ground plane doesn't show materials properly
• Training functionality not verified

Confidence Score: 2/5

• This PR contains several syntax/logic errors that will cause runtime failures and should be fixed before merging.
• Score of 2 reflects: (1) duplicate property definition in app_launcher.py, (2) missing Gf import in prims.py causing NameError, (3) hardcoded cuda device in interactive_scene.py breaking CPU support. Additionally, extensive commented-out code and acknowledged issues (ground plane materials, training not verified) indicate WIP status.
• source/isaaclab/isaaclab/app/app_launcher.py (duplicate property), source/isaaclab/isaaclab/sim/utils/prims.py (missing import), source/isaaclab/isaaclab/scene/interactive_scene.py (hardcoded device)

Important Files Changed

File Analysis

Filename	Score	Overview
source/isaaclab/isaaclab/app/app_launcher.py	2/5	Major refactor to support standalone mode without Omniverse. Contains duplicate visualizer property definition which will cause issues.
source/isaaclab/isaaclab/sim/utils/prims.py	2/5	New create_default_xform_ops function uses Gf module but doesn't import it from pxr, causing NameError at runtime.
source/isaaclab/isaaclab/scene/interactive_scene.py	2/5	Hardcoded device="cuda" in grid_transforms call will fail on CPU-only systems. Should use self.device instead.
source/isaaclab/isaaclab/sim/simulation_context.py	3/5	Major refactor implementing standalone simulation context without Isaac Sim base class. Singleton pattern with new _initialized flag.
source/isaaclab/isaaclab/app/settings_manager.py	4/5	New file providing unified settings interface for both Omniverse and standalone modes. Uses singleton stored in sys.modules.
source/isaaclab/isaaclab/scene/cloner.py	3/5	New cloner module with Newton integration for environment replication. Uses global mutable CLONE dictionary.
source/isaaclab/isaaclab/sim/utils/stage.py	4/5	Refactored stage utilities to use USD core APIs instead of omni APIs. Supports remote file downloading via S3/HTTP.
source/isaaclab/isaaclab/utils/assets.py	4/5	Replaced omni.client with boto3/requests for S3 and HTTP asset retrieval. Adds new cloud storage support.

Sequence Diagram

sequenceDiagram
    participant User
    participant AppLauncher
    participant SettingsManager
    participant SimulationApp
    participant SimulationContext
    participant NewtonManager
    participant Cloner

    User->>AppLauncher: Initialize with args
    AppLauncher->>AppLauncher: _check_if_omniverse_required()
    alt Omniverse Required
        AppLauncher->>SimulationApp: Create SimulationApp
        AppLauncher->>SettingsManager: initialize_carb_settings()
        SettingsManager->>SettingsManager: Use carb.settings
    else Standalone Mode
        AppLauncher->>SettingsManager: _store_settings_standalone()
        SettingsManager->>SettingsManager: Use Python dict
    end
    
    User->>SimulationContext: Create SimulationContext(cfg)
    SimulationContext->>SettingsManager: Get settings
    SimulationContext->>SimulationContext: Create stage in memory
    SimulationContext->>NewtonManager: Configure solver
    
    User->>Cloner: clone_environments()
    Cloner->>Cloner: newton_replicate()
    Cloner->>NewtonManager: set_builder()
    Cloner->>Cloner: usd_replicate()

Loading

[greptile-apps]

Additional Comments (1)

1. source/isaaclab/isaaclab/app/app_launcher.py, line 227-245 (link)

   syntax: Duplicate visualizer property definition. The property is defined twice (lines 227-235 and 237-245). The second definition shadows the first.

_{43 files reviewed, 3 comments}

_{Edit Code Review Agent Settings | Greptile}

[greptile-apps]

syntax: Missing Gf import. The new create_default_xform_ops function (line 1752) uses Gf.Vec3d, Gf.Vec3f, Gf.Rotation, Gf.Quatd, Gf.Quatf, and Gf.Matrix4d but Gf is not imported.

Suggested change

	from pxr import Sdf, Usd, UsdGeom, UsdPhysics, UsdShade
	from pxr import Gf, Sdf, Usd, UsdGeom, UsdPhysics, UsdShade

[greptile-apps]

logic: Hardcoded device="cuda" will cause failures on CPU-only systems. Should use self.device to respect the configured device.

Suggested change

	self._default_env_origins, self.env_orientations = grid_transforms(self.cfg.num_envs, self.cfg.env_spacing, up_axis="z", device="cuda")
	self._default_env_origins, self.env_orientations = grid_transforms(self.cfg.num_envs, self.cfg.env_spacing, up_axis="z", device=self.device)

[kellyguo11]

Closing, newer PR available at #4165