Skip to content

Commit 2804ff3

Browse files
glvov-bdaigarylvovjsmith-bdai
authored
Create Manager Based Cartpole Vision Example Environments (#995)
# Description Adds manager based cartpole vision example environments. Also uses the`convert_perspective_depth_to_orthogonal_depth ` functionality introduced in #976 , contains a duplicate copy of the method and test for completeness of the PR. Will be synced with main to remove this duplicate copy once #976 is merged into main, or #976 will be synced with main if this PR is merged first ## Type of change - New feature (non-breaking change which adds functionality) - This change requires a documentation update ## Screenshots ![image](https://github.com/user-attachments/assets/4cce3923-b199-4469-b8d0-9d8d8abb3456) ## Checklist - [x] I have run the [`pre-commit` checks](https://pre-commit.com/) with `./isaaclab.sh --format` - [x] I have made corresponding changes to the documentation - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] I have updated the changelog and the corresponding version in the extension's `config/extension.toml` file - [x] I have added my name to the `CONTRIBUTORS.md` or my name already exists there ## Testing Training converged in both RGB and Depth at similar rates to the direct environments --------- Signed-off-by: garylvov <[email protected]> Signed-off-by: glvov-bdai <[email protected]> Co-authored-by: garylvov <[email protected]> Co-authored-by: garylvov <[email protected]> Co-authored-by: James Smith <[email protected]>
1 parent f8d80cb commit 2804ff3

File tree

7 files changed

+267
-8
lines changed

7 files changed

+267
-8
lines changed

‎docs/source/overview/environments.rst‎

Lines changed: 10 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -53,8 +53,12 @@ Classic environments that are based on IsaacGymEnvs implementation of MuJoCo-sty
5353
| | | |
5454
| | |cartpole-direct-link| | |
5555
+------------------+-----------------------------+-------------------------------------------------------------------------+
56-
| |cartpole| | |cartpole-camera-rgb-link| | Move the cart to keep the pole upwards in the classic cartpole control |
57-
| | | using perceptive inputs |
56+
| |cartpole| | |manager-camera-rgb-link| | Move the cart to keep the pole upwards in the classic cartpole control |
57+
| | | and perceptive inputs |
58+
| | |manager-camera-dpt-link| | |
59+
| | | |
60+
| | |cartpole-camera-rgb-link| | |
61+
| | | |
5862
| | |cartpole-camera-dpt-link| | |
5963
+------------------+-----------------------------+-------------------------------------------------------------------------+
6064

@@ -68,11 +72,14 @@ Classic environments that are based on IsaacGymEnvs implementation of MuJoCo-sty
6872

6973
.. |humanoid-direct-link| replace:: `Isaac-Humanoid-Direct-v0 <https://github.com/isaac-sim/IsaacLab/blob/main/source/extensions/omni.isaac.lab_tasks/omni/isaac/lab_tasks/direct/humanoid/humanoid_env.py>`__
7074
.. |ant-direct-link| replace:: `Isaac-Ant-Direct-v0 <https://github.com/isaac-sim/IsaacLab/blob/main/source/extensions/omni.isaac.lab_tasks/omni/isaac/lab_tasks/direct/ant/ant_env.py>`__
75+
.. |manager-camera-rgb-link| replace:: `Isaac-Cartpole-RGB-v0 <https://github.com/isaac-sim/IsaacLab/blob/main/source/extensions/omni.isaac.lab_tasks/omni/isaac/lab_tasks/manager_based/classic/cartpole/cartpole_camera_env_cfg.py>`__
76+
.. |manager-camera-dpt-link| replace:: `Isaac-Cartpole-Depth-v0 <https://github.com/isaac-sim/IsaacLab/blob/main/source/extensions/omni.isaac.lab_tasks/omni/isaac/lab_tasks/manager_based/classic/cartpole/cartpole_camera_env_cfg.py>`__
7177
.. |cartpole-direct-link| replace:: `Isaac-Cartpole-Direct-v0 <https://github.com/isaac-sim/IsaacLab/blob/main/source/extensions/omni.isaac.lab_tasks/omni/isaac/lab_tasks/direct/cartpole/cartpole_env.py>`__
78+
.. |manager-camera-rgb-link| replace:: `Isaac-Cartpole-RGB-v0 <https://github.com/isaac-sim/IsaacLab/blob/main/source/extensions/omni.isaac.lab_tasks/omni/isaac/lab_tasks/manager_based/classic/cartpole/cartpole_camera_env_cfg.py>`__
79+
.. |manager-camera-dpt-link| replace:: `Isaac-Cartpole-Depth-v0 <https://github.com/isaac-sim/IsaacLab/blob/main/source/extensions/omni.isaac.lab_tasks/omni/isaac/lab_tasks/manager_based/classic/cartpole/cartpole_camera_env_cfg.py>`__
7280
.. |cartpole-camera-rgb-link| replace:: `Isaac-Cartpole-RGB-Camera-Direct-v0 <https://github.com/isaac-sim/IsaacLab/blob/main/source/extensions/omni.isaac.lab_tasks/omni/isaac/lab_tasks/direct/cartpole/cartpole_camera_env.py>`__
7381
.. |cartpole-camera-dpt-link| replace:: `Isaac-Cartpole-Depth-Camera-Direct-v0 <https://github.com/isaac-sim/IsaacLab/blob/main/source/extensions/omni.isaac.lab_tasks/omni/isaac/lab_tasks/direct/cartpole/cartpole_camera_env.py>`__
7482

75-
7683
Manipulation
7784
~~~~~~~~~~~~
7885

‎source/extensions/omni.isaac.lab/docs/CHANGELOG.rst‎

Lines changed: 10 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,13 @@
11
Changelog
22
---------
33

4+
0.22.15 (2024-09-20)
5+
~~~~~~~~~~~~~~~~~~~~
6+
7+
Added
8+
^^^^^
9+
10+
* Added :meth:`grab_images` to be able to use images for an observation term in manager based environments
411

512
0.24.14 (2024-09-20)
613
~~~~~~~~~~~~~~~~~~~~
@@ -9,10 +16,9 @@ Added
916
^^^^^
1017

1118
* Added :meth:`convert_perspective_depth_to_orthogonal_depth`. :meth:`unproject_depth` assumes
12-
that the input depth image is orthogonal. The new :meth:`convert_perspective_depth_to_orthogonal_depth`
13-
can be used to convert a perspective depth image into an orthogonal depth image, so that the point cloud
14-
can be unprojected correctly with :meth:`unproject_depth`.
15-
19+
that the input depth image is orthogonal. The new :meth:`convert_perspective_depth_to_orthogonal_depth`
20+
can be used to convert a perspective depth image into an orthogonal depth image, so that the point cloud
21+
can be unprojected correctly with :meth:`unproject_depth`.
1622

1723
0.24.13 (2024-09-08)
1824
~~~~~~~~~~~~~~~~~~~~

‎source/extensions/omni.isaac.lab/omni/isaac/lab/envs/mdp/observations.py‎

Lines changed: 36 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -17,7 +17,7 @@
1717
import omni.isaac.lab.utils.math as math_utils
1818
from omni.isaac.lab.assets import Articulation, RigidObject
1919
from omni.isaac.lab.managers import SceneEntityCfg
20-
from omni.isaac.lab.sensors import RayCaster
20+
from omni.isaac.lab.sensors import Camera, RayCaster, RayCasterCamera, TiledCamera
2121

2222
if TYPE_CHECKING:
2323
from omni.isaac.lab.envs import ManagerBasedEnv, ManagerBasedRLEnv
@@ -182,6 +182,41 @@ def body_incoming_wrench(env: ManagerBasedEnv, asset_cfg: SceneEntityCfg) -> tor
182182
return link_incoming_forces.view(env.num_envs, -1)
183183

184184

185+
def grab_images(
186+
env: ManagerBasedEnv,
187+
sensor_cfg: SceneEntityCfg = SceneEntityCfg("tiled_camera"),
188+
data_type: str = "rgb",
189+
convert_perspective_to_orthogonal: bool = False,
190+
normalize: bool = True,
191+
) -> torch.Tensor:
192+
"""Grab all of the latest images of a specific datatype produced by a specific camera.
193+
194+
Args:
195+
env: The environment the cameras are placed within.
196+
sensor_cfg: The desired sensor to read from. Defaults to SceneEntityCfg("tiled_camera").
197+
data_type: The data type to pull from the desired camera. Defaults to "rgb".
198+
convert_perspective_to_orthogonal: Whether to convert perspective
199+
depth images to orthogonal depth images. Defaults to False.
200+
normalize: Set to True to normalize images. Defaults to True.
201+
202+
Returns:
203+
The images produced at the last timestep
204+
"""
205+
sensor: TiledCamera | Camera | RayCasterCamera = env.scene.sensors[sensor_cfg.name]
206+
images = sensor.data.output[data_type]
207+
if (data_type == "distance_to_camera") and convert_perspective_to_orthogonal:
208+
images = math_utils.convert_perspective_depth_to_orthogonal_depth(images, sensor.data.intrinsic_matrices)
209+
210+
if normalize:
211+
if data_type == "rgb":
212+
images = images / 255
213+
mean_tensor = torch.mean(images, dim=(1, 2), keepdim=True)
214+
images -= mean_tensor
215+
elif "distance_to" in data_type or "depth" in data_type:
216+
images[images == float("inf")] = 0
217+
return images.clone()
218+
219+
185220
"""
186221
Actions.
187222
"""

‎source/extensions/omni.isaac.lab_tasks/docs/CHANGELOG.rst‎

Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,15 @@
11
Changelog
22
---------
33

4+
0.10.6 (2024-09-25)
5+
~~~~~~~~~~~~~~~~~~~
6+
7+
Added
8+
^^^^^
9+
* Added ``Isaac-Cartpole-RGB-Camera-v0`` and ``Isaac-Cartpole-Depth-Camera-v0``
10+
manager based camera cartpole environments.
11+
12+
413
0.10.5 (2024-09-11)
514
~~~~~~~~~~~~~~~~~~~
615

‎source/extensions/omni.isaac.lab_tasks/omni/isaac/lab_tasks/manager_based/classic/cartpole/__init__.py‎

Lines changed: 21 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -10,6 +10,7 @@
1010
import gymnasium as gym
1111

1212
from . import agents
13+
from .cartpole_camera_env_cfg import CartpoleDepthCameraEnvCfg, CartpoleRGBCameraEnvCfg
1314
from .cartpole_env_cfg import CartpoleEnvCfg
1415

1516
##
@@ -28,3 +29,23 @@
2829
"sb3_cfg_entry_point": f"{agents.__name__}:sb3_ppo_cfg.yaml",
2930
},
3031
)
32+
33+
gym.register(
34+
id="Isaac-Cartpole-RGB-Camera-v0",
35+
entry_point="omni.isaac.lab.envs:ManagerBasedRLEnv",
36+
disable_env_checker=True,
37+
kwargs={
38+
"env_cfg_entry_point": CartpoleRGBCameraEnvCfg,
39+
"rl_games_cfg_entry_point": f"{agents.__name__}:rl_games_camera_ppo_cfg.yaml",
40+
},
41+
)
42+
43+
gym.register(
44+
id="Isaac-Cartpole-Depth-Camera-v0",
45+
entry_point="omni.isaac.lab.envs:ManagerBasedRLEnv",
46+
disable_env_checker=True,
47+
kwargs={
48+
"env_cfg_entry_point": CartpoleDepthCameraEnvCfg,
49+
"rl_games_cfg_entry_point": f"{agents.__name__}:rl_games_camera_ppo_cfg.yaml",
50+
},
51+
)

‎source/extensions/omni.isaac.lab_tasks/omni/isaac/lab_tasks/manager_based/classic/cartpole/agents/rl_games_camera_ppo_cfg.yaml‎

Lines changed: 95 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,95 @@
1+
params:
2+
seed: 42
3+
4+
# environment wrapper clipping
5+
env:
6+
# added to the wrapper
7+
clip_observations: 5.0
8+
# can make custom wrapper?
9+
clip_actions: 1.0
10+
11+
algo:
12+
name: a2c_continuous
13+
14+
model:
15+
name: continuous_a2c_logstd
16+
17+
# doesn't have this fine grained control but made it close
18+
network:
19+
name: actor_critic
20+
separate: False
21+
space:
22+
continuous:
23+
mu_activation: None
24+
sigma_activation: None
25+
26+
mu_init:
27+
name: default
28+
sigma_init:
29+
name: const_initializer
30+
val: 0
31+
fixed_sigma: True
32+
cnn:
33+
type: conv2d
34+
activation: relu
35+
initializer:
36+
name: default
37+
regularizer:
38+
name: None
39+
convs:
40+
- filters: 32
41+
kernel_size: 8
42+
strides: 4
43+
padding: 0
44+
- filters: 64
45+
kernel_size: 4
46+
strides: 2
47+
padding: 0
48+
- filters: 64
49+
kernel_size: 3
50+
strides: 1
51+
padding: 0
52+
53+
mlp:
54+
units: [512]
55+
activation: elu
56+
initializer:
57+
name: default
58+
59+
load_checkpoint: False # flag which sets whether to load the checkpoint
60+
load_path: '' # path to the checkpoint to load
61+
62+
config:
63+
name: cartpole_camera
64+
env_name: rlgpu
65+
device: 'cuda:0'
66+
device_name: 'cuda:0'
67+
multi_gpu: False
68+
ppo: True
69+
mixed_precision: False
70+
normalize_input: False
71+
normalize_value: True
72+
num_actors: -1 # configured from the script (based on num_envs)
73+
reward_shaper:
74+
scale_value: 1.0
75+
normalize_advantage: True
76+
gamma: 0.99
77+
tau : 0.95
78+
learning_rate: 1e-4
79+
lr_schedule: adaptive
80+
kl_threshold: 0.008
81+
score_to_win: 20000
82+
max_epochs: 500
83+
save_best_after: 50
84+
save_frequency: 25
85+
grad_norm: 1.0
86+
entropy_coef: 0.0
87+
truncate_grads: True
88+
e_clip: 0.2
89+
horizon_length: 64
90+
minibatch_size: 2048
91+
mini_epochs: 4
92+
critic_coef: 2
93+
clip_value: True
94+
seq_length: 4
95+
bounds_loss_coef: 0.0001

‎source/extensions/omni.isaac.lab_tasks/omni/isaac/lab_tasks/manager_based/classic/cartpole/cartpole_camera_env_cfg.py‎

Lines changed: 86 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,86 @@
1+
# Copyright (c) 2022-2024, The Isaac Lab Project Developers.
2+
# All rights reserved.
3+
#
4+
# SPDX-License-Identifier: BSD-3-Clause
5+
6+
import omni.isaac.lab.sim as sim_utils
7+
from omni.isaac.lab.envs.mdp.observations import grab_images
8+
from omni.isaac.lab.managers import ObservationGroupCfg as ObsGroup
9+
from omni.isaac.lab.managers import ObservationTermCfg as ObsTerm
10+
from omni.isaac.lab.managers import SceneEntityCfg
11+
from omni.isaac.lab.sensors import TiledCameraCfg
12+
from omni.isaac.lab.utils import configclass
13+
14+
from omni.isaac.lab_tasks.manager_based.classic.cartpole.cartpole_env_cfg import CartpoleEnvCfg, CartpoleSceneCfg
15+
16+
##
17+
# Scene definition
18+
##
19+
20+
21+
@configclass
22+
class CartpoleRGBCameraSceneCfg(CartpoleSceneCfg):
23+
tiled_camera: TiledCameraCfg = TiledCameraCfg(
24+
prim_path="{ENV_REGEX_NS}/Camera",
25+
offset=TiledCameraCfg.OffsetCfg(pos=(-7.0, 0.0, 3.0), rot=(0.9945, 0.0, 0.1045, 0.0), convention="world"),
26+
data_types=["rgb"],
27+
spawn=sim_utils.PinholeCameraCfg(
28+
focal_length=24.0, focus_distance=400.0, horizontal_aperture=20.955, clipping_range=(0.1, 20.0)
29+
),
30+
width=80,
31+
height=80,
32+
)
33+
34+
35+
@configclass
36+
class CartpoleDepthCameraSceneCfg(CartpoleSceneCfg):
37+
tiled_camera: TiledCameraCfg = TiledCameraCfg(
38+
prim_path="{ENV_REGEX_NS}/Camera",
39+
offset=TiledCameraCfg.OffsetCfg(pos=(-7.0, 0.0, 3.0), rot=(0.9945, 0.0, 0.1045, 0.0), convention="world"),
40+
data_types=["distance_to_camera"],
41+
spawn=sim_utils.PinholeCameraCfg(
42+
focal_length=24.0, focus_distance=400.0, horizontal_aperture=20.955, clipping_range=(0.1, 20.0)
43+
),
44+
width=80,
45+
height=80,
46+
)
47+
48+
49+
@configclass
50+
class RGBObservationsCfg:
51+
"""Observation specifications for the MDP."""
52+
53+
@configclass
54+
class RGBCameraPolicyCfg(ObsGroup):
55+
"""Observations for policy group."""
56+
57+
image = ObsTerm(func=grab_images, params={"sensor_cfg": SceneEntityCfg("tiled_camera"), "data_type": "rgb"})
58+
59+
def __post_init__(self) -> None:
60+
self.enable_corruption = False
61+
self.concatenate_terms = True
62+
63+
policy: ObsGroup = RGBCameraPolicyCfg()
64+
65+
66+
@configclass
67+
class DepthObservationsCfg:
68+
@configclass
69+
class DepthCameraPolicyCfg(RGBObservationsCfg.RGBCameraPolicyCfg):
70+
image = ObsTerm(
71+
func=grab_images, params={"sensor_cfg": SceneEntityCfg("tiled_camera"), "data_type": "distance_to_camera"}
72+
)
73+
74+
policy: ObsGroup = DepthCameraPolicyCfg()
75+
76+
77+
@configclass
78+
class CartpoleRGBCameraEnvCfg(CartpoleEnvCfg):
79+
scene: CartpoleSceneCfg = CartpoleRGBCameraSceneCfg(num_envs=1024, env_spacing=20)
80+
observations = RGBObservationsCfg()
81+
82+
83+
@configclass
84+
class CartpoleDepthCameraEnvCfg(CartpoleEnvCfg):
85+
scene: CartpoleSceneCfg = CartpoleDepthCameraSceneCfg(num_envs=1024, env_spacing=20)
86+
observations = DepthObservationsCfg()

0 commit comments

Comments
 (0)