150 GenMOS Example Configurations

In this wiki, we go through an example set of configurations.

These configurations are from the integration of GenMOS with Viam on UR5e (done in December, 2022). The configurations below come from this file viam/config/ur5_exp1_viamlab.py.

Also check out the configurations for Spot (e.g. spot_exp1_local_lab121.py) and MOVO (e.g. movo_exp1_scili8_living_room.py) as additional examples.

Top-level Configuration

######### THE FOLLOWING IS USED FOR LOCAL SEARCH TEST #########
target_objects = ["Cup"]
CONFIG_LOCAL = {
    "agent_config": {
        "agent_class": "MosAgentTopo3D",
        "agent_type": "local",
        'belief': LOCAL_BELIEF,
        "search_region": {"3d": SEARCH_REGION_3D},
        'robot': {
            'id': 'robot0',
            'no_look': True,
            'sensors': [GRIPPER_CAMERA],
            'detectors': make_detectors("local", *target_objects),
            'action': LOCAL_ACTION,
            "reachable": LOCAL_REACHABLE,
        },
        'objects': make_objects(*target_objects),
        'targets': target_objects
    },

    "task_config": {
        "max_steps": 200
    },

    "planner_config": LOCAL_PLANNER_CONFIG
}

For clarity and organization, we define the following dictionaries somewhere else:

• LOCAL_BELIEF
• SEARCH_REGION_3D
• GRIPPER_CAMERA
• LOCAL_ACTION
• LOCAL_REACHABLE
• LOCAL_PLANNER_CONFIG

We provide their values in the following sections.

Define Objects

Target objects, specified in "targets", should be a list of object names that is a subset of all objects in "objects".

One could specify the configuration of objects as follows:

OBJECTS = {
    'Cup': {'class': 'cup',
            'transition': {'class': 'genmos_object_search.oopomdp.StaticObjectTransitionModel'},
            'color': [0.67, 0.61, 0.15, 0.8],
            'viz_type': 'cube'}
}

def make_objects(*objects):
    result = {}
    for obj in objects:
        result[obj] = OBJECTS[obj]
    return result

The output of make_objects is a dictionary mapping from object name (e.g. 'Cup') to a dictionary. This function may be convenient if you have a large OBJECTS dictionary and you only want to use a subset of them in the search task.

LOCAL_BELIEF

These parameters are used when initializing and updating the belief for local search.

LOCAL_BELIEF = {"visible_volume_params": {"num_rays": 150,
                                          "step_size": 0.2,
                                          "voxel_res": 2},
                "init_params": {"num_samples": 3000,
                                "prior_from_occupancy": True,
                                "occupancy_height_thres": 0.2,
                                "occupancy_blow_up_res": 4,
                                "occupancy_fill_height": True}}

SEARCH_REGION_3D

These parameters are used to configure the 3D search region:

SEARCH_REGION_3D = {"res": 0.08,
                    "octree_size": 32,
                    "region_size_x": 2.5,
                    "region_size_y": 2.0,
                    "region_size_z": 1.5,
                    "center_x": -0.5,
                    "center_y": -1.65,
                    "center_z": -0.25,
                    "debug": False}

GRIPPER_CAMERA

Defines the geometry of the gripper camera's FOV (a viewing frustm).

GRIPPER_CAMERA = {'name': 'gripper_camera',
                  'params': {'fov': 45,
                             'far': 1.3,
                             'near': 0.2,
                             'aspect_ratio': 1.3,
                             'occlusion_enabled': True}}

LOCAL_ACTION

Configuration for local search action space

LOCAL_ACTION = {'topo': LOCAL_TOPO,
                'policy': {'cost_scaling_factor': 1.0}}

Parameters for topological graph-based viewpoint action space

LOCAL_TOPO = {'num_nodes': 10,
              'sep': 0.3,
              'debug': False,
              'resample_thres': 0.4,
              # The 3D box within which samples of viewpoint positions will be drawn.
              # This is useful for an eye-in-hand robot arm to specify a reachable
              # volume by its end-effector. 
              'sample_space': {
                  "center_x": -0.05,
                  "center_y": -0.4,
                  "center_z": 0.25,
                  "size_x": 2.00,
                  "size_y": 1.50,
                  "size_z": 2.00
              },
              # because the sample space may be off the search region,
              # it is not so important to filter by importance.
              'pos_importance_thres': -1}

              ## These parameters are useful for a mobile robot, e.g. Spot
              ## which inflates point cloud as obstacles to avoid sampling
              ## topo nodes in collision with obstacles.  Since UR5e is an
              ## arm and the gripper camera we used has no depth, we don't
              ## use these parameters.
              # '3d_proj_2d': {'layout_cut': 0.4,
              #                'floor_cut': 0.15,
              #                'brush_size': 0.2,
              #                'inflation': 0.5}}

LOCAL REACHABLE

Parameters related to defining the reachable space by the robot's camera

LOCAL_REACHABLE = {"min_height": 1.2,
                   #"max_height": 1.3}  # the maximum height here is determined by the sample_space defined in LOCAL_TOPO above.

LOCAL_PLANNER_CONFIG

Parameters used by the POMDP planner.

LOCAL_PLANNER_CONFIG = {"planner": "pomdp_py.POUCT",
                        "planner_params": {
                            "exploration_const": 1000,
                            "max_depth": 6,
                            "num_sims": 400,
                            "show_progress": True}}