This repo originates from AnyTeleop Project
![Retargeting with different hands.](/dexsuite/dex-retargeting/raw/main/example/vector_retargeting/teaser.webp)
pip install dex_retargeting
To run the example, you may need additional dependencies for rendering and hand pose detection.
git clone https://github.com/dexsuite/dex-retargeting
cd dex-retargeting
pip install -e ".[example]"
This type of retargeting can be used for applications like teleoperation, e.g. AnyTeleop.
Tutorial on retargeting from human hand video
This type of retargeting can be used post-process human data for robot imitation, e.g. DexMV.
Tutorial on retargeting from hand-object pose dataset
URDF parsers, such as ROS, physical simulators, real robot driver, and this repository, may parse URDF files with
different joint orders. To use dex-retargeting
results with other libraries, handle joint ordering explicitly using
joint names, which are unique within a URDF file.
Example: Using dex-retargeting
with the SAPIEN simulator
from dex_retargeting.seq_retarget import SeqRetargeting
retargeting: SeqRetargeting
sapien_joint_names = [joint.get_name() for joint in robot.get_active_joints()]
retargeting_joint_names = retargeting.joint_names
retargeting_to_sapien = np.array([retargeting_joint_names.index(name) for name in sapien_joint_names]).astype(int)
# Use the index map to handle joint order differences
sapien_robot.set_qpos(retarget_qpos[retargeting_to_sapien])
This example retrieves joint names from the SAPIEN robot and SeqRetargeting
object, creates a mapping
array (retargeting_to_sapien
) to map joint indices, and sets the SAPIEN robot's joint positions using the retargeted
joint positions.
This repository is derived from the AnyTeleop Project and is subject to ongoing enhancements. If you utilize this work, please cite it as follows:
@inproceedings{qin2023anyteleop,
title = {AnyTeleop: A General Vision-Based Dexterous Robot Arm-Hand Teleoperation System},
author = {Qin, Yuzhe and Yang, Wei and Huang, Binghao and Van Wyk, Karl and Su, Hao and Wang, Xiaolong and Chao, Yu-Wei and Fox, Dieter},
booktitle = {Robotics: Science and Systems},
year = {2023}
}
The robot hand models in this repository are sourced directly from dex-urdf. The robot kinematics in this repo are based on pinocchio. Examples use SAPIEN for rendering and visualization.
The PositionOptimizer
leverages methodologies from our earlier
project, From One Hand to Multiple Hands.
Additionally, the DexPilotOptimizer
is crafted using insights from DexPilot.