Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Robomimic Version Incompatability #130

Open
Benjamin-So opened this issue Dec 8, 2024 · 0 comments
Open

Robomimic Version Incompatability #130

Benjamin-So opened this issue Dec 8, 2024 · 0 comments

Comments

@Benjamin-So
Copy link

Benjamin-So commented Dec 8, 2024

Thank you for sharing such great work!

TLDR:
How can I visualize trajectories for all demos in the Robomimic transportation task with enlarged dimensions? Using Robomimic's built-in script to render videos, I encounter errors due to differences in XML metadata between Diffusion Policy and Robomimic. How were the hdf5 files preprocessed for Diffusion Policy's version of the Robomimic transport task, as the backgrounds also differ?

Full Description
I wanted to see if it would be possible to visualize the trajectories for all demos in the transportation task but with enlarged dimensions. For example, I tried stitching the image observations in the hdf5 files, but using this method I can only generate videos with 84x84 dimensions (in the Robomimic transportation case). Is there a clean way to achieve higher resolution?

Robomimic provides a built-in script for executing actions stored in an hdf5 file within their environment and saving the resulting video. The script is described here: Robomimic Dataset Contents.

Using this script, I ran:
python playback_dataset.py --dataset /pathto/diffusion_policy/data/robomimic/datasets/transport/mh/image.hdf5 --use-actions --render --render_image_names shouldercamera0_image

However, I encountered an error due to differences in the structure of the XML metadata for the hdf5 transport task provided by Diffusion Policy. This raises a key question:

How were the hdf5 files preprocessed for the Robomimic tasks in Diffusion Policy? I also noticed differences in the background when visualizing image observations for the transport task in Diffusion Policy compared to the original Robomimic dataset.

Would appreciate any insights or guidance!


I obtained the video below by executing the actions provided in the transport demo_0 hdf5 file downloaded from the robomimic website. I used the command provided by robomimic to rollout the trajectory.

playback_dataset_with_actions.mp4

I obtained the video below by stitching together the observations in the image transport demo_0 task:

shouldercamera0_image.1.mp4
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant