[Optimization] Reduce Rerun memory footprint via automatic image downsampling

[wuxiaoqiang12]

Description:

This PR optimizes the visualization pipeline by introducing an optional image downsampling mechanism for Rerun.

The Problem:
When recording or replaying high-resolution datasets (e.g., 1080p * n), Rerun's default behavior of caching full-resolution frames leads to excessive RAM consumption and system lag, especially during long recording sessions on edge devices.

The Solution:
Added a dynamic resizing logic in log_rerun_data. If the environment variable LEROBOT_RERUN_MAX_IMAGE_WIDTH is set, images sent to Rerun will be downsampled to that width while maintaining the aspect ratio.

Key Features:

• Non-intrusive: Uses an environment variable to avoid breaking existing API signatures.
• Efficiency: Uses cv2.INTER_AREA for high-quality downsampling.
• Preservation: This optimization only affects the Rerun preview; the original high-quality data saved to disk or used for training remains untouched.

Environment Variable:

• LEROBOT_RERUN_MAX_IMAGE_WIDTH: Set the maximum width (e.g., 480 or 640) for images displayed in Rerun.

Testing:

• Recorded a dataset at 1920x1080.
• With export LEROBOT_RERUN_MAX_IMAGE_WIDTH=480: Memory usage remained stable throughout the entire session.

[imstevenpmwork]

Hello @wuxiaoqiang12, thanks for your contribution!

I have a few quick questions:

rerun provides an API to limit its maximum RAM usage. This limit can be set by setting the LEROBOT_RERUN_MEMORY_LIMIT env variable when calling rr.spawn inside the init_rerun function. By default, this is set to use 10% of available memory.

This makes me wonder:

1. Are you calling the init_rerun function when creating the visualizer instance?
2. If you are, is the memory limit not working as expected then ...?
3. If it is working, would setting this memory limit address the issue you described?

Thanks!

[atyshka]

I think a better solution would be adding support for compression to rerun. Rerun supports on the fly jpeg compression, or we could modify the code to directly access the png/mp4 files from the dataset.