LeRobot v0.4.0: Supercharging OSS Robot Learning

Summary

LeRobot v0.4.0 expands the open-source robot-learning stack with scalable datasets, simulation benchmarks, processing tools, policies, and hardware integrations. LeRobotDataset v3.0 introduces chunked episodes, video streaming, unified Parquet metadata, improved loading and memory efficiency, plus utilities for editing and converting datasets. A modular Processor system separates policy and robot pipelines, while direct Accelerate integration simplifies multi-GPU training; the release also adds LIBERO, Meta-World, PI0, PI0.5, and GR00T N1.5 support. A plugin architecture enables third-party robots, cameras, and teleoperators to be installed without changing the core library, with Reachy 2 and phone teleoperation included, alongside a self-paced Robot Learning Course and hands-on tutorial.

Takeaways

1. 1

   LeRobotDataset v3.0 stores episode metadata in unified Parquet files and supports chunked episodes and video streaming for datasets at the OXE level, stated as over 400GB.

2. 2

   The Processor system uses separate pipelines for batched policy tensors and individual robot observations or actions, handling transformations such as normalization, tokenization, device movement, and action unnormalization.

3. 3

   The hardware plugin system lets third-party robots, cameras, and teleoperators be distributed as separate Python packages; the release adds Reachy 2 and phone-based teleoperation support.