Live UMI

Introduction

Effective and generalizable policy learning for robotic manipulation requires a large amount of diverse, accurate training data. Approaches such as teleoperation require specialized and expensive machinery, making them unscalable. Skilled human operators are also required, increasing the difficulty and time needed to obtain usable training data. Setups such as leader-follower attempt to mitigate this skill-gap, but still require specialized equipment that limits action diversity and the number of accomplishable tasks. On the other hand, there are large amounts of human demonstration videos online. But to transform this data into usable training data for policy learning is challenging due to discrepancies between direct human actions and robotic arms.

The Universal Manipulation Interface (UMI) solves these issues by providing a scalable, easy-to-use platform for diverse data collection. With a hand-held gripper, a camera captures all data necessary to bridge the gap between human demonstration and robot policy. The intuitive nature of the gripper allows for fast data collection, while the gripper interface is directly applicable to robotic arms, facilitating direct policy learning.

Quick Summary

Live UMI builds upon previous UMI variants by using LiDAR inertial SLAM and a fisheye lens wrist camera. It features major improvements to the gear and linkage system, increasing ease of use and long-term reliability. Custom fingers make highly dexterous tasks easy, while still allowing for heavy object manipulation. All the while, Live UMI remains bimanual and completely in-the-wild, able to be used anywhere without external stations or connections. This is due to the onboard Raspberry Pi 5, LCD screen, buttons, and swappable battery pack.

Data collection is as simple as pressing a button to begin recording an episode. Once done, everything is automatically saved, post-processed, and uploaded to Hugging Face.

Videos

Images