UMI-Live open-sources a two-handed robot data rig

Amish Sethi has released UMI-Live, a bimanual handheld rig that he describes as completely open source and that runs from handheld demonstrations all the way to a trained robot policy.

Amish Sethi

@amish_sethi04

X

You press one button and both hands start recording.

Amish Sethi

@amish_sethi04

X

Can collecting robot data be as easy as 𝗽𝗿𝗲𝘀𝘀𝗶𝗻𝗴 𝗿𝗲𝗰𝗼𝗿𝗱?

The pitch is that collecting robot data should be as easy as pressing record. Each gripper carries its own camera, LiDAR, compute and battery. That means no base stations to mount and nothing to calibrate in the scene, which is usually the annoying part of handheld data collection. One button starts both hands recording.

The pipeline does the boring part

While you keep collecting, the pipeline processes finished episodes in parallel and turns them into LeRobot datasets, uploaded to Hugging Face automatically. So the gap between waving two grippers around in a kitchen and having a usable dataset sitting in a repo is meant to close on its own.

Sethi says VLAs (models that turn camera images and text into robot actions) fine-tuned mostly on this data succeeded in 10 out of 10 trials with π0.5 and 8 out of 10 with MolmoAct2 on a real bimanual task.

Shown versus claimed

Those success numbers cover one task and come from the team itself, with no third party running the same test. Nobody outside the group has reproduced them yet. What is not in doubt is what shipped, because the hardware designs, the build guides, the code and roughly 40 hours of data are all out.

That combination is the interesting bit. Plenty of handheld data rigs have been shown off, fewer have arrived with the files you need to build one and the data already collected on it. If the build guides hold up, a lab or a hobbyist can copy the rig rather than reinvent it.

Chris Paxton, who posts regularly about manipulation research, flagged it simply as an open source UMI solution.