Grounded ships an API that turns everyday video into robot data
The company claims state of the art hand tracking under 1 cm, and Lerrel Pinto says the hard part is still turning human data into robots.
Grounded Superintelligence says its API is live, aimed at turning ordinary first-person video into training data for robots. The company claims state of the art results on hand-tracking benchmarks at under 1 cm, state of the art results on SLAM benchmarks (simultaneous localization and mapping, working out where the camera is while building a map of the scene), and in-the-wild egocentric video converted into enriched data in minutes.
In-the-wild ego data -> enriched data in minutes
It plugs into tools people already use. The post lists integrations with Hugging Face, LeRobot and Rerun, and says the API is built for BitRobot Network's RoboCap suite. A technical report is linked from the launch post.
Why researchers reacted
Lerrel Pinto said the field is heading toward an abundance of human data and called it good to see Grounded pushing on the infrastructure layer for it. His caveat is the interesting bit. He said what is less clear is how that gets turned into an abundance of robots.
Jiafei Duan said RoboCap is the way forward for egocentric data collection, and that Grounded's model will significantly increase the quality of robot ego data collected in the wild.
Shown versus claimed
The numbers are the company's own. The benchmark claims come from Grounded's launch post and its technical report, not from an outside evaluation, and nobody in the sources has independently run them. None of the posts show a policy trained on data from this pipeline beating anything on a robot.
What is actually new here is plumbing, not a robot demo. The pitch is that you point a head-mounted camera at a person doing a normal task, push the video through the API, and get out tracked hands and camera poses in a format a training pipeline can read, in minutes rather than whatever it took before.
The reason people who follow this care is the bet underneath it. Teleoperation (a human driving the robot remotely to record demonstrations) is slow and expensive per hour of data. Human video from a head camera is cheap and there is a lot of it. If the tracking is accurate enough, the argument goes, that becomes the main data source for manipulation. Pinto's line is the honest version of where that stands. The supply of human data is arriving. The conversion step is not solved yet.
What’s less clear is how we’ll turn that into an abundance of robots. The next few months should be fun
RoboCap is definiitely the way forward for Egocentric data collection