Digit's IROS tote demo ran on about 500 teleop demos
Chris Paxton gave up the training recipe behind Agility's booth demo, and was blunt that it is research, not production.
We now know what was under the hood of Agility Robotics' Digit demo at IROS. Jonathan Stephens talked to Chris Paxton, who built it, and posted the recipe. About 500 real-world teleoperated demonstrations (a human controlling the robot remotely) trained the manipulation policy. A whole-body controller was trained in simulation. The whole thing was then dropped into a venue the robot had never seen.
Chris was also very clear that this is still research, not something currently running in production.
The task was plain. Digit picked objects out of a tote, walked them across the booth, and repeated until the tote was empty. Stephens puts success at roughly 80 to 90% in the booth, with crowds, harsh lighting and distractions all around it. He called it the most impressive demo he saw at IROS, and also one of the simplest.
Teleop is in the data, not the demo
Worth being precise about where the human sits. The demo itself ran autonomously. The teleoperation is upstream, in the 500 demonstrations used to train the manipulation policy. That is a normal setup for learned manipulation, but it is the detail that gets lost when a clip travels without context.
Paxton was also explicit about the status of the work. Stephens wrote that Chris "was also very clear that this is still research, not something currently running in production." That matters because booth demos get read as product announcements, and this one is not.
What it adds to the earlier number
Paxton had already said the demo ran mobile manipulation for long stretches at the show, which we covered earlier. What was missing then was the recipe. Now there is a demo count, a controller split, and a success rate.
The number that should get attention is 500. That is a small pile of data for a policy that then held up at 80 to 90% in an environment nobody trained for, under trade show lighting with people crowding the workspace. Stephens framed it as a look at how fast learned manipulation policies are moving out of clean training rooms and into messy rooms.
Paxton, for his part, kept it short on his own account. "It was really cool to get a chance to show our robot doing whole-body mobile manipulation using end to end policies," he wrote.
It was really cool to get a chance to show our robot doing whole-body mobile manipulation using end to end policies.
What the sources do not say is how many total picks that 80 to 90% covers, or how failures were handled when they happened. Nobody has published that yet.