A model SSH'd into a robot and learned cornhole in 6 tries
Takahiro Miki says the model found its own way onto the robot, built a sim scene, then landed the throw on the sixth real attempt.
Takahiro Miki described running a demo where he prompted a model to connect to a robot, and it worked out how to get in over SSH (a standard way of logging into a machine remotely) by itself. When he asked it to move the robot's neck, it went through the codebase and figured out how to command each joint.
Then came the game. Miki says they showed the model cornhole, a bean bag toss game, and it recorded video of the play. From that it built a simulation scene and optimized the motion inside it. When Miki asked for the motion to be dynamically stable, it started checking ZMP (zero moment point, a classical balance criterion for legged robots) and optimized for stability.
After that it moved to hardware. Miki says it iterated on the real robot and succeeded after only 6 trials, which he credits to very good spatial understanding.
The tools, not the weights
The part Miki singles out is not the neural network doing everything end to end. It is the model picking up decades-old robotics software and using it like a tool. He says it used MuJoCo, a physics simulator, for forward and inverse kinematics, dynamics and collision detection, then reached for a set of optimization methods to generate the motion.
That is a different shape of story from most humanoid demos, where a policy is trained on a pile of demonstrations and then run. Here the loop looks more like an engineer. Watch the task, build a model of it, solve it with the classical math, test on hardware, fix what breaks. The self-improvement loop of watching a demo and practicing in simulation before going to hardware has shown up elsewhere recently, including an earlier real to sim to real result.
What is not in the post
Miki does not name the model, and he does not name the robot beyond saying it has a neck and joints he could ask it to move. There is no success rate across repeated runs, no detail on how much human prompting sat between each step, and no indication of how many of those 6 trials were failures that the model diagnosed on its own versus with help. Six trials to a working throw is the headline number, and it is the only number in the post.
So treat this as one researcher's account of one demo, not a benchmark. What is concrete is the sequence Miki reports. Connect, read code, watch, simulate, optimize for stability, then land it on the real machine.
I was impressed how good it can use the classical robotics method as a tool.
It had a very good spatial understanding and with only 6 trial it already succeeded.
Sound like you were at the demo
One email a day. Under three minutes. The demos, the deployments and the part the press release left out.