COWBOY grasps in the wild after sim-only training
One policy drives base, arm, dexterous hand and active vision together, with no real-robot fine-tuning.
Jiahui (Jim) Yang has released COWBOY, a single generalist grasping policy that coordinates a robot's base, arm, dexterous hand and active vision end to end. It was trained entirely in simulation and, he says, deployed zero-shot in the wild, meaning no fine-tuning on the real robot before it ran. The work is a CoRL 2026 paper, with @JasonJZLiu and @LiuMuxinl as collaborators.
Trained in simulation, deployed zero-shot in the wild.
Why whole-body matters
Most grasping demos move an arm while everything else stays put. COWBOY treats the whole machine as one system. The base can reposition, the hand shapes itself to the object, and the cameras point where the policy wants to look, all from the same policy rather than a stack of hand-tuned modules. Yang frames the problem as context. His argument is that policies in everyday spaces have to generalize across objects and across the spatial contexts those objects sit in, so the robot adapts its motion and grasps to both object and scene geometry.
Shown versus claimed
The posts describe the recipe and the zero-shot deployment claim. They do not give success rates, the number of objects tried, or how often the real robot failed. Those numbers would be in the CoRL paper, not in the thread, so nobody reading the posts alone knows how reliable this is yet.
The sim-to-real year
The reaction from the field was about the trend as much as the result. Jiafei Duan replied that sim2real, training in simulation and running on the real robot, is the theme of 2026. That tracks with what has been landing lately. Zero-shot transfer is the part everyone wants, because real-robot data collection is the expensive, slow step, and a policy that skips it entirely is the cheapest path to a new task.
If the paper's numbers hold up, the interesting bit is not that it grasped something. It is that base, hand and gaze learned to move together without a human stitching them.
Sim2real is the new theme of this year!
Sound like you were at the demo
One email a day. Under three minutes. The demos, the deployments and the part the press release left out.