A playable world model runs at 50+ FPS on one RTX 5090
Jimmy Wang put his 3D World Simulator online, and Chris Paxton says the cracks in it are a data problem, not a model problem.
Jimmy Wang released 3D World Simulator, an interactive world model (a learned simulator you can poke at instead of a hand built physics engine) that he says runs at 50+ FPS on a single RTX 5090. It is playable in a browser right now, which is the part that got people clicking.
The model predicts inside what Wang calls a unified 3D representation space. From that it generates novel view appearance, multi-view consistent RGB-D (color plus depth that agrees across cameras), 3D geometry, and spatially aligned tactile signals. That last one is the unusual item on the list. A learned simulator that outputs touch, lined up with the place on the object where touch would happen, is not a standard feature.
Paxton likes it and still finds the seams
Chris Paxton called it the future of robotics simulation and said it is incredibly cool that you can play around with it and interact with it. Then he listed what he sees breaking. The workspace is limited. There are gripper closure artifacts, meaning the model gets wobbly exactly when the hand closes on something. He said he would love to see the same approach run at frontier lab scale.
His diagnosis matters more than the complaint. Paxton says this is another example of the physics problem, and that every time he has seen it, the cause was training data rather than the model. His reasoning is that robot training data is usually too narrowly focused on successes to build a general simulator. A model that has only ever watched grasps that worked has no idea what a slip, a miss or a bad squeeze looks like, so it guesses badly when you drive it somewhere strange.
He also called that an easy issue to fix conceptually, if not practically, and said people are working on it. He went out of his way to say none of it was a criticism of the work.
Why the reaction is the story
Lukas Ziegler added that simulators are one of the most important things to work on in robotics these days. That is the context for why a single person's demo pulled this much attention. Training policies needs cheap worlds to train in, and a learned world model that runs real time on one consumer card is a different cost structure from a studio built simulation.
simulators are one of the most important things to work on in robotics these days
Shown versus claimed. The frame rate and the list of outputs are Wang's claims about his own model, and the demo is public so anyone can try it. Nobody in the sources has benchmarked it against a conventional physics simulator, and the one expert who did try it came back saying the data behind it is too thin.
you can see how incredibly *data limited* this sort of approach is
it's a training data issue, not a model one
Sound like you were at the demo
One email a day. Under three minutes. The demos, the deployments and the part the press release left out.