Real robot catches the error in RoboSimStudio's AI-built sim
The agent modelled a bar support as fixed, the hardware proved otherwise, and the fix went back into the simulation.
The headline number on RoboSimStudio was the 56 minutes it took an AI agent to build a physics twin of a robot task from a smartphone video, which we covered earlier. The more interesting part is what happened when that simulation met hardware.
The task is bimanual, one arm holding a bar, the other wiping a marker stripe off it with a foam sponge. In simulation the bar support was modelled as fixed, so the sponge kept contact through the whole wipe. On the real robot the support moved, the bar shifted, and contact was lost.
The agent fixed its own model
The agent went back into the twin and replaced the fixed constraint with a free rigid body. The revised simulation then reproduced the tipping behaviour the original version could not. That is a full loop, from video to sim to real hardware and back into the sim, with the error found by physics rather than by a human reading code.
RoboSimStudio comes from researchers at UC Berkeley and NVIDIA Robotics and is built on Newton. It is an open toolkit, and every physics parameter is exposed as something a human can move with a slider or an agent can edit directly. That matters for this loop. If a parameter is buried, the agent cannot correct it after a real world run.
No CAD, no calibrated cameras
The input was a smartphone video. No CAD files of the objects, no calibrated camera rig. Claude and GPT-6 Astra each built a simulation of the scene independently, and one of them finished in 56 minutes.
Shown versus claimed
What is documented here is one task, one correction. The sources describe a single wiping demo and a single modelling error found and patched, not a general claim that agents can debug any simulation against any robot. Whether the same loop holds across harder contact physics, deformable objects or longer tasks is not something the sources address.
Still, this is the part of digital twin work that usually gets skipped in demos. Building a sim fast is a parlour trick if nobody checks it against reality. Here somebody checked, the sim was wrong in a specific and ordinary way, and the correction was automatic.
Lukas Ziegler's thread has the project link and the full walkthrough.
That is the loop actually closing. The real world found the modelling error and the fix went back into the twin.
Open toolkit, every physics parameter exposed as something a human can slide or an agent can edit.
Sound like you were at the demo
One email a day. Under three minutes. The demos, the deployments and the part the press release left out.