Paxton waves off paper on tricking robots into stabbing
A CoRL late breaking result says corrupted camera input makes frontier models act dangerously. One well known roboticist says that proves nothing.
Robocurve PBC says it can fool frontier models into doing dangerous things by tampering with what they see. In a post about the work, the account reports a 100% attempt rate on stabbing a baby doll, along with producing toxic gas and pouring water into a toaster. The team says the result held across every frontier model it tested.
Video tampering can mislead every frontier model we tested into harmful actions
The work is being presented at CoRL Late Breaking Results, under the title "You Can't See Everything", with the subtitle describing perceptual attacks on embodied AI.
The pushback
Chris Paxton was not impressed. His reply, in full, was "I too would stab a baby if I thought I was seeing an apple. Come on guys, this is nothing".
I too would stab a baby if I thought I was seeing an apple. Come on guys, this is nothing
That is the whole argument in one line. The disagreement is about framing rather than about the numbers. If you corrupt the video feed going into a model, the model acting on a false picture of the world is a perception failure. Calling it a safety failure, Paxton's point goes, makes it sound like the policy itself is dangerous when it is doing exactly what you would expect given bad input.
The counter view, implied by Robocurve framing it as an attack, is that real robots will run in places where the camera feed can be degraded or manipulated, and a system that happily stabs when its eyes lie is a system you should not deploy. Nothing in the two posts resolves which reading wins.
What is shown and what is not
The posts do not say which frontier models were tested, how the video tampering was done, or whether these were physical robots or model outputs scored in some other way. Robocurve says attempt rate, not success rate, which is a different thing from a robot actually completing a harmful action. The full paper is due at CoRL Late Breaking Results, and until then nobody outside the team has seen the method.
Sound like you were at the demo
One email a day. Under three minutes. The demos, the deployments and the part the press release left out.