Astra demos hit a reproducibility wall
Three well known researchers say the viral clips look great and nobody outside the posters can repeat them.
The Astra clips keep coming and the people who build robots for a living have started asking for receipts.
Chris Paxton, who has been picking apart robot demos all month, posted that "everyone still astra-posting and yet i seem to keep finding problems it can't solve." Chen Tessler put it more bluntly with a photo of pizzas he had made, saying the posts look amazing but he has no idea how to reproduce them. Ken Goldberg went at the sourcing, asking of one result why authors with no previous publications had posted something potentially major anonymously. Jiafei Duan asked a poster the simplest question of the lot, how much a single run costs.
everyone still astra-posting and yet i seem to keep finding problems it can't solve
What is actually circulating
Individual clips. The sources show no success rates, no trial counts and no reproduction by anyone outside the people posting them. That matters more than usual here because outsiders who did get hands on Astra earlier hit walls that the viral clips do not show.
Paxton is not a pure skeptic on this one. He also said that a commercially available model doing this well is still pretty impressive to him, and that he is going to have to rethink his life. Two things can be true. The model can be genuinely good and the evidence being shared can still be unverifiable.
The fake paper guide, again
With the ICRA deadline days away, Paxton reposted his satirical guide to faking a robotics paper. The list reads like a checklist of what is missing from the Astra threads. Do not let other people run comparisons. Control the demo environment carefully, lighting, objects, initial robot configuration, so you can overfit the scene and get smooth motions. Never show your failures. Pick weak baselines. Cherry pick your benchmarks. He ends it with "obviously don't do these things please", and he singles out Physical Intelligence and NVIDIA as the counterexample, labs that open source their models so anyone can benchmark them.
Shown versus claimed. Nobody in these sources says the Astra results are fake, and Goldberg explicitly allows that Astra is good at many things. What they are saying is narrower and harder to argue with. A demo you cannot run yourself is a demo, not a result.
The thread to watch is Goldberg's. An anonymous account with no publication history posting a potentially major robotics claim is the kind of thing that either gets a name and a method attached within a week or quietly stops being mentioned. Nobody in these sources has verified it yet.
They look amazing, but I have no idea how to reproduce them.
This is potentially major news but the authors have no previous publications…Why is it anonymous?