DeFM is a foundation model that sees only depth
Manthan Patel put the weights, code and a demo out the same day he announced it.
Manthan Patel has released DeFM, a foundation model (a big pre-trained backbone that other systems build on top of) made for depth images rather than RGB photos. Weights, code and a demo went out the same day as the announcement.
Vision foundation models like DINO and SigLip now power much of robot learning. But they are built for RGB. What about depth?
The pitch starts from a gap. As Patel puts it, vision foundation models like DINO and SigLip now power much of robot learning, and they are built for RGB. Depth, the channel that tells a robot how far away everything is, has not had the same kind of general purpose backbone. DeFM is his answer to that.
What the demo shows
The clip Patel posted is a feature visualisation, which is the standard way to show what a backbone has learned. Feed it a depth image and different parts of the scene light up as distinct features. He points to stairs, railings, walls and car parts each coming out separately, from depth alone, with no colour image anywhere in the input.
That is the interesting part. Stairs and a railing are easy to tell apart in a camera photo because they look different. In a depth image they are both just geometry, so a model that pulls them apart has learned structure rather than texture.
What is not out yet
The sources do not include benchmark numbers, model size, training data, or any downstream robot policy running on DeFM. Nobody has posted a head to head against an RGB backbone either. So treat this as a release with a demo attached, not a proven result.
The useful thing is that you do not have to take anyone's word for it. Weights and code are public on day one, which means anyone outside the team can load it, run it on their own depth data and say whether it holds up. That is a faster feedback loop than most announcements get.
Chris Paxton, who follows this area closely, summed it up as a depth-first foundation model for robotics. Early reaction, not a verdict.
The only input is a depth image. Yet stairs, railings, walls and car parts each get their own features.
Sound like you were at the demo
One email a day. Under three minutes. The demos, the deployments and the part the press release left out.