Mistral's Robostral Navigate: Single-Camera AI Beats Multi-Sensor Robotics
Mistral's Robostral Navigate: a state of the art robotics navigation model
We are thrilled to introduce Robostral Navigate, our new 8B model that enables robots to autonomously navigate complex environments using only a single RGB camera. By combining simulation-trained data with efficient reinforcement learning, we achieve state-of-the-art performance on unseen benchmarks, outperforming systems that rely on LiDAR or multiple cameras. This breakthrough makes advanced embodied AI accessible and adaptable for real-world applications across manufacturing, logistics, and hospitality.
Our method transforms training runs that would take months into runs that complete in days.
- iandanforth
It's implied, and I'm hoping it's true, that this is a map-less navigation. Which is impressive. This kind of task is much easier if you have a pre-captured map of the environment, but if they are doing this without a map it's great. Historically you were always faced with "The Kidnapped Robot" problem where robots that didn't know where they were couldn't navigate even a little bit. Here the robot appears to be able to follow directions as long as they are interpretable from its current vision (or via dead reckoning).
- HanClinto
What is the realistic path to getting to play with this? I would love to hook this up to OpenClaw for hobbyist exploration. My dream has been to embody OpenClaw into a farm robot (been looking at adapting one of those RC lawnmowers that is tracked and built for mowing steep hills) so that I can assign it various tasks around our acreage -- "Explore the fenceline take pictures of the plants. Find all of the poison ivy and invasive honeysuckle and spray it with your Roundup sprayer. Repeat this every week and report the species map after every pass. Come back to the barn and charge yourself whenever you get low."
It's not hard to put OpenClaw into a robot body (numerous YouTube videos showing people doing this sort of thing), but when you dig in and see what people have done, the actual movement portion is always the clunkiest part (and this matches my own experiments as-such as well). It feels like an 8B model like this would be perfect for solving pathing and navigation issues.
Anyone who may be more experienced with Mistral (or companies like them) -- are they interested in hobbyist builders who would be experimenting with things like this? Or are they primarily looking for commercial partners? I would be willing to pay a license fee to use the model in my experiments, but if I'm just one guy, I'm not sure they'd want to work with me unless I were building a business out of it (which I'm not).
- humanperhaps
This looks to not be an openly available model, but I think if it were, availability of an easy single-camera navigation setup could allow for a lot of cool hobbyist projects.
- dwa3592
This is very cool. Congratulations to the Mistral team. Map less navigation in the outside world has been around for quite a while. But map less navigation inside the buildings is relatively new. Some stanford researchers trained a vision model (PIGEON) which could tell the geo-location from any image. It was not released publicly due to privacy nightmarish (stalking!) possibilities but I am assuming similar type of tech has gone behind this robot. if someone knows more, feel free to correct.
here's the link to the PIGEON paper - https://lukashaas.github.io/PIGEON-CVPR24/
- Gecko4072
Mistral seems to be going wide and niche. Could be a smart strategy going forward.
- montroser
I'm ready for my home helper robot that makes dinner and does the dishes and takes out the trash.
But I'm scared for when those home helpers get drafted to fight in wars, either for or against me...
- aktenlage
I'm impressed, because it is so minimalistic.
On the other hand it reminds me of 2010ish robotic demo videos from academic researchers and willow garage.
Problem with robotics is usually that it's easy to produce a good-enough looking demo, but it's really hard to make somethin work in the general case. As exemplified by autonomous driving.
- mil22
> achieves 76.6% on R2R-CE (Room-to-Room in Continuous Environments)
I would like to know what it did the other 23.4% of the time!