Beyond LLMs: The Promise and Limits of World Models
Simulating everything, sort of: The promise and limits of world models
I explore how the AI landscape is shifting from large language models to world models capable of simulating physical environments. Experts like Yann LeCun and Fei-Fei Li argue that while LLMs are eloquent, they lack spatial grounding. Companies such as World Labs, Runway, and Google DeepMind are pouring billions into these technologies to enable real-time interaction for robotics and 3D asset generation, marking a potential new frontier beyond the current LLM bubble.
Today, leading AI technology such as large language models (LLMs) have begun to transform how we access and work with abstract knowledge. Yet they remain wordsmiths in the dark; eloquent but inexperienced, knowledgeable but ungrounded.
- kinow
I work in a project with a similar goal, I think, but that was not related to AI: climate digital twins. I worked on two, EDITO (European Digital Twin of the Ocean, edito.eu), and currently on DestinE ClimateDT (Destination Earth Climate Adaptation Digital Twin, https://destination-earth.eu/).
These digital twins aim to simulate climate, oceans, atmosphere, wind speed, temperature, etc., regionally or globally. They apply physics, mathematics, software engineering, and high-performance computing to simulate the world. Both digital twins I mentioned use EuroHPC resources like CSC LUMI, BSC MareNostrum5, CINECA Leonardo, and soon others (and we also use other smaller HPCs).
Now there are some ongoing work about integrating AI/ML, LLM, etc.. I find it a better approach than building for LLM/AI-first, as it integrates existing components, and allows for AI to be used too. I think this could possibly be integrated as a component in a solution like this described in the article, as a component.
(There are other digital twins, for heart, human body, cities, urban climate, rivers, biodiversity, etc.)
- Lawyer24
This is good to know. I’ve been building a world. I’m starting to find more world builders