Karpathy: Opus 5 turns Lord of the Rings into a 3D world with 5,500 lines of code
Karpathy’s Pelican
Andrej Karpathy gave Opus 5 the first paragraph of Lord of the Rings and a $10 token budget, asking for a Three.js render. The model spent two hours writing 5,500 lines of code that procedurally rendered the story into a janky but fun 3D scene. Karpathy sees this as a step beyond simple SVG tests, envisioning on-demand, hyper-custom worlds like an ephemeral GTA. He notes LLMs still struggle to audit their own work in such domains, as Opus had to take slow, error-prone screenshots.
It's an example where we go from "no one would ever do this" to "sure, why not, it's ~free".
- jmugan
A lot of people are posting here about how bad the end product is, but that is kind of the point. Models have moved beyond generating images to a new kind of benchmark that better exposes understanding of the physical world, and we can use benchmarks like this to measure future progress. (Of course, it will have to be a qualitative/subjective measurement.)
- bredren
I worked with an LLM to build a ~3D animation of the Back to the Future delorean Time Machine as a way to spice up the hero on a docs page.
That took a fair amount of custom tuning and I had to create a tuning view to get some of the behaviors right.
But it was enough fun that I generalized it to take in ~any scene description from a film. It goes out and gets more detailed descriptions and film stills if available but also takes custom stills if you provide them.
My test scene was the Gauntlet scene from Apocalypto. It is low fidelity but does a pretty amazing sequence with somewhat believable physics of the javelins etc.
Here is the docs page with the vertical takeoff / 88 miles an hour time travel: https://contextify.sh/docs
I can share some of the Apocalypto bit if anyone is interested.
- HarHarVeryFunny
It seems pretty clear that Anthropic models have been specifically trained to be good at generating three.js (JavaScript 3-D Graphics) code, so given current state of AI code generation in general, I don't find three.js models/animations as indicative of anything other than the model's ability to write three.js code.
When Fable was first released the day-1 demos of it on Twitter (presumably from people who were given early access, and/or Anthropic employees) were pretty much 100% three.js stuff. Yes, it looks nice, but it doesn't tell me any better than an Erdos proof whether the LLM will be able to run my vending machine.
- qwertox
I'd rather have them battle on the topic "Who builds a better Google Wave for LLM chats" to explore the space of how AI studios could be.
"Getting Started with Google Wave": https://www.youtube.com/watch?v=eKUAqNGVwX0
- dundarious
I can forgive the modeling being godawful jank (windows floating in the air, disconnected from the house). But I expected it to have a better understanding of the text. Instead, we have Bilbo's "disappearance" interpreted as him magically transporting or cloaking, and similarly for his reappearance.
- toolslive
Reading the title, I was thinking "Karpathy? I don't know this chess player."
(The Pelikan is a well known chess opening, and famous chess players often have book titles like "X's Y" where X is the player, and Y is the opening)
- fzeindl
Regarding the argument about LLMs having difficulties auditing their work:
I wonder whether we are entering the era of throwaway software. Just like cheap plastics and improved processes has enabled us to rapidly manufacture anything we want for a very low price, maybe LLMs give us the same for software. Produce it cheaply and if it breaks throws it away and reproduce it.
- mold_aid
The tilde thing remains uniquely obnoxious in a field that seems want to mangle language for fun, so that's innovative I guess