Teaching a World Model to Play Pokémon
Teaching a World Model to Play Pokemon
A developer trains a JEPA-based world model on Pokémon Red to select a starter Pokémon from Professor Oak's lab. The model learns to predict future screen embeddings, but suffers from latent collapse, where all embeddings become identical. The fix is SIGReg, which forces embeddings to follow an isotropic Gaussian distribution. After rollout fine-tuning, the model successfully plans a sequence of button presses to choose Squirtle.
In the worst case, the encoder starts to learn to embed every screenshot as the same embedding c. Then, the predictor also begins to learn to predict that the next embedding will be c. This is called latent collapse and means that the model has found a trivial way to reduce the loss function–by collapsing all embeddings to the same embedding.
- dang
Dare we have two AI-plays-Pokemon threads at the same time?
Show HN: Jev Plays Pokémon Red - https://news.ycombinator.com/item?id=49845172
- stmonty
Author here, feel free to ask me any questions. Though I am no expert in this, just someone trying to learn and have some fun doing it.
- VCFundedGenYer
Why is this a dupe of the other thread?