Explorative Modeling: Unlocking a Third Pretraining Axis for Generative AI
Explorative modeling: Train on the best of K guesses
I introduce Explorative Modeling, a new paradigm that solves the averaging problem in generative tasks by training on the best of K guesses. This approach enables true end-to-end generation, offering massive gains in sample and FLOP efficiency while matching diffusion models on control tasks with significantly less compute.
The 'optimal' guess is a spot that no darts ever land.
- Straw
Although this is a neat idea, the author appears to be confused about how generative modelling works.
They repeatedly claim that previous approaches rely on factorization to reduce things to guessable chunks to avoid the 'blur problem'. This is a misunderstanding. Previous approaches solve this problem by modelling a distribution as output rather than a point. Factorization is one way of representing the distribution, but the key point is that even for the small chunks we predict an output distribution, not a point estimate.
Factored models have no issue producing TV static as an image. Each individual guess is impossible, and yet they don't produce a constant grey averaged image.
Normalizing flows, though not considered particularly efficient, are not factored at all, do not proceed in small steps, and yet still have no issues with blur because they predict a distribution.
Their approach appears to be a hard version of a latent variable model. It may be a good idea, but it isn't a fundamental change in the way they suggest.
- ollin
This paper shows a nice integration of older winner-take-all ideas for learning K-modal generative models (see e.g. https://arxiv.org/abs/1612.00197, https://arxiv.org/abs/2211.14286) into modern diffusion/flow pipelines. As-implemented, I think it has some downsides:
1. K-1 extra forward passes during training
2. Inaccurate sampling behavior (will sample all K modes with equal likelihood, rather than sampling them proportionally)
However, I think both of these downsides can be mitigated by adjusting the implementation a bit more (you can have the model predict K modes jointly in one forward pass, along with probabilities of each being the min-loss mode, which you can then use for properly-weighted mode sampling at inference).
That said, I'm not entirely sure if this strategy is as generally applicable as the authors are hoping. In particular:
1. For highly-conditional image generation (like modern commercial diffusion pipelines, which use a big LLM preprocessor), most of the low-frequency color/layout decisions are already made for you by the conditioning signal. The diffusion process mostly needs to generate high-frequency details, for which there are a huge number of equally-valid modes.
2. For LLMs themselves, the sequence-generation process is already factored into a discrete classification problem, and there's no mode smearing issue to fix.
- kamranjon
This is amazing and I think will probably end up being a pretty important development.
I was just reading this great breakdown of how diffusion Gemma works: https://newsletter.maartengrootendorst.com/p/a-visual-guide-...
In reference to the difficulties with applying this to autoregressive LLMs - I wonder if these type of hybrids might be a good candidate for this approach.