Google's EmbeddingGemma 2 unifies text, images, audio, and video on-device
Google's EmbeddingGemma 2 is a 740M-parameter multimodal embedding model built on Gemma 4 and released under Apache 2.0. It maps text, images, audio, and video into one shared space, supports up to 8K tokens, and runs on a Pixel 11 Pro with as little as ~191MB RAM for text-only weights. It leads sub-1B models on MTEB Code and MAEB, and its modular encoders and Matryoshka vectors cut storage up to 6x.
It can help find a specific video clip from a voice memo, or search through hours of audio recordings based on a text query, all processed by a single, natively multimodal model.
- simonw
I really appreciate that EmbeddingGemma 2 is under the Apache 2.0 license.
For embedding models in particular, I don't think it makes sense to use a closed, proprietary, hosted-only model.
Most applications of embedding models involve calculating thousands or even millions of embedding vectors and storing them for later comparison.
If your model is proprietary, the vendor is likely someday going to decide to stop offering that model. They'll have a better model to replace it, but you still need to pay to re-calculate those millions of stored existing vectors.
(In April 2024 OpenAI offered to "cover the financial cost of users re-embedding content with these new models" - https://openai.com/index/gpt-4-api-general-availability/ - but I don't think that's something we can rely on from every provider.)
Notably, I don't want to host the model myself. I'd much rather pay a provider for a hosted model while knowing that if they ever stop hosting it I can run the open weights version myself - or find another vendor who can do that for me.
- Nautman
It's also very neat that this can be used for "Jev"-like tasks with text and image.
https://developers.google.com/edge/mediapipe/solutions/decis...
- flockonus
Hats off to google for offering OSS (or at least open weights + license) a model that would be probably pretty closed to what they would ship in their Android phones.