Ternlight: A 7 MB Embedding Model Running Entirely in Your Browser
Ternlight – 7 MB embedding model that runs in browser (WASM)
I built Ternlight, a tiny embedding model that fits in just 7 MB and runs directly on your CPU without needing any API calls. You can drop it into your project as a single npm package to perform semantic search in milliseconds. Whether you are using the full version or the 5 MB mini variant, you get fast, local embeddings with zero network latency, making it perfect for tasks like searching React docs right in the browser.
Drop it in, embed text in milliseconds, and never call a server.
- soycaporal
Hobby project, I wanted to "ship a useful model in a web browser". so I distilled a small sentence encoder from MiniLM with ternary quantization-aware training. Also wrote the inference engine from scratch and shipped in Rust → WASM SIMD.
It's an embeddings model, not an LLM: text goes in, a 384-dim vector comes out, and cosine similarity between two vectors tells you how related the texts are — regardless of shared words ("reset my password" ↔ "I forgot my password" → 0.88). Used for semantic search, FAQ/intent matching, and clustering. Running it on-device means search-as-you-type semantic search is performant with no API dependencies.
Demo (2k React docs, fully on-device): https://ternlight-demo.vercel.app
Two tiers on npm:
- @ternlight/base (7 MB, ~5 ms/embed, more capable embedings)
- @ternlight/mini (5 MB wire, ~2.5 ms/embed).
Bundled for Node and browsers.
Repo - see technical details (MIT, training pipeline included): https://github.com/soycaporal/ternlight
Curious if this is something useful, what are the use cases for on-device embeddings.
- dirteater_
This is cool!
but also maybe you could put a button on the landing page to trigger the demo because it's a bit startling to hear my fans go crazy when opening a webpage.
- kamranjon
This would be a pretty cool addition to the duckdb HNSW search project I found on here some time ago: https://github.com/jasonjmcghee/portable-hnsw
What I think is really cool is that the search happens using http range queries across statically hosted parquet files.
I think things like this could bloom into a relatively open and distributed search ecosystem that isn’t controlled by major corporations.
- aziis98
This would be nice as an Astro (or generic meta-framework plugin) that automatically parses all generated html files and generates a small db of embeddings.
This way on the frontend you can lazily load this. Maybe you could even store the HNSW in chunks and just load the pieces you need for your specific search query.
i.e. like https://pagefind.app/ but to get fully static vector search.
- aetherspawn
Can the 30 second embedding time be done beforehand and sent to the browser?
Inference is nice and quick after that.