Meta's Muse Glimmer: A 30B Open-Source Model That Runs on Your Device

Remember when we needed 200 servers for an enterprise website because Apache used one process or thread per connection - and Nginx collapsed that into a single box overnight? That moment for LLMs is near. It’s going to move us from the big iron era of AI to small portable brains. Nature has already proved it’s possible with 20 watts and very little heat generation. And I think the data center buildout will end in carnage.

https://xcancel.com/finkd/status/2086755195535413696

"... Soon we'll also release the weights for Muse Spark 1.2, our latest foundation model..."

This is bigger news - good for self hosting enthusiasts and a strategically sound move for Meta. Any push towards 'anti Chinese' models will directly benefit Meta as the competition on the frontier open-weights American models is almost non-existent. Meta will have no problem being #1.

PSA: Fast RAM isn't going to be getting cheaper anytime soon. Acquiring inference hardware is a really good way to own an appreciating hard asset. Learning how to use it and cool it is a hacker's journey worth taking. My 4090 I bought in late 2022 for $1600 is selling for a cool $3,489.95 right now, and going strong under nominal use. My DRR5 has tripled in value, my nvmes almost doubled. I grabbed a 128GB M5 Max MacBook Pro when they were still available and told all my friends to buy at least one. With that and a base M4 Studio 36GB, HuggingFace rates that hardware as:

> Amazing!

You have a total of 128.94 TFLOPS of computing power. 71.3% percentile on scale of "GPU Poor" to "GPU Rich"

The way I see it, these are amazing machines that the richest folks are hovering up. I think they should be in the hands of regular people as much as possible. They depend on an incredibly global, increasingly fragile supply chain. If the become impossible to produce, their value would increase tremendously. I think they will become really valuable to you to use the tokens directly, but if that isn't the case, they can be rented out or resold. Please don't just buy any hold. Let's try to get as many people that can use them for decent things that help humans. For example:

https://spectrum.ieee.org/small-language-models-ai-pharmaceu...

Meta is rocking AI. As of last week I have been using their excellent muse coding harness with their model Muse Spark 1.2.

Starting this morning I am running their new local 30B model muse-glimmer on my old MacMini 32G using Ollama (remember to increase the context size!) and pi coding harness. I am getting good results with muse-glimmer running locally, with the caveat that everything runs slowly (e.g., give it a task and then go walk outside or do Qi Gong exercises for a while).

The gguf is up and works, I don’t know if it’s them or unsloth that’s facilitated this but it’s nice because e.g. Inkling still doesn’t appear to have support in llama.cpp which makes it irrelevant to a class of user.

Unfortunately I don’t have enough experience with Qwen 27B to immediately compare, but I do it’s Qwen 3.6 35B A3. It’s much slower obviously but it seems to be way more efficient with its thinking to the point that using it might actually be faster. I find Qwen and some others rehash the same things over and over when thinking without getting anywhere, in mg limited checks here Muse is much better.

With the business model for API based LLMs looking iffy at best it seems like we’re heading back to the “server under your desk” era of IT again.

Some interesting findings from the chat template designs:

1. The template name is Onyx ATEM as found in the tool call exception message

2. It appears to be following a harmony-style chat template. But the tool use seems to be a xml like :<atem:function_calls> / <atem:invoke> / <atem:parameter>

3. atem: a internal joke of meta in reverse?

https://huggingface.co/meta-models/Muse-Glimmer-30B/blob/mai...

It is interesting but it does look like a careful distillation of (Spark and) biggers open-weight models.

The progress compared to Qwen3.6 27B is good, not that impressive, it's a 4 months old model. (kuto to them to compare to 27B dense and not 35B MoE, it's more fair to do so).

It is very probable that Qwen3.8 27B will crush Glimmer-30B on most benchmarks.

Unsloth has quantized versions uploaded: https://huggingface.co/unsloth/Muse-Glimmer-30B-GGUF

The quantized releases often change in the weeks following release as new improvements are discovered, so either use a tool that checks HuggingFace for new versions or manually check back in a few days or weeks to check for improved versions.

Initial reports are good. It hasn't been out long enough for anyone to really test thoroughly, but the people I know who have stable non-public test cases are reporting impressive results compared to even Qwen3.6 27B. That's a good sign that this might not be benchmaxxed (trained to excel at public benchmarks with less impressive performance on general tasks) which has been becoming common with recent releases.

www.reddit.com/r/localllama is a good place to keep up with the details from people who are actually using it. It feels strange to recommend a subreddit over Hacker News, but on this topic the /r/localllama threads are much more on topic right now if you're looking for information about the model.

There are some initial reports that even the 2-bit quantization is looking somewhat usable. That might make it small enough to squeeze into 16GB GPUs. I'd take those reports with a grain of salt because early tests are often optimistic and I've yet to see good results from anything 3-bit or less, but it should be fun to experiment with.