Edge0-145B runs a 145B-parameter AI model on a laptop with just 29GB of memory

I'm really impressed with the Blender examples. It's amazing how far "tool use" has come, when it now means using a tool as complicated as Blender to perform work that would take a human days to complete. I'm a little skeptical the actual models are workable, in the sense that a human could come in after and manually refine and apply better textures, etc. Historically, agent-made designs are messy as hell, and by the time you clean them up, you haven't saved any time. But, that's begun to get better; the best models are now very good at making clean shape-based SVGs, for example, that are no messier than what a human would make in Inkscape or Illustrator. But, I'll definitely give it a try.

> Sharing matrices substantially reduces the weights that must be stored and moved while keeping the expert count unchanged. Each group of 64 experts shares two large matrices, while each expert retains a small set of parameters that captures its differences.

145B parameters with only peak 29GB memory usage - really impressive, truly something you can actually run on a decent spec MBP

Is this a derived from flash next? That's the impression I get from reading over everything. Using Strata + Qwen 3.8 Flash Next now and I am beyond impressed. It's basically as capable as 5.6 Luna in my hands so far. If this is an upgrade I can't wait to try it out.

Thy have shipped this as NVFP4 so its Nvidia blackwell only, presumably their inference engine doesn't support anything outside CUDA 13 specified either. Will be interesting to see if this gains some other quants and other inference engines pick it up and we get some standard GGUF files, its a bit of weird release at the moment rather than GGUF file downloads the variants link on HF is leading to a directory of stuff which contains the model spread across a variety of files.

> Runs on a compatible personal laptop, such as a MacBook Pro

I'm curious, do normal people (so not HN professional developers or entrepreneurs) have that, as actually personal device (not from their tech company)?

I imagine some might have but I feel, and that's me so I'm asking the broader community here, that the average PC user has at best a gaming rig, but that's already a minority, and a lot more often a low end laptop with 16GB of normal RAM, not unified or VRAM. Am I out of touch?

It's amazing the level of optimizations we are discovering. At this rate, I wouldn't be surprised if the trend starts reversing, ie. instead of releasing bigger and bigger models, companies start releasing smaller, more capable, specialized models for specific applications

Over the past few weeks, I've been putting together a small, 6.5M param model. It is conversational, does tool calling, RAG, mathematical modeling/calculations, basic physical simulations, and multi-step chained reasoning. It's still very much _not_ a frontier model. But it already does more than I could get a qwen 2.5 0.5b param model to do with finetuning just 1 year ago

Most of the improvements come from being able to direct claude to apply different ideas from recent research that keep coming out. AI labs seem to be focused on 1B+ models, so not much comes out about small models, but many of the things being discovered in research with billion+ model, are adaptable to small ones as well

Harness improvements like context pruning, malformed tool call healing also very helpful to boost smaller local models. Doing alot of work in this area lately.

Qwen 27B in Terminal Bench 2.1 73->85 for instance.