Swiftlet - Run 80B Qwen in 4.3 GB RAM on Mac, 35B on iPhone
Show HN: Run an 80B Qwen in 4.3 GB of RAM on a Mac, and a 35B on an iPhone
Swiftlet is a Swift + Metal runtime that brings Qwen3-Next and Qwen3.5/3.6 MoE hybrid models to ordinary Apple devices, including iPhones. By keeping only the small dense core resident and streaming routed experts from storage on demand, it runs an 80B model in just 4.3 GB of RAM on a Mac and a 35B model on an iPhone. With runtime-compiled Metal shaders, it achieves 4.5-11 tokens per second on M5 Macs. The project includes a CLI, an OpenAI-compatible server, and an open-source iOS app (Priv AI) for on-device chat. Built with Claude Code, it's a breakthrough in local AI accessibility.
As far as we know, that is the first time a model of this class has run natively on a phone.
- dghlsakjg
I know everyone wants to crap all over these setups that are impractical, but this is how progress happens.
People will keep plugging away at this and figure out how to avoid wearing the hard drive, how to make it run faster, custom hardware buses etc.
Keep going! I personally can't wait for the day when a 1t param model runs off a $200 SSD instead of a $50k rack of Nvidia chips.
- CyLith
I know relatively little about the workings of LLMs, but I keep seeing projects like this that run massive MoE models using very modest amounts of RAM, perhaps excessively so. I wonder, is there a way to make the RAM usage tunable? I have a Macbook with 32 GB of RAM, and it'd be great if I could run the same model but take advantage of the additional RAM to make it run faster.
- AHASIC
I read a comment on here a few months back I wanna restate. Basically, there is a good chance that Apple is betting that the LLMs in the future will be so efficient that those that consumers will use everyday will be easily computed by the iPhone or even bigger ones on Macs. Honestly makes the most sense that we are heading that way in a few years latest.