Lumabri - Run huge MoE models on a P2P swarm
Show HN: Lumabri – Run Moe Models on a P2P Swarm with Colibri
Lumabri lets you run massive mixture-of-experts models like GLM, OLMoE, and DeepSeek on a peer-to-peer swarm. One machine shares a model, and any other machine can chat with it without downloading the entire model upfront—bytes are fetched on demand and cached locally. Built in pure C with no dependencies, it works on CPU and SSD first, with GPUs only making it faster. Anyone can join, even without a GPU, and donate disk or compute to help the swarm. It's a new way to democratize access to huge AI models.
Networks that pool GPUs recruit from the few; lumabri recruits from everyone.
- brainless
I am sorry I did not understand all of it. But, would this allow running large MoE LLMs on a local network with experts spread out over multiple cheaper GPUs (or even CPUs)? This would perhaps be more useful than over the Internet, within offices for example.
- s2l
What if one wishes to use various busybox nodes within the house?
All the iot devices contributing to matmul but within a LAN?
- snovv_crash
Cool idea. How do you handle temperature in the verification?