Mesh LLM: Run Distributed AI on Your Own Hardware with iroh

Mesh LLM: distributed AI computing on iroh

I built Mesh LLM to stop teams from surrendering control and paying endless bills to big AI providers. By pooling GPUs across your own devices using iroh, we create a private mesh that runs massive models locally or splits them across machines. You get a standard OpenAI-compatible API at localhost while keeping your data secure and your costs low.

"The popular models are monoliths, and reaching them through an API key is a surrender of control over your data, hardware, and costs."