Apple's MLX Runs a Trillion-Parameter LLM on Four Mac Studios at 36 Tokens per Second
Apple ML Research: Muse Spark Open with MLX on Mac Studio
Apple's MLX framework now runs frontier-class models locally on Mac Studio. A cluster of four M5 Ultra Mac Studios generates over 36 tokens per second with the 816B-parameter Muse Spark Open model, using RDMA over Thunderbolt 5 and MLX's new jaccl communication collectives. The M5 Ultra's GPU Neural Accelerators deliver up to 4.6× faster time to first token and 1.6× faster decode than M3 Ultra, all within about 300W.
On a cluster of four Mac Studios, Muse Spark Open generates over 36 tokens per second, well above average human reading speed.