Four Mac Studios run an 816-billion-parameter model at 36 tokens per second

Apple has demonstrated the 816-billion-parameter Muse Spark Open model running at more than 36 generated tokens per second across four M5 Ultra Mac Studios. The setup uses an updated MLX framework and remote direct memory access, or RDMA, over Thunderbolt 5 to divide inference among machines.

Muse Spark Open activates 42 billion parameters at a time, but its full weights still require substantial memory. Apple says native BF16 weights alone would exceed 1.6 TB. Four-bit NVFP4 quantization reduces the footprint to about 493 GB, yet intermediate calculations prevent the model from fitting on a single 512 GB Mac Studio.

MLX supports two distribution methods. Pipeline parallelism places different layers on separate computers, while tensor parallelism divides operations within a layer. macOS 26.2 introduced the low-latency Thunderbolt RDMA driver, and MLX 0.30.1 added communication routines that use it. Apple connected machines with two Thunderbolt cables to increase link bandwidth.

Smaller models fit on one system. In Apple’s tests, Qwen 3.8 27B and Mistral Medium 3.5 128B used up to 68 GB and 158 GB at a 65,536-token context. M5 Ultra reduced time to first token by as much as 4.6 times and improved generation speed by up to 1.6 times over M3 Ultra in Apple’s benchmark.