Four Mac Studios run an 816-billion-parameter model at 36 tokens per second
Apple’s updated MLX distributes a quantized mixture-of-experts model over Thunderbolt 5 using RDMA. The setup enables private desktop inference, but the largest model still needs several high-memory machines.