Choosing between llama.cpp, llamafile, LM Studio and Ollama may matter less than tuning the software for the machine underneath it. Mozilla benchmarked the four local model servers with identical model files and found that configuration produced the largest performance differences.
The tests covered a Mac Studio M4, an Nvidia L40S Linux server and a Steam Deck, using three Qwen model sizes. All four tools ultimately rely on llama.cpp to serve GGUF model files, helping explain why prompt-processing performance stayed within a few percentage points when the weights and environment were held constant.
Specific settings made a much bigger difference. Enabling CUDA graphs raised llamafile’s decoding speed by 16.8% for the smallest model on the L40S. Updating the Vulkan shader toolchain improved prompt processing by up to 63% for the 9-billion-parameter model on Steam Deck. The best speculative-decoding setting also differed between Apple Metal and Nvidia CUDA.
Mozilla describes the results as a practical snapshot rather than a final ranking. The useful takeaway for people running models locally is to test build flags, acceleration backends and decoding settings on their own hardware before switching servers.