Mozilla.ai benchmarked four local LLM servers, llama.cpp, llamafile, LM Studio, and Ollama, on a Mac Studio M4 Max using Metal, a Linux server with an NVIDIA L40S, and a Steam Deck using Vulkan. The tests used Qwen models at 0.8B, 9B, and 27B, the same GGUF files across runtimes, and prompt sizes from 1,024 to 8,192 tokens.
The central finding is that build and configuration choices moved local inference throughput more than the choice of server. The mechanism is shared code: all four serve GGUF through llama.cpp, so prompt processing lands within a few percent once weights and environment are fixed, while the real differences come from compile flags, shader toolchains, and host-side per-token overhead. Enabling CUDA graphs in llamafile raised L40S decode by 16.8% on the 0.8B model, and a newer Vulkan shader toolchain raised Steam Deck prompt processing by up to 63.3% on the 9B model.
This matters now because agents and large codebases push local servers toward long prompts, where prompt-processing throughput matters most. One implication is that speculative-decoding draft length should be tuned per platform, since draft 2 won on Metal while draft 4 beat draft 2 on the L40S by up to 16%. The study does not establish a general ranking of the servers, and its authors describe it as a snapshot of specific versions, models, and hardware.
Worth watching is whether LM Studio changes its prebuilt runtime, since the authors attribute its deficit to that runtime rather than to its engine.