The note says the standard runtime is insufficient for these files, and reports approximately 20 tokens per second on an M5 Pro, later observing 44 after a server restart. It also records a Metal tensor-API warning, so the performance figure is observational rather than a benchmark result.
Raw SignalSep 25, 20261 min read1 source
Bonsai 2 27B: Near-Lossless Compression in a 9x Smaller Footprint
In one minute
Simon Willison published a local execution recipe for Bonsai 2 27B using a roughly 5.95 GB GGUF model file and Prism’s llama.cpp fork.