{"type":"post","id":"bonsai-2-27b-near-lossless-compression-in-a-613c4a","slug":"bonsai-2-27b-near-lossless-compression-in-a-613c4a","title":"Bonsai 2 27B: Near-Lossless Compression in a 9x Smaller Footprint","description":"Bonsai 2 27B makes a 27B-class model practically deployable on a local Mac by distributing a roughly 5.95 GB GGUF, but its runtime remains specialized.","retrieval_nugget":"Bonsai 2 27B makes a 27B-class model practically deployable on a local Mac by distributing a roughly 5.95 GB GGUF, but its runtime remains specialized.","published_at":"2026-09-25","updated_at":"2026-09-25","record_date":"2026-09-25","date_kind":"published_at","topics":[],"source_urls":["https://simonwillison.net/2026/Sep/17/hn-49747390"],"schema_version":"newruntime-agent-readable-v0.2","stable_id":"post:bonsai-2-27b-near-lossless-compression-in-a-613c4a","status":"published","visuals":[],"editorial_provenance":{"schema_version":"newruntime-editorial-copy-v1","content_status":"source_grounded_final","final_copy_sha256":"sha256:b1d7c44740f38140556e2ab8bb07b60c69489e0a1305a92daf730fcb26f3a572","reviewed_at":"2026-09-25T18:01:12.423Z","source_evidence_count":1,"verified_claim_count":4,"site_analysis_schema_version":"newruntime-site-analysis-v1","site_object_kind":"field_note","observed_fact_count":4,"implication_count":2,"watch_condition_count":2,"related_record_count":0},"analysis":{"schema_version":"newruntime-site-analysis-v1","object_kind":"field_note","thesis":"Bonsai 2 27B makes a 27B-class model practically deployable on a local Mac by distributing a roughly 5.95 GB GGUF, but its runtime remains specialized.","observed_facts":[{"text":"The public source’s installation command downloads a GGUF model file described as approximately 5.95 GB.","source_urls":["https://simonwillison.net/2026/Sep/17/hn-49747390"]},{"text":"The public source says the GGUFs need Prism’s llama.cpp fork and provides a Prism runtime download.","source_urls":["https://simonwillison.net/2026/Sep/17/hn-49747390"]},{"text":"The author reports approximately 20 tokens per second on an M5 Pro, later observing 44 after restarting the server.","source_urls":["https://simonwillison.net/2026/Sep/17/hn-49747390"]},{"text":"The startup log quoted by the source says Metal tensor API support was disabled in that environment.","source_urls":["https://simonwillison.net/2026/Sep/17/hn-49747390"]}],"mechanism":"The mechanism is packaging compression and runtime support together: the smaller model artifact lowers storage and local-serving requirements, while a modified inference stack interprets its format.","why_now":"This matters now because the release is accompanied by a runnable local path instead of only an abstract compression claim.","implications":["The implication is that teams can test 27B-scale capability under local-data and local-cost constraints, but must treat runtime compatibility as part of the deployment decision.","Specialized runtime dependencies may constrain portability, reproducibility, and operational support until broader tooling adopts the format."],"evidence_boundary":"Evidence boundary: the source is a single practitioner report, not an independent quality evaluation, a controlled throughput benchmark, or proof of the release title’s near-lossless and 9x claims.","watch_conditions":["Watch whether Prism support reaches mainstream runtimes and whether independent evaluations reproduce quality, throughput, and stability across hardware.","Watch for controlled comparisons against conventional quantized 27B models on the same prompts and hardware."],"related_records":[],"new_branch_reason":"This starts a distinct analytical record because no prior New Runtime record was supplied for this model-and-runtime release."},"editor":{"role":"editor","name":"Andrey Reshetnikov","profile":"https://newruntime.com/owner-profile.md","profile_json":"https://newruntime.com/owner-profile.json","same_as":["https://www.linkedin.com/in/reshetnikov1/","https://t.me/lapetuse"]},"routes":{"html":"https://newruntime.com/posts/bonsai-2-27b-near-lossless-compression-in-a-613c4a/","markdown":"https://newruntime.com/posts/bonsai-2-27b-near-lossless-compression-in-a-613c4a.md","json":"https://newruntime.com/posts/bonsai-2-27b-near-lossless-compression-in-a-613c4a.json"},"next_reads":[{"type":"collection","path":"/posts/","url":"https://newruntime.com/posts/","title":"Field Notes","reason":"Open the latest editorial analysis.","media_type":"text/html","json_url":"https://newruntime.com/posts.json"},{"type":"collection","path":"/source-ledger.json","url":"https://newruntime.com/source-ledger.json","title":"Source Ledger","reason":"Inspect the public source evidence graph.","media_type":"application/json"}]}