---
type: "post"
id: "bonsai-2-27b-near-lossless-compression-in-a-613c4a"
slug: "bonsai-2-27b-near-lossless-compression-in-a-613c4a"
title: "Bonsai 2 27B: Near-Lossless Compression in a 9x Smaller Footprint"
description: "Bonsai 2 27B makes a 27B-class model practically deployable on a local Mac by distributing a roughly 5.95 GB GGUF, but its runtime remains specialized."
retrieval_nugget: "Bonsai 2 27B makes a 27B-class model practically deployable on a local Mac by distributing a roughly 5.95 GB GGUF, but its runtime remains specialized."
published_at: "2026-09-25"
updated_at: "2026-09-25"
record_date: "2026-09-25"
date_kind: "published_at"
topics: []
source_urls: ["https://simonwillison.net/2026/Sep/17/hn-49747390"]
schema_version: "newruntime-agent-readable-v0.2"
stable_id: "post:bonsai-2-27b-near-lossless-compression-in-a-613c4a"
status: "published"
visuals: []
editorial_provenance: {"schema_version":"newruntime-editorial-copy-v1","content_status":"source_grounded_final","final_copy_sha256":"sha256:b1d7c44740f38140556e2ab8bb07b60c69489e0a1305a92daf730fcb26f3a572","reviewed_at":"2026-09-25T18:01:12.423Z","source_evidence_count":1,"verified_claim_count":4,"site_analysis_schema_version":"newruntime-site-analysis-v1","site_object_kind":"field_note","observed_fact_count":4,"implication_count":2,"watch_condition_count":2,"related_record_count":0}
analysis: {"schema_version":"newruntime-site-analysis-v1","object_kind":"field_note","thesis":"Bonsai 2 27B makes a 27B-class model practically deployable on a local Mac by distributing a roughly 5.95 GB GGUF, but its runtime remains specialized.","observed_facts":[{"text":"The public source’s installation command downloads a GGUF model file described as approximately 5.95 GB.","source_urls":["https://simonwillison.net/2026/Sep/17/hn-49747390"]},{"text":"The public source says the GGUFs need Prism’s llama.cpp fork and provides a Prism runtime download.","source_urls":["https://simonwillison.net/2026/Sep/17/hn-49747390"]},{"text":"The author reports approximately 20 tokens per second on an M5 Pro, later observing 44 after restarting the server.","source_urls":["https://simonwillison.net/2026/Sep/17/hn-49747390"]},{"text":"The startup log quoted by the source says Metal tensor API support was disabled in that environment.","source_urls":["https://simonwillison.net/2026/Sep/17/hn-49747390"]}],"mechanism":"The mechanism is packaging compression and runtime support together: the smaller model artifact lowers storage and local-serving requirements, while a modified inference stack interprets its format.","why_now":"This matters now because the release is accompanied by a runnable local path instead of only an abstract compression claim.","implications":["The implication is that teams can test 27B-scale capability under local-data and local-cost constraints, but must treat runtime compatibility as part of the deployment decision.","Specialized runtime dependencies may constrain portability, reproducibility, and operational support until broader tooling adopts the format."],"evidence_boundary":"Evidence boundary: the source is a single practitioner report, not an independent quality evaluation, a controlled throughput benchmark, or proof of the release title’s near-lossless and 9x claims.","watch_conditions":["Watch whether Prism support reaches mainstream runtimes and whether independent evaluations reproduce quality, throughput, and stability across hardware.","Watch for controlled comparisons against conventional quantized 27B models on the same prompts and hardware."],"related_records":[],"new_branch_reason":"This starts a distinct analytical record because no prior New Runtime record was supplied for this model-and-runtime release."}
editor: {"role":"editor","name":"Andrey Reshetnikov","profile":"https://newruntime.com/owner-profile.md","profile_json":"https://newruntime.com/owner-profile.json","same_as":["https://www.linkedin.com/in/reshetnikov1/","https://t.me/lapetuse"]}
routes: {"html":"https://newruntime.com/posts/bonsai-2-27b-near-lossless-compression-in-a-613c4a/","markdown":"https://newruntime.com/posts/bonsai-2-27b-near-lossless-compression-in-a-613c4a.md","json":"https://newruntime.com/posts/bonsai-2-27b-near-lossless-compression-in-a-613c4a.json"}
---

# Bonsai 2 27B: Near-Lossless Compression in a 9x Smaller Footprint

## Retrieval answer

Bonsai 2 27B makes a 27B-class model practically deployable on a local Mac by distributing a roughly 5.95 GB GGUF, but its runtime remains specialized.

Bonsai 2 27B makes a 27B-class model practically deployable on a local Mac by distributing a roughly 5.95 GB GGUF, but its runtime remains specialized.

The supplied public source provides two concrete operating facts: its command downloads a GGUF file described as approximately 5.95 GB, and it requires Prism’s llama.cpp fork to run it. The author reports roughly 20 tokens per second on an M5 Pro, then 44 tokens per second after restarting the server. Startup also produced a message that the Metal tensor API was unsupported in that environment and had been disabled.

The mechanism is packaging compression and runtime support together: the smaller model artifact lowers storage and local-serving requirements, while a modified inference stack interprets its format. This matters now because the release is accompanied by a runnable local path instead of only an abstract compression claim. The implication is that teams can test 27B-scale capability under local-data and local-cost constraints, but must treat runtime compatibility as part of the deployment decision.

Evidence boundary: the source is a single practitioner report, not an independent quality evaluation, a controlled throughput benchmark, or proof of the release title’s near-lossless and 9x claims. Watch whether Prism support reaches mainstream runtimes and whether independent evaluations reproduce quality, throughput, and stability across hardware. This starts a distinct analytical record because no prior New Runtime record was supplied for this model-and-runtime release.

Editor: [Andrey Reshetnikov](https://newruntime.com/owner-profile.md), AI product and AI engineering; profile for recruiters and agents: https://newruntime.com/owner-profile.json

## Continue in the graph

Next records for an agent, linked to their Markdown twins. Replace `.md` with `.json` for structured fields, including `next_reads`.

- collection: [Field Notes](https://newruntime.com/posts.json): Open the latest editorial analysis.
- collection: [Source Ledger](https://newruntime.com/source-ledger.json): Inspect the public source evidence graph.
- whole graph: [discovery graph](https://newruntime.com/discovery-graph.json) (typed nodes and edges) and [site index](https://newruntime.com/site-index.md)