Thesis
Compact model releases make local agent deployment a memory-and-power allocation decision, but published capability retention varies by task.
Observed facts
PrismML announced Bonsai 2 models at 2B, 4B, and 9B parameters with reported footprints of 0.43GB, 0.95GB, and 2GB.
1PrismML reports the 9B model retains 85.7% of Qwen3.5 9B's Tau2Bench score and 83.2% of its Omni_doc score.
1PrismML lists NVIDIA GPU support through llama.cpp and Apple-device support through MLX, with weights under Apache 2.0.
1
Mechanism
Ternary weights restricted to −1, 0, and +1 reduce model storage, allowing smaller deployments within device memory budgets. Smaller storage can enable on-device inference, but the benchmark results show that compression does not preserve every capability equally.
Why now
PrismML has released downloadable weights under Apache 2.0 and describes support for common NVIDIA and Apple deployment paths, making the tradeoff actionable for developers evaluating local assistants.
Implications
Teams can test whether the memory and latency benefits justify task-specific quality loss before moving sensitive or intermittent workloads on-device.
Local execution may reduce dependence on network access, but the announcement alone does not establish privacy, total energy use, or production reliability.
Evidence boundary
All performance figures and comparisons here are PrismML-reported; the public announcement does not provide independent validation, and results may vary by hardware, workload, and evaluation setup.
Watch conditions
Independent replication across devices and representative tool-use workflows would show whether the claimed footprint and speed translate into reliable local agents.