Field note
Three experiments show the local-inference boundary moving across different hardware tiers. Unsloth says a forthcoming Qwen3.8-27B configuration will fit in 17 GB. A community Gemma run reaches an iPhone 13 Pro with severe speed and memory limits. DeepSeek V4 Flash can be quantized and served locally, but its 284-billion-parameter mixture-of-experts weights still make memory and loader support the hard constraints.
These are not comparable performance records. A model fitting in memory does not imply interactive speed, useful context, thermal stability, or acceptable battery life. Sparse activation reduces compute per token but does not remove the need to store the full weight set. Experimental forks and aggressive quantization can also change reliability.
Every local claim should carry a configuration contract: exact checkpoint, quantization, backend and commit, RAM and VRAM, context size, prompt and decode speed, power, heat, and quality delta. The trend is real—more capability is available without a hosted API—but the operational threshold is a matrix, not one headline number.