What happened
OpenBMB's October 20, 2026 announcement describes LES as the inference system behind on-device deployment of MiniCPM-V 4.7 35B-A3B. It supports iOS, Android, and HarmonyOS. The model has about 35 billion total parameters and activates about 3 billion per token. ModelBest defines real-time decoding as above 8 tokens per second and calls its approach lossless because it preserves the original network structure and inference mechanism: it does not reduce the number of experts, activated experts, or layers, fix expert selection through prompts, or turn the MoE model into a dense one.
The challenge is memory and data movement. At four bits per parameter, the weights alone would occupy about 17.5 GB, before metadata, KV cache, and runtime buffers. OpenBMB notes that an iPhone 18 Pro Max has 12 GB of unified memory shared by the system, CPU, GPU, and apps, so the complete weights cannot remain resident. LES instead keeps expert weights in flash storage and loads them as needed. That makes flash-read latency a key constraint: the model must select experts, load their weights, and compute, and a serial sequence leaves computation waiting on storage.
LES uses the current layer's router input as an early input to the next layer's router, predicting which expert weights may be needed and starting their reads while current-layer computation proceeds. The prediction schedules loading only; the real router still decides which experts are actually computed and how their outputs are combined. ModelBest reports about 80% recall for its candidate experts. Increasing the candidate set from top-k to top-2k raises recall by about 15%, but also increases read volume and buffer use. A ready-first policy lets experts whose weights have arrived begin computing while other selected weights are still being read.
The system also changes storage and computation. LES reorganizes GGUF expert weights so each expert's matrices are contiguous, enabling more efficient reads, and uses mixed precision: expert weights at Q3_K while remaining weights stay at Q4_K_M, calibrated with imatrix. ModelBest reports a 14.44 GB weight file, down from 19.7 GB at Q4_K_M, a 26.7% reduction. It reports a five-benchmark average of 78.12 against 78.28 at FP16. For compute, the team adapted DSpark speculative decoding to llama.cpp; a draft module proposes candidate tokens and the target model verifies them together.
In reported measurements, decoding exceeded 20 tokens per second on an iPhone 18 Pro Max, with prefill at 113.6–176.1 tokens per second depending on input length. An M4 iPad reached 18.0 decoding tokens per second and about 110 prefill tokens per second. On a 12 GB iPhone, the ablation starts at 3.38 tokens per second when expert weights are read on demand from flash, then reaches 6.57 with lookahead scheduling, 11.34 with scheduling and storage changes, 17.16 after mixed-precision quantization, and 20.51 with DSpark. The implementation is open-sourced in the tc-mb/llama.cpp-omni repository.
What the archive adds
New Runtime's earlier coverage described ModelBest and OpenBMB's MiniCPM-V 4.7 35B-A3B as a multimodal MoE model intended for local use on phones and tablets (October 23, 2026 note). A separate note covered MoEShift's approach to running a different MoE model locally on Apple Silicon by streaming experts from SSD (November 1, 2026 note). LES is a new technical fact on the broader local-inference story, not a restatement: it supplies a scheduling design and measured device results for MiniCPM-V 4.7.
Trend signal
This does not yet fit a pattern New Runtime tracks. LES is relevant to local model deployment, but the current pattern set does not specifically cover inference optimization, flash offloading, or on-device model performance. Its results may become useful evidence for a broader pattern if comparable systems show that model usability increasingly depends on coordinated runtimes and storage design rather than model size alone.
Why it matters
For teams building private, offline, or latency-sensitive AI features, LES offers a concrete engineering lesson: fitting weights into memory is not the only route to local inference. A system can keep rarely needed MoE experts in flash and try to hide read latency through lookahead scheduling, contiguous storage, and ready-first execution. Speculative decoding then increases the useful output from each target-model pass.
The trade-off is explicit rather than magical. On the M4 iPad, ModelBest reports that a low-memory configuration around 3 GB reaches 9.6–10.7 tokens per second, while allocating a larger cache raises memory to 7.8 GB and speed to 18 tokens per second. That gives product teams a potential tuning dimension: set a memory budget and decide how much throughput to trade for it. But these figures are vendor measurements on named devices, not a guarantee for other hardware, models, prompts, or thermal conditions. The implementation and benchmarks are worth evaluating before committing a product to a similar design.
What happens next
- Independent users will attempt to reproduce LES throughput and memory figures on supported Apple devices. (3 months, 60%). Watch for: public benchmarks or issue reports for tc-mb/llama.cpp-omni that state device, cache window, quantization, and sustained decoding speed.
Limits and uncertainty
The announcement's performance and benchmark figures are reported by ModelBest, the system's developer; the announcement does not establish independent replication. It gives results for an iPhone 18 Pro Max and an M4 iPad, but not a broad device matrix or enough detail here to predict performance across other phones, workloads, or thermal conditions. The five-benchmark average is reported, but the announcement does not name those benchmarks in the result summary. Its “lossless” label refers to preserving architecture and routing, while the quantization section reports a small measured score decrease against FP16; readers should not interpret the term as proof of identical outputs or quality in every task. The announced support for iOS, Android, and HarmonyOS does not by itself establish equivalent performance on each platform.
Skill to build now
If you are evaluating local MoE inference, reproduce the memory-throughput trade-off on one target device before selecting a configuration. Record the model and quantization, cache window, peak memory including KV cache, decoding speed, prefill speed, and sustained temperature or power. Compare on-demand loading with any available prefetching and speculative-decoding options. LES's reported c=0 and c=128 iPad figures show why a single “tokens per second” number is not enough to choose a deployment budget.
Reader takeaway
LES is a useful systems result, not a promise that every 35B model will run quickly on every phone. ModelBest's measurements suggest that offloading MoE experts to flash can work when scheduling, storage layout, quantization, caching, and decoding are optimized together. For builders, the next step is to test the full memory, speed, and thermal trade-off on the hardware they plan to ship.
Open question
How much of LES's reported speedup will hold across other MoE models and phone generations under independently reproduced workloads?