What happened
The Hugging Face repository litert-community/NVIDIA-Nemotron-3.5-Bolt-E4B converts nvidia/NVIDIA-Nemotron-3.5-Bolt-E4B-BF16 into LiteRT-LM’s .litertlm format. The conversion was made with litert-torch export_hf 0.10.0 for LiteRT-LM v0.18.0. LiteRT-LM is presented as supporting Android, iOS, Windows, Linux, macOS, IoT, and Web/WASM, with C++, Python, Kotlin, Swift, JavaScript, and Flutter APIs.
The repository supplies two artifacts. The 4.40 GB NVIDIA-Nemotron-3.5-Bolt-E4B_int4.litertlm file is intended for CPU and GPU use. It has int4 fully connected weights, int4 embedding tables, a 2,048-token KV cache, and prefill signatures for 128, 512, and 1,024 tokens. The 5.04 GB NVIDIA-Nemotron-3.5-Bolt-E4B_qualcomm_sm8850.litertlm file is compiled ahead of time for the Snapdragon SM8850 NPU, with a different quantization recipe and the same 2,048-token cache.
On a Galaxy S26 GPU, the repository reports 868 tokens per second for a forced 1,024-token prefill and 15.2 tokens per second for decoding. It reports 12.2 tokens per second decode performance for the NPU artifact. The author says the device reduced clock limits during runs, so these are sustained rather than peak results. The GPU benchmark used LiteRT-LM v0.18.0; the NPU result used a LiteRT-LM build from the main branch, Qualcomm QNN libraries from QAIRT 2.47.0, and a hardware-cache-update flag. The card explicitly says it did not test whether released LiteRT-LM builds run the NPU file.
The model remains text-only in these conversion files. Bolt E4B is described as a small language model further post-trained by NVIDIA from Gemma 4 E4B-it for tool use, instruction following, and input grounding in local and embedded assistants. Although the checkpoint carries Gemma 4 image and audio encoders, the conversion leaves them out. The card offers a direct CLI route: install litert-lm, then run the repository through litert-lm run. Thinking is off by default; enabling it can return the model’s thought separately from its answer, while prompt, thought, and answer share the 2,048-token cache.
What the archive adds
This is a new deployment event for a model already covered in two forms: the original 4 GB Bolt E4B release and its Ollama availability. It also follows Google’s Gemma 4 E4B QAT Q4_0 distribution. The new fact is LiteRT-LM packaging and phone-specific measurements, not a new Bolt model.
Trend signal
No listed pattern is a sufficiently direct fit. This is a concrete runtime conversion and benchmark report, but it does not establish a broader shift in how agent systems are designed or operated.
Why it matters
Local-model decisions are constrained by more than parameter counts. Teams need an artifact that fits a device, a runtime they can integrate, an acceptable cache size, and measured latency under realistic thermal behavior. This repository supplies those details for a specific Android-class target: a 4.40 GB GPU-oriented file, a 5.04 GB NPU-oriented file, a 2,048-token cache, and separate prefill and decode figures.
That makes it useful for prototyping bounded text tasks such as extraction, short assistant turns, or offline device interactions. It does not make Bolt E4B a general replacement for a hosted multimodal system: the supplied LiteRT artifacts omit image and audio encoders, and the NPU path has release-readiness uncertainty. Builders should treat this as an integration option to validate on their own devices and workloads.
What happens next
- A released LiteRT-LM build gains documented compatibility with the SM8850 NPU artifact. (3 months, 55%). Watch for: release notes or a repository update replacing the main-branch requirement.
- More Gemma 4-derived local models receive LiteRT-LM packages with device measurements. (6 months, 65%). Watch for: additional
.litertlmrepositories that disclose cache size, quantization, and sustained throughput.
Limits and uncertainty
The performance and accuracy figures are self-reported in the repository, not an independent benchmark. Phone tests were incomplete: the GPU GSM8K run covered 52 questions and the NPU run 99, because device time ran out. The listed measurements cover a Galaxy S26 configuration and do not establish behavior on other Android devices, memory tiers, or thermal conditions. The source also does not show end-to-end application latency, battery impact, tool-use reliability, safety behavior, or long-context quality. Most importantly for NPU adopters, released LiteRT-LM compatibility was not tested.
Skill to build now
Create a small on-device acceptance harness before choosing a local runtime. Run the supplied GPU artifact with three prompt bands that match your product, record engine-start time, time to first token, sustained decode rate, memory use, and output correctness, then repeat after a warm device run. Keep the task within the documented 2,048-token cache unless you have tested eviction behavior. For teams considering the NPU file, make released-runtime compatibility a blocking check rather than assuming the repository’s main-branch setup will translate to production.
Reader takeaway
Nemotron 3.5 Bolt E4B now has a LiteRT-LM route for local text inference with unusually concrete Galaxy S26 measurements. The GPU file is the clearer starting point; the NPU artifact is promising but depends on an unreleased LiteRT-LM build in the reported setup. Evaluate it as a device-specific deployment package, not as a generic mobile-model result.
Open question
Will a released LiteRT-LM version support the SM8850 NPU artifact with the reported performance and without the main-branch runtime dependency?