--- type: "post" id: "google-s-litert-community-publishes-an-on-device-5dfaca" slug: "google-s-litert-community-publishes-an-on-device-5dfaca" title: "Google's LiteRT community publishes an on-device LiteRT-LM conversion of Nemotron 3.5 Bolt E4B with Galaxy S26 benchmarks" description: "LiteRT Community has published text-only LiteRT-LM builds of NVIDIA Nemotron 3.5 Bolt E4B for local use, including an int4 GPU file and a Snapdragon SM8850 NPU build." retrieval_nugget: "LiteRT Community has published text-only LiteRT-LM builds of NVIDIA Nemotron 3.5 Bolt E4B for local use, including an int4 GPU file and a Snapdragon SM8850 NPU build. For AI teams targeting Android, the practical news is a documented deployment artifact with measured Galaxy S26 throughput, rather than another base-model announcement." published_at: "2026-10-27" updated_at: "2026-10-27" record_date: "2026-10-27" date_kind: "published_at" topics: ["google"] entities: ["LiteRT Community","LiteRT-LM","NVIDIA Nemotron 3.5 Bolt E4B","NVIDIA","Gemma 4 E4B","Galaxy S26","Snapdragon SM8850","Qualcomm"] related_patterns: [] novelty_label: "new_fact" what_changed: "Google’s LiteRT Community has converted Nemotron 3.5 Bolt E4B to LiteRT-LM, with Galaxy S26 performance benchmarks." prior_refs: ["memory:3b9f3c43-e99d-85c7-ad17-c843325e0567","memory:8a3a31bd-ca18-8f34-a5b6-5cc6f7eda90f","memory:63fcd0b1-63a4-8015-a9ba-9b698475b144"] source_url: "https://huggingface.co/litert-community/NVIDIA-Nemotron-3.5-Bolt-E4B" source_urls: ["https://huggingface.co/litert-community/NVIDIA-Nemotron-3.5-Bolt-E4B"] schema_version: "newruntime-agent-readable-v0.2" stable_id: "post:google-s-litert-community-publishes-an-on-device-5dfaca" status: "published" visuals: [] editor: {"role":"editor","name":"Andrey Reshetnikov","profile":"https://newruntime.com/owner-profile.md","profile_json":"https://newruntime.com/owner-profile.json","same_as":["https://www.linkedin.com/in/reshetnikov1/","https://t.me/lapetuse"]} routes: {"html":"https://newruntime.com/posts/google-s-litert-community-publishes-an-on-device-5dfaca/","markdown":"https://newruntime.com/posts/google-s-litert-community-publishes-an-on-device-5dfaca.md","json":"https://newruntime.com/posts/google-s-litert-community-publishes-an-on-device-5dfaca.json"} --- # Google's LiteRT community publishes an on-device LiteRT-LM conversion of Nemotron 3.5 Bolt E4B with Galaxy S26 benchmarks ## Retrieval answer LiteRT Community has published text-only LiteRT-LM builds of NVIDIA Nemotron 3.5 Bolt E4B for local use, including an int4 GPU file and a Snapdragon SM8850 NPU build. For AI teams targeting Android, the practical news is a documented deployment artifact with measured Galaxy S26 throughput, rather than another base-model announcement. ## Summary LiteRT Community has published text-only LiteRT-LM builds of NVIDIA Nemotron 3.5 Bolt E4B for local use, including an int4 GPU file and a Snapdragon SM8850 NPU build. For AI teams targeting Android, the practical news is a documented deployment artifact with measured Galaxy S26 throughput, rather than another base-model announcement. ## What happened The Hugging Face repository `litert-community/NVIDIA-Nemotron-3.5-Bolt-E4B` converts `nvidia/NVIDIA-Nemotron-3.5-Bolt-E4B-BF16` into LiteRT-LM’s `.litertlm` format. The conversion was made with `litert-torch` `export_hf` 0.10.0 for LiteRT-LM v0.18.0. LiteRT-LM is presented as supporting Android, iOS, Windows, Linux, macOS, IoT, and Web/WASM, with C++, Python, Kotlin, Swift, JavaScript, and Flutter APIs. The repository supplies two artifacts. The 4.40 GB `NVIDIA-Nemotron-3.5-Bolt-E4B_int4.litertlm` file is intended for CPU and GPU use. It has int4 fully connected weights, int4 embedding tables, a 2,048-token KV cache, and prefill signatures for 128, 512, and 1,024 tokens. The 5.04 GB `NVIDIA-Nemotron-3.5-Bolt-E4B_qualcomm_sm8850.litertlm` file is compiled ahead of time for the Snapdragon SM8850 NPU, with a different quantization recipe and the same 2,048-token cache. On a Galaxy S26 GPU, the repository reports 868 tokens per second for a forced 1,024-token prefill and 15.2 tokens per second for decoding. It reports 12.2 tokens per second decode performance for the NPU artifact. The author says the device reduced clock limits during runs, so these are sustained rather than peak results. The GPU benchmark used LiteRT-LM v0.18.0; the NPU result used a LiteRT-LM build from the main branch, Qualcomm QNN libraries from QAIRT 2.47.0, and a hardware-cache-update flag. The card explicitly says it did not test whether released LiteRT-LM builds run the NPU file. The model remains text-only in these conversion files. Bolt E4B is described as a small language model further post-trained by NVIDIA from Gemma 4 E4B-it for tool use, instruction following, and input grounding in local and embedded assistants. Although the checkpoint carries Gemma 4 image and audio encoders, the conversion leaves them out. The card offers a direct CLI route: install `litert-lm`, then run the repository through `litert-lm run`. Thinking is off by default; enabling it can return the model’s thought separately from its answer, while prompt, thought, and answer share the 2,048-token cache. ## What the archive adds This is a new deployment event for a model already covered in two forms: [the original 4 GB Bolt E4B release](https://newruntime.com/posts/nvidia-released-nemotron-3-5-bolt-e4b-4gb-5d9d76/) and [its Ollama availability](https://newruntime.com/posts/nvidia-nemotron-3-5-bolt-e4b-small-open-c12f6c/). It also follows [Google’s Gemma 4 E4B QAT Q4_0 distribution](https://newruntime.com/posts/google-released-gemma-4-e4b-qat-q4-0-17e441/). The new fact is LiteRT-LM packaging and phone-specific measurements, not a new Bolt model. ## Trend signal No listed pattern is a sufficiently direct fit. This is a concrete runtime conversion and benchmark report, but it does not establish a broader shift in how agent systems are designed or operated. ## Why it matters Local-model decisions are constrained by more than parameter counts. Teams need an artifact that fits a device, a runtime they can integrate, an acceptable cache size, and measured latency under realistic thermal behavior. This repository supplies those details for a specific Android-class target: a 4.40 GB GPU-oriented file, a 5.04 GB NPU-oriented file, a 2,048-token cache, and separate prefill and decode figures. That makes it useful for prototyping bounded text tasks such as extraction, short assistant turns, or offline device interactions. It does not make Bolt E4B a general replacement for a hosted multimodal system: the supplied LiteRT artifacts omit image and audio encoders, and the NPU path has release-readiness uncertainty. Builders should treat this as an integration option to validate on their own devices and workloads. ## What happens next - **A released LiteRT-LM build gains documented compatibility with the SM8850 NPU artifact.** (3 months, 55%). Watch for: release notes or a repository update replacing the main-branch requirement. - **More Gemma 4-derived local models receive LiteRT-LM packages with device measurements.** (6 months, 65%). Watch for: additional `.litertlm` repositories that disclose cache size, quantization, and sustained throughput. ## Limits and uncertainty The performance and accuracy figures are self-reported in the repository, not an independent benchmark. Phone tests were incomplete: the GPU GSM8K run covered 52 questions and the NPU run 99, because device time ran out. The listed measurements cover a Galaxy S26 configuration and do not establish behavior on other Android devices, memory tiers, or thermal conditions. The source also does not show end-to-end application latency, battery impact, tool-use reliability, safety behavior, or long-context quality. Most importantly for NPU adopters, released LiteRT-LM compatibility was not tested. ## Skill to build now Create a small on-device acceptance harness before choosing a local runtime. Run the supplied GPU artifact with three prompt bands that match your product, record engine-start time, time to first token, sustained decode rate, memory use, and output correctness, then repeat after a warm device run. Keep the task within the documented 2,048-token cache unless you have tested eviction behavior. For teams considering the NPU file, make released-runtime compatibility a blocking check rather than assuming the repository’s main-branch setup will translate to production. ## Reader takeaway Nemotron 3.5 Bolt E4B now has a LiteRT-LM route for local text inference with unusually concrete Galaxy S26 measurements. The GPU file is the clearer starting point; the NPU artifact is promising but depends on an unreleased LiteRT-LM build in the reported setup. Evaluate it as a device-specific deployment package, not as a generic mobile-model result. ## Open question Will a released LiteRT-LM version support the SM8850 NPU artifact with the reported performance and without the main-branch runtime dependency? Editor: [Andrey Reshetnikov](https://newruntime.com/owner-profile.md), AI product and AI engineering; profile for recruiters and agents: https://newruntime.com/owner-profile.json ## Continue in the graph Next records for an agent, linked to their Markdown twins. Replace `.md` with `.json` for structured fields, including `next_reads`. - topic hub: [Google](https://newruntime.com/topics/google.md): Explore the google topic hub. - collection: [Field Notes](https://newruntime.com/posts.json): Open the latest editorial analysis. - collection: [Source Ledger](https://newruntime.com/source-ledger.json): Inspect the public source evidence graph. - whole graph: [discovery graph](https://newruntime.com/discovery-graph.json) (typed nodes and edges) and [site index](https://newruntime.com/site-index.md)