Recursive's 125 tokens/s claim for its 40B-total, 2.5B-active coding model is plausible on paper. The figure depends on one runtime at short context, and no one has replicated it. How Flint behaves on a laptop at the 256k context Recursive recommends is still unknown, and the only long-context number available points in Flint's favour.
Summary
Recursive has released Flint under Apache 2.0. It is a mixture-of-experts coding model with about 40B total parameters, roughly 2.5B of them active per token. Alongside it, Recursive shipped rec, an inference engine tuned for M4 Pro, M5 Pro and M5 Max Macs. According to Recursive, Flint runs at more than 125 tokens/s on an M4 Pro MacBook Pro and 150 tokens/s on a 64GB RTX Spark at short contexts. The company also says Flint leads sub-50B MoE models on SWE-Bench Pro v2, Terminal-Bench 2.1 and SWE-Bench Verified. All of these figures come from Recursive, and no one has replicated them independently.
What happened
According to Recursive's announcement, Flint supports up to 512k tokens of context, and the company recommends staying within 256k. Recursive lists compatibility with llama.cpp, Ollama, Unsloth, LM Studio, vLLM and SGLang. It also released rec, its own engine for recent Apple Silicon Pro and Max chips.
The Hugging Face repository (RecursiveSI/flint) is ungated. It lists 39,785,287,456 BF16 parameters and about 79.6GB of safetensors weights. No hosted inference provider is listed, and there is no managed API or price. The repository points to community quantizations for llama.cpp-style runtimes. It does not state the 2.5B active figure, hardware requirements, training data or benchmark results; those come from the announcement.
Recursive gives its speed figures only for short contexts. It publishes nothing on throughput at long context, on comparisons with other models under matched settings, or on independent benchmark runs.
vLLM's day-0 support post is the one outside data point. It reports roughly 500 tokens/s on a single GB200, including at 100K context, and credits custom small-batch MoE and decode kernels written for Flint.
What the archive adds
The archive adds three useful pieces of context.
- An August 2026 note on local inference (d1) argued that "fit does not equal usable." Getting a model into memory through quantization is only the first step. Whether it runs fast enough for real work is the actual test. The note drew on cases such as Unsloth's small-footprint claims and DeepSeek V4 Flash. Flint's launch materials answer the first question (it fits on 64GB machines once quantized) and give only a partial answer to the second.
- vLLM's day-0 post for Flint (d3) reports about 500 tokens/s on one GB200, including at 100K context. It attributes the result to kernels written for small-batch MoE decoding. This is a datacenter part, not a laptop. Still, it is the only long-context figure anywhere in the evidence, and it shows Flint's architecture does not fall apart at 100K, at least on hardware with plenty of compute.
- An archived runtime entry, Uzu (d2), is a local inference runtime built for Apple Silicon. Together with Recursive's rec, it suggests vendors and third parties increasingly treat the Mac runtime as a product in its own right, not a by-product of a generic one. That matters for reading speed claims: the number belongs to the runtime as much as to the model.
On the training side, archive coverage of Ai2's Olmo-core 3 describes large expert pools with only four experts active per token. That is the same sparse design Flint relies on to keep the cost of each token low.
Trend signal
Sparse MoE models split the two resources a local machine has to supply. Total parameters decide how much memory the model needs. Active parameters decide how many bytes are read for each generated token. Flint pushes that split to about 16:1, total to active.
As a result, decode speed on a laptop now depends heavily on the runtime. A generic engine and a tuned one can read the same 2.5B active weights at very different efficiency. That is why Recursive shipped rec and why vLLM wrote Flint-specific kernels on day zero.
For anyone comparing models, a tokens/s figure is now a property of the combination of model, runtime, quantization, context length and hardware. It is not a property of the model alone. The material Recursive published fixes the model and the hardware, but it fixes neither the quantization nor the context length beyond "short."
Why now
1) Sparse MoE design. With about 2.5B active out of 40B total, according to Recursive, decode reads only a small slice of memory per token. That makes laptop-class decode speeds plausible once the weights are quantized.
2) Tuned runtimes are now part of the release. Recursive shipped rec for Apple Silicon. vLLM shipped Flint-specific small-batch MoE kernels on day zero and reported about 500 tokens/s at up to 100K context on a GB200 (d3). Speed has become a runtime result as much as a model result.
3) Quantization tooling has matured. The archive note on local inference (d1) records that the open question for local models has moved from "does it fit" to "is it usable." Flint's 64GB-class hardware targets assume quantized weights.
4) Local hardware targets. Recursive names M4/M5 Pro and Max Macs and a 64GB RTX Spark, all machines with enough memory to hold a quantized 40B MoE.
Connections to other events
Here are the numbers as far as public data allows. Everything below is our arithmetic, not anything Recursive has published.
Memory: about 79.6GB of BF16 weights, according to the repository, does not fit on a 64GB machine. At around 8 bits per weight, 39.8B parameters take about 40GB. At around 4 bits they take about 20GB. KV cache and runtime overhead come on top of both. We do not know which format Recursive used for its speed figures.
Decode ceiling: Apple's published spec puts M4 Pro memory bandwidth at about 273 GB/s; this is an outside figure, not from the evidence. Reading 2.5B active weights takes about 1.25GB per token at 4 bits and about 2.5GB at 8 bits. That gives rough ceilings near 220 tokens/s and 110 tokens/s respectively. The real ceiling is lower than either, because attention layers, shared or router weights and KV-cache reads add bytes to every token beyond the expert weights. A claim of more than 125 tokens/s therefore fits a low-bit format with a well-tuned engine. It is not something you can assume for any runtime you happen to use.
Prefill: a forward pass costs roughly 2 FLOPs per active parameter per token, so about 5 GFLOPs per token. Ingesting 100k tokens is therefore at least about 5×10^14 FLOPs before attention, and attention cost grows with context length. We do not have a reliable sustained-compute figure for the M4 Pro GPU under this workload, so we cannot turn that into seconds. That is exactly the measurement Recursive has not published.
Hypothesis, clearly labelled: on a laptop, time-to-first-token and KV-cache memory at 100k+ context may limit an agent loop before decode speed does. The counter-signal is vLLM's ~500 tokens/s at 100K on GB200 (d3), which shows Flint's long-context behaviour can be good when compute is plentiful. Whether that carries over to a Mac is open.
The benchmark leads are limited to "sub-50B MoE." That category is narrow and says nothing about dense models with similar memory cost or about hosted agents. Recursive also does not disclose the agent harness and settings behind each score in the material we have.
Why it matters
Flint is worth trying as a local backend for coding agents. The Apache 2.0 licence, ungated weights and support in vLLM, SGLang and llama.cpp-family tools remove most of the integration work.
Treat every published tokens/s figure as specific to one runtime. A number measured with rec on an M4 Pro tells you little about Ollama on the same machine, and a GB200 figure tells you nothing about a laptop.
Size hardware for the context your agent actually uses. A 64GB machine very likely holds low-bit weights plus a moderate context. Whether it stays responsive near 256k has not been shown.
If you are comparing Flint with hosted coding assistants, the numbers that matter are wall-clock time and pass rate per task on your own repository, not decode speed or category-limited leaderboard leads.
What happens next
- A third party, not Recursive, will publish a Flint score on Terminal-Bench 2.1 or SWE-Bench Pro v2 with the harness and settings disclosed, and it will come in at least 3 points below Recursive's reported figure. (6 months, 50%). Watch for: A leaderboard entry or reproducible eval repository naming the agent harness, quantization and context settings used for Flint.
- At least one hosted inference provider will list Flint with per-token pricing, despite its 'local' positioning. (3 months, 60%). Watch for: Flint appearing under Inference Providers on its Hugging Face page or in a provider's model catalogue.
- Recursive or the community will publish Apple Silicon throughput or time-to-first-token figures for Flint at 100k+ context. (6 months, 65%). Watch for: A long-context table in rec release notes, or widely referenced llama.cpp/Ollama benchmark threads at 128k context on M4/M5 Macs.
Limits and uncertainty
The memory and bandwidth figures are rough estimates. They rely on the repository's parameter count and an outside Apple bandwidth spec. Recursive has not published its quantization format, attention design or KV-cache size per token. If Flint uses a memory-efficient attention scheme, long-context costs could be far lower than a naive estimate. vLLM's 100K result hints at this.
The rec engine may include prefill optimizations we cannot see. We could not estimate prefill time on a Mac without a trustworthy sustained-compute figure.
We have no matched-settings comparisons with other models or other runtimes on the same hardware, and no disclosed benchmark harness details. That makes it impossible to say how wide the spread between runtimes is.
The benchmark claims may well hold up under replication. Our point is only that they have not been tested yet.
Skill to build now
Learn to read speed claims for local MoE models by regime and runtime rather than by headline.
First exercise: serve Flint, or any MoE model you already run, in two runtimes, for example llama.cpp and vLLM or rec, at one fixed quantization. Send prompts of 8k, 64k and 160k tokens built from your own repository. Log time-to-first-token, decode tokens/s and peak memory in one table. Then calculate a decode ceiling: your hardware's memory bandwidth divided by the active-weight bytes per token. Note that the result is an upper bound, because attention and KV reads add bytes. Comparing that ceiling with each runtime's measured speed shows how much of the gap comes from the runtime.
Reader takeaway
Before you pick Flint as your agent backend, measure it yourself in the runtime you will actually use. Do it at three context sizes built from real repository content: 8k, 64k and 160k tokens. At each size, record time-to-first-token, decode tokens/s and peak memory. Then run five real backlog tasks end to end. If time-to-first-token at 64k is too slow for your agent loop, use Flint for short-context edits and keep a hosted model for work that spans the whole repository.
Open question
Does vLLM's roughly 500 tokens/s at 100K context on a GB200 reflect an attention design that will also stay responsive on a compute-limited laptop? Or does the far smaller compute budget of a Mac make prefill, rather than decode, the step that decides whether Flint works for repository-wide agent tasks?