---
type: "post"
id: "model-add-flint-recursivesi-flint-bcba9d"
slug: "model-add-flint-recursivesi-flint-bcba9d"
title: "[Model] Add Flint (RecursiveSI/flint)"
description: "vLLM merged support for Recursive’s Flint model, including BF16 and NVFP4 GGUF checkpoints and kernels tuned for small-batch inference on a single Blackwell GPU."
retrieval_nugget: "vLLM merged support for Recursive’s Flint model, including BF16 and NVFP4 GGUF checkpoints and kernels tuned for small-batch inference on a single Blackwell GPU. The pull request reports a 2.7× improvement in one NVFP4 decode benchmark against stock vLLM layers, but the result is specific to a GB200 and does not generalize to every serving setup."
published_at: "2026-11-04"
updated_at: "2026-11-04"
record_date: "2026-11-04"
date_kind: "published_at"
topics: ["inference"]
entities: ["vLLM","Recursive","Flint","NVIDIA GB200"]
related_patterns: []
novelty_label: "new_fact"
what_changed: "Adds implementation details for vLLM’s Flint support: both BF16 and NVFP4 GGUF checkpoints, Flint-specific kernels, and architecture and numerical behavior."
prior_refs: []
source_url: "https://github.com/vllm-project/vllm/pull/62938"
source_urls: ["https://github.com/vllm-project/vllm/pull/62938"]
schema_version: "newruntime-agent-readable-v0.2"
stable_id: "post:model-add-flint-recursivesi-flint-bcba9d"
status: "published"
visuals: []
editor: {"role":"editor","name":"Andrey Reshetnikov","profile":"https://newruntime.com/owner-profile.md","profile_json":"https://newruntime.com/owner-profile.json","same_as":["https://www.linkedin.com/in/reshetnikov1/","https://t.me/lapetuse"]}
routes: {"html":"https://newruntime.com/posts/model-add-flint-recursivesi-flint-bcba9d/","markdown":"https://newruntime.com/posts/model-add-flint-recursivesi-flint-bcba9d.md","json":"https://newruntime.com/posts/model-add-flint-recursivesi-flint-bcba9d.json"}
markdown_url: "https://newruntime.com/posts/model-add-flint-recursivesi-flint-bcba9d.md"
---

# [Model] Add Flint (RecursiveSI/flint)

## Retrieval answer

vLLM merged support for Recursive’s Flint model, including BF16 and NVFP4 GGUF checkpoints and kernels tuned for small-batch inference on a single Blackwell GPU. The pull request reports a 2.7× improvement in one NVFP4 decode benchmark against stock vLLM layers, but the result is specific to a GB200 and does not generalize to every serving setup.

## Summary
vLLM merged support for Recursive’s Flint model, including BF16 and NVFP4 GGUF checkpoints and kernels tuned for small-batch inference on a single Blackwell GPU. The pull request reports a 2.7× improvement in one NVFP4 decode benchmark against stock vLLM layers, but the result is specific to a GB200 and does not generalize to every serving setup.

## What happened
vLLM pull request #62938, merged on October 22, 2026, adds Recursive’s Flint to the serving engine. Flint is described in the pull request as a mixture-of-experts model with 40 billion total parameters and 2.5 billion active parameters. The implementation supports two released checkpoints: `RecursiveSI/flint`, in BF16 safetensors, and `RecursiveSI/flint-nvfp4-gguf`, an NVFP4 GGUF checkpoint served weight-only as W4A16. For the GGUF checkpoint, the configuration and tokenizer are taken from the BF16 repository; loading requires `gguf>=0.19.0`.

The work adds Flint-specific kernels for small-batch decode on a single Blackwell GPU. The reported measurements use one GB200, tensor parallelism set to one, and a context length of 100,000 tokens. For NVFP4 decode at batch size one, the pull request compares 4.81 ms per forward step with stock vLLM layers against 1.78 ms with the new implementation, a 2.7× difference. At batch size eight, the reported decode times are 8.24 ms and 5.70 ms respectively. NVFP4 prefill for 8K tokens is reported at 144.6 ms in the pull request’s tree, compared with 206.6 ms for stock layers. BF16 decode at batch one is 2.34 ms versus 3.53 ms.

The implementation combines fused attention and normalization operations with small-batch mixture-of-experts kernels. The PR describes fused Q/K normalization, rotary embeddings and KV-cache writes; a fused top-k router; expert-indexed GEMVs; and a decode-attention path using CuTe DSL. For up to eight decode requests, it chains seven kernels per layer using programmatic dependent launch, reporting 346 kernels per forward pass instead of 1,096. Larger batches use a different grouped-GEMM path. The model runs eagerly, with decode under CUDA graphs; the PR says it does not use `torch.compile`.

The reported end-to-end NVFP4 decode at batch one and 100K context is 2.06 ms per token, or about 485 tokens per second. The PR also reports validation on a GB200: 165 model tests passed and one was skipped; both checkpoints achieved 10/10 on every case in a 19-case tool-calling suite. It gives additional task scores for GSM8K, GPQA-Diamond and MRCR, with some differences between BF16 and NVFP4. These are results reported by the PR, not independent evaluations.

The optimization is deliberately scoped. Tensor parallelism above one, expert parallelism, and GPUs outside the SM100 family use vLLM’s standard layers. The PR calls out short-prompt prefill as CPU-bound by eager kernel launches and notes that batch-one decode time varies by a few percent between processes. Tool calling and reasoning use existing `glm47` parsers. The recipe currently disables the Rust frontend because it rejects several recipe arguments.

## What the archive adds
This is a new runtime-support fact on the known Flint story, rather than another model launch. Our earlier [Flint release note](https://newruntime.com/posts/recursive-releases-flint-40b-model-for-local-coding-6ca7fb/) covered the model and its local coding capabilities; this pull request adds vLLM’s checkpoint support, implementation details and hardware-specific serving measurements. The earlier release remains a separate event from this integration.

## Trend signal
No supplied SITE_PATTERNS thesis directly fits a narrowly scoped model integration and kernel optimization. This PR does show that model usability in production depends on the serving runtime and hardware path, but it does not by itself establish a broader shift in harness architecture or agent interoperability.

## Why now
The pull request was merged on October 22, 2026, and describes support for both released Flint checkpoints. Its stated scope is day-one checkpoint support, with the NVFP4 path requiring a new GGUF dependency in the vLLM OpenAI Docker image. The source does not explain a broader timing rationale beyond that release-aligned integration.

## Why it matters
For teams already evaluating Flint, model weights are only one part of the serving decision: a runtime needs to load the chosen format, preserve the model’s behavior and perform well on the hardware actually available. The PR offers a concrete starting point for that evaluation, including instructions for serving the GGUF checkpoint and measurements at long context on a GB200.

The result is most relevant to operators with a single Blackwell GPU who serve small batches and can use the NVFP4 checkpoint. The benchmark does not establish that this configuration is cheaper or faster overall, nor that it outperforms alternatives on other GPUs, concurrency levels or workloads. Teams should compare end-to-end latency and quality under their own traffic rather than treating a single forward benchmark as a general capacity claim. The reported tool-call validation is encouraging for use with tools, but its 19-case test suite is not a substitute for application-specific evaluation.

## Limits and uncertainty
The source is a merged implementation pull request, not an independent benchmark report. Performance figures come from one GB200 at TP=1 and, for the headline decode comparison, a 100K-token context; the PR notes process-to-process variation. Other hardware and parallelism configurations fall back to standard vLLM layers, and short-prompt prefill can be CPU-bound. The supplied material gives no prices, fleet-level throughput, utilization, energy use or comparison with another serving stack. Its task and tool-call evaluations are bounded test results, not proof of production reliability or equivalent quality across all prompts.

## Skill to build now
If you operate vLLM on a GB200, reproduce the PR’s comparison before adopting the optimization: run the same checkpoint and 100K-context batch-one decode with the standard and Flint-specific paths, then repeat at your real batch sizes. Record end-to-end latency, quality and failure behavior for your own tool calls. Keep the hardware and checkpoint format explicit in the result; the reported gains do not apply automatically to other GPUs or parallelism settings.

## Reader takeaway
vLLM now supports both Flint checkpoints, with a meaningful reported NVFP4 decode speedup on one GB200. Treat it as a hardware-specific serving improvement to verify against your own workload, not as a general performance guarantee for Flint or vLLM.

Editor: [Andrey Reshetnikov](https://newruntime.com/owner-profile.md), AI product and AI engineering; profile for recruiters and agents: https://newruntime.com/owner-profile.json

## Continue in the graph

Next records for an agent, linked to their Markdown twins. Replace `.md` with `.json` for structured fields, including `next_reads`.

- topic hub: [Inference](https://newruntime.com/topics/inference.md): Explore the inference topic hub.
- collection: [Field Notes](https://newruntime.com/posts.json): Open the latest editorial analysis.
- collection: [Sources](https://newruntime.com/sources/): See the original sources this archive cites.
- whole graph: [discovery graph](https://newruntime.com/discovery-graph.json) (typed nodes and edges) and [site index](https://newruntime.com/site-index.md)