---
type: "post"
id: "betelgeuse-llama-cpp-fork-pages-moe-experts-from-fb5996"
slug: "betelgeuse-llama-cpp-fork-pages-moe-experts-from-fb5996"
title: "Betelgeuse: llama.cpp fork pages MoE experts from NVMe to run a 125B Qwen3.8-Flash-Next on a 12 GB RTX 3060"
description: "Betelgeuse is an experimental MIT-licensed fork of llama.cpp designed to run a 125B Qwen3.8-Flash-Next mixture-of-experts model on an RTX 3060 with 12 GB of VRAM."
retrieval_nugget: "Betelgeuse is an experimental MIT-licensed fork of llama.cpp designed to run a 125B Qwen3.8-Flash-Next mixture-of-experts model on an RTX 3060 with 12 GB of VRAM. It matters because it pages routed expert weights from local NVMe into a bounded GPU cache instead of requiring the entire model to reside in VRAM."
published_at: "2026-10-31"
updated_at: "2026-10-31"
record_date: "2026-10-31"
date_kind: "published_at"
topics: []
entities: ["Betelgeuse","llama.cpp","Qwen3.8-Flash-Next","NVIDIA RTX 3060"]
related_patterns: []
novelty_label: "new_fact"
what_changed: "Betelgeuse’s llama.cpp fork adds NVMe-paged MoE inference, targeting Qwen3.8-Flash-Next on a 12 GB RTX 3060."
prior_refs: ["memory:17a3e1b1-ba34-8990-ae23-a4a7bfd7b04e","memory:a8cb1a51-c12d-8ddf-a83b-0510336c6bf2"]
source_url: "https://github.com/Betelgeuse-AI/Betelgeuse"
source_urls: ["https://github.com/Betelgeuse-AI/Betelgeuse"]
schema_version: "newruntime-agent-readable-v0.2"
stable_id: "post:betelgeuse-llama-cpp-fork-pages-moe-experts-from-fb5996"
status: "published"
visuals: []
editor: {"role":"editor","name":"Andrey Reshetnikov","profile":"https://newruntime.com/owner-profile.md","profile_json":"https://newruntime.com/owner-profile.json","same_as":["https://www.linkedin.com/in/reshetnikov1/","https://t.me/lapetuse"]}
routes: {"html":"https://newruntime.com/posts/betelgeuse-llama-cpp-fork-pages-moe-experts-from-fb5996/","markdown":"https://newruntime.com/posts/betelgeuse-llama-cpp-fork-pages-moe-experts-from-fb5996.md","json":"https://newruntime.com/posts/betelgeuse-llama-cpp-fork-pages-moe-experts-from-fb5996.json"}
---

# Betelgeuse: llama.cpp fork pages MoE experts from NVMe to run a 125B Qwen3.8-Flash-Next on a 12 GB RTX 3060

## Retrieval answer

Betelgeuse is an experimental MIT-licensed fork of llama.cpp designed to run a 125B Qwen3.8-Flash-Next mixture-of-experts model on an RTX 3060 with 12 GB of VRAM. It matters because it pages routed expert weights from local NVMe into a bounded GPU cache instead of requiring the entire model to reside in VRAM.

## Summary

Betelgeuse is an experimental MIT-licensed fork of llama.cpp designed to run a 125B Qwen3.8-Flash-Next mixture-of-experts model on an RTX 3060 with 12 GB of VRAM. It matters because it pages routed expert weights from local NVMe into a bounded GPU cache instead of requiring the entire model to reside in VRAM.

## What happened

Betelgeuse-AI published Betelgeuse, an open-source C++/CUDA fork aimed at frontier-scale mixture-of-experts, or MoE, inference on consumer hardware. The repository identifies the release as v0.6.0-alpha-orionis under the MIT license. It provides two binaries, `bel-server` and `bel-cli`. Its stated target machine is specific: Linux x86-64, CUDA sm_86, an NVIDIA RTX 3060 with 12 GB of VRAM, 16 GB of system RAM, and local NVMe storage. This is not a generic claim that any small GPU can run a large model; it is an implementation built around a defined storage and memory configuration.

The central design choice is to retain dense computation on the GPU while paging only routed MoE expert weights from storage. Betelgeuse stores experts in a local, 4 KiB-aligned `experts.bin` file. For each token, it brings selected experts into a bounded pool of VRAM slots rather than keeping every expert resident. The project also describes a host-side victim cache and buffered `io_uring` reads. The result is a hierarchy: GPU memory holds the dense path and the currently useful experts, system memory can retain evicted experts, and NVMe backs the larger expert set.

The README points users to a roughly 68 GB, four-file Qwen3.8-Flash-Next bundle on Hugging Face, named `Betelgeuse-AI/Alpha-Orionis-XS`. The supplied project material reports 16.80 tokens per second on a first request and 22.03 tokens per second when warm. It contrasts those figures with a user-reported 0.9 tokens per second for upstream llama.cpp. However, the reported results came from separate historical runs, rather than from a controlled same-machine benchmark included in the material. The contrast is therefore useful as a project claim, not as a settled performance comparison.

The repository does not present Betelgeuse as a finished local-serving platform. Its own material calls the project experimental and partly broken. A related project post says testing was limited to 512-token context. That boundary matters because paging behavior can change materially with longer contexts, different prompts, routing patterns, cache state, and request mixes. The release demonstrates a working approach and provides software that builders can inspect and test, but it does not establish production reliability, broad hardware support, or stable performance under sustained traffic.

The technical premise follows directly from MoE structure. A MoE model can have a large total parameter count while selecting only some experts for a token. Betelgeuse uses that distinction to move capacity pressure away from VRAM and onto storage bandwidth, caching, and data movement. It does not make storage equivalent to GPU memory: it creates an explicit runtime that manages the gap between them.

## What the archive adds

This is a new implementation in an active local-MoE inference thread. [Slotstream runs a 125B MoE model on 16 to 64 GB Macs by streaming experts from SSD.](https://newruntime.com/posts/slotstream-runs-a-125b-moe-model-on-16-1dfd4c/) described an Apple Silicon-specific SSD-streaming approach; Betelgeuse instead targets an RTX 3060 and local NVMe. [A recent llama.cpp change](https://newruntime.com/posts/llama-cpp-merges-a-qwen4exp-change-that-cuts-e05367/) reduced Qwen3.8-Flash-Next compute-buffer memory by 2–4 GB, whereas this release addresses expert-weight residency.

## Trend signal

No supplied Pattern or archive assumption genuinely captures this storage-and-inference implementation change.

## Why now

The timing is explained by a practical mismatch: the cited Qwen3.8-Flash-Next bundle is roughly 68 GB, while the target RTX 3060 has 12 GB of VRAM. Betelgeuse is an attempt to make that mismatch operational by separating dense computation from expert storage and by managing expert residency at runtime. The accompanying recent llama.cpp memory reduction for the same model family makes the surrounding local-inference work especially relevant, but it solves a different part of the memory budget.

## Connections to other events

The local-inference thread includes a different hardware path in [Slotstream runs a 125B MoE model on 16 to 64 GB Macs by streaming experts from SSD.](https://newruntime.com/posts/slotstream-runs-a-125b-moe-model-on-16-1dfd4c/). Both projects rely on the fact that not every MoE expert must be active for each token, but their target platforms differ: Slotstream is described for Apple Silicon Macs, while Betelgeuse specifies CUDA sm_86 and an RTX 3060. The related llama.cpp update concerns compute-buffer memory rather than streaming expert weights.

## Why it matters

For teams evaluating local models, total parameter count is becoming a less useful first filter than the model's active path and its data-movement requirements. MoE architectures activate only a subset of experts per token, and Betelgeuse attempts to use that property by treating fast local storage as a slower extension of VRAM. That can expand the set of models a developer can experiment with on an existing desktop GPU without claiming that all workloads will be practical.

The trade-off is operational rather than magical. The inference path now depends on NVMe behavior, cache warmth, expert-routing patterns, context length, and the balance between reads and computation. A headline throughput number can omit cold-start costs and latency variation that matter to interactive tools, background automations, or agent systems. Builders should regard the release as an option for testing capacity-constrained local inference, not as evidence that a 12 GB GPU has become interchangeable with a larger-memory deployment.

It also makes the runtime design visible. The useful engineering question is no longer only which model quantization fits into VRAM. It is which tensors must be resident, which can be fetched, how predictable fetches are, and whether the resulting latency envelope fits the application. That is a testable systems question, and the project supplies a concrete implementation to examine.

## What happens next

- **Independent testers will publish longer-context measurements for the stated RTX 3060 configuration.** (3 months, 60%). Watch for: reproducible prompts, cold and warm runs, and disk-I/O reporting.
- **Storage-paged MoE experiments will compare cache policies and drive behavior rather than tokens per second alone.** (6 months, 55%). Watch for: p95 latency, context-length, and NVMe-read metrics.

## Limits and uncertainty

The supplied material does not include an independent benchmark, a reproducible long-context evaluation, or a controlled same-hardware comparison with upstream llama.cpp. The 16.80 and 22.03 tokens-per-second figures are self-reported, come from separate historical runs, and are tied to the named RTX 3060 configuration. The material does not establish concurrency behavior, time to first token, tail latency, reliability under continuous service, output quality, or how different NVMe drives affect results.

The project’s experimental and partly broken status is material to any adoption decision. Testing limited to 512-token context also leaves a meaningful gap for applications that need longer prompts, retrieval context, multi-turn history, or sustained serving. The source establishes a release and its stated mechanism; it does not establish a general local-deployment recommendation.

## Skill to build now

Create a local-inference acceptance test before trying storage-paged models. Record cold-start time, warm throughput, time to first token, p95 latency, disk reads, and failure rate at the context lengths your application actually uses. Run the same prompt set with a warm cache and an empty cache, then repeat after sustained requests. That small harness will show whether a memory-saving approach improves your workload or merely improves a narrow benchmark.

## Reader takeaway

Betelgeuse offers a concrete implementation of a useful local-inference idea: keep the dense path on the GPU and page selected MoE experts from NVMe. It may let builders experiment with a much larger model on a 12 GB GPU, but the present evidence is self-reported, short-context, and hardware-specific. Treat it as an experimental system to measure, not a proven serving shortcut.

## Open question

Can independent long-context tests reproduce the reported throughput while keeping latency, cache behavior, and NVMe pressure acceptable for real applications?

Editor: [Andrey Reshetnikov](https://newruntime.com/owner-profile.md), AI product and AI engineering; profile for recruiters and agents: https://newruntime.com/owner-profile.json

## Continue in the graph

Next records for an agent, linked to their Markdown twins. Replace `.md` with `.json` for structured fields, including `next_reads`.

- collection: [Field Notes](https://newruntime.com/posts.json): Open the latest editorial analysis.
- collection: [Source Ledger](https://newruntime.com/source-ledger.json): Inspect the public source evidence graph.
- whole graph: [discovery graph](https://newruntime.com/discovery-graph.json) (typed nodes and edges) and [site index](https://newruntime.com/site-index.md)