---
type: "post"
id: "cerebras-launches-disaggregated-inference-bf0ec6"
slug: "cerebras-launches-disaggregated-inference-bf0ec6"
title: "Cerebras Launches Disaggregated Inference"
description: "Cerebras says a production deployment that split prompt processing from response generation delivered five times the throughput per megawatt of a CS-3-only setup, with no loss in token-generation speed."
retrieval_nugget: "Cerebras says a production deployment that split prompt processing from response generation delivered five times the throughput per megawatt of a CS-3-only setup, with no loss in token-generation speed. The result makes workload placement—not just accelerator choice—a consequential lever for inference capacity, though the company’s measurement is tied to one endpoint and deployment."
published_at: "2026-11-18"
updated_at: "2026-11-18"
record_date: "2026-11-18"
date_kind: "published_at"
topics: ["inference"]
entities: ["Cerebras","Cerebras Inference Cloud","CS-3","AMD Instinct","GPT-OSS-120B","AWS","Fireworks AI","Gimlet Labs"]
related_patterns: []
novelty_label: "new_fact"
what_changed: "Cerebras has put its disaggregated inference system into production, pairing GPU prefill with Cerebras decode to deliver five times more tokens per megawatt."
prior_refs: []
source_url: "https://www.cerebras.ai/blog/cerebras-launches-disaggregated-inference"
source_urls: ["https://www.cerebras.ai/blog/cerebras-launches-disaggregated-inference"]
schema_version: "newruntime-agent-readable-v0.2"
stable_id: "post:cerebras-launches-disaggregated-inference-bf0ec6"
status: "published"
visuals: []
editor: {"role":"editor","name":"Andrey Reshetnikov","profile":"https://newruntime.com/owner-profile.md","profile_json":"https://newruntime.com/owner-profile.json","same_as":["https://www.linkedin.com/in/reshetnikov1/","https://t.me/lapetuse"]}
routes: {"html":"https://newruntime.com/posts/cerebras-launches-disaggregated-inference-bf0ec6/","markdown":"https://newruntime.com/posts/cerebras-launches-disaggregated-inference-bf0ec6.md","json":"https://newruntime.com/posts/cerebras-launches-disaggregated-inference-bf0ec6.json"}
markdown_url: "https://newruntime.com/posts/cerebras-launches-disaggregated-inference-bf0ec6.md"
---

# Cerebras Launches Disaggregated Inference

## Retrieval answer

Cerebras says a production deployment that split prompt processing from response generation delivered five times the throughput per megawatt of a CS-3-only setup, with no loss in token-generation speed. The result makes workload placement—not just accelerator choice—a consequential lever for inference capacity, though the company’s measurement is tied to one endpoint and deployment.

## Summary
Cerebras says a production deployment that split prompt processing from response generation delivered five times the throughput per megawatt of a CS-3-only setup, with no loss in token-generation speed. The result makes workload placement—not just accelerator choice—a consequential lever for inference capacity, though the company’s measurement is tied to one endpoint and deployment.

## What happened
Cerebras announced disaggregated inference on November 17, 2026. In the configuration described, AMD Instinct GPUs handle prefill: processing a prompt and producing intermediate model state. Cerebras CS-3 systems handle decode, generating the response one token at a time. The company says this arrangement is now in production on Cerebras Inference Cloud. Moving prefill to GPUs lets Cerebras systems devote their serving capacity to decode.

Cerebras reports that the production configuration delivered five times the throughput per megawatt of a CS-3-only, or aggregated, configuration, with no loss in token-generation speed. The comparison used a shared GPT-OSS-120B endpoint through Cerebras Inference Cloud on November 14. The company normalized the aggregated baseline to 1× and says the disaggregated setup reached 5×. It does not give absolute tokens-per-second or power figures in the announcement.

The design targets a difference between the two stages. Prefill processes the input; decode repeatedly reads model weights to generate successive tokens. Cerebras argues that its wafer-scale systems are advantageous for decode because they place weights in on-chip SRAM. The company says a single chip has tens of gigabytes of SRAM and tens of petabytes per second of memory bandwidth, comparing this with a typical GB300 GPU as 350× the SRAM capacity and 5,400× the memory bandwidth. Those are Cerebras’s stated comparisons, not independent measurements presented in the announcement.

For infrastructure partners, Cerebras describes a reference architecture that co-locates partner-selected prefill accelerators and Cerebras systems on a unified network. Its end-to-end stack routes requests between prefill and decode pools, manages replica capacity and health, supports prefill engines such as vLLM or SGLang, and includes a Cerebras decode engine. A component called KV Portal transfers intermediate state, including KV caches, between the engines over RDMA.

Partners can instead use a decode-only SDK, bringing their own inference framework and prefill engine while using Cerebras’s decode engine and KV Portal. Cerebras says this option is suited to partners seeking more control; it also includes a mock decode replica and a reference integration. AWS, Fireworks AI, Gimlet Labs and General Compute have early access. Cerebras says it is expanding access and plans broad deployment across hundreds of megawatts in its cloud and partner deployments by the end of 2027. No general price or self-serve access terms are announced.

## What the archive adds
New Runtime's earlier coverage described Cerebras’s CS-4 system and its Hot Chips 2026 architecture. This announcement is a new deployment result: it moves the discussion from system design to a production claim about throughput per megawatt. [Earlier coverage of CS-4 and wafer-scale architecture](https://newruntime.com/posts/infinigence-ai-and-three-body-universe-launch-sophon-cc8c5e/) is not the relevant Cerebras note, so no link is attached here.

## Trend signal
This supports the pattern that **agent economics moves to completed work** (https://newruntime.com/patterns/agent-economics-moves-to-completed-work/) only indirectly: Cerebras frames the benefit as more inference throughput from a constrained power budget, not cost per verified task. The stronger signal is that infrastructure vendors are optimizing capacity around workload stages. This does not yet fit a pattern New Runtime tracks.

## Why now
Cerebras says demand for fast inference grew on its own cloud and through partners during 2026, while agentic applications such as Muse, Copilot and Codex require interactive response speeds. It cites a power-constrained environment and a Goldman Sachs estimate that inference demand could grow much faster than data-center capacity. The deployment is presented as a response to that capacity gap, with broader rollouts planned for 2027.

## Why it matters
For teams operating inference services, a single accelerator type may not be the best way to serve every stage of a request. If prompt processing and token generation have different compute and memory requirements, assigning them to different hardware pools can raise aggregate throughput without sacrificing the decode speed users notice. That could make available power go further as interactive agents increase request volume.

The tradeoff is operational. A split architecture requires routing, capacity and health management, a fast network, and transfer of intermediate state such as KV caches. Partners must weigh those integration costs against the throughput gain under their own models, prompt lengths, concurrency and latency targets. Cerebras’s result is a useful reason to test stage separation, not proof that fivefold gains will transfer to another workload or vendor’s stack.

## Limits and uncertainty
The fivefold figure comes from Cerebras’s production deployment and its comparison of one shared GPT-OSS-120B endpoint before and after enabling disaggregation. The announcement does not provide absolute throughput, power consumption, test duration, latency distributions, utilization, or the hardware configuration details needed to reproduce the result. It also does not report an independent audit of this measurement. Cerebras cautions that observed speed comparisons can vary with workload, configuration, date and model. Partner access is expanding, but pricing, deployment requirements and generally available access terms are not specified.

## Reader takeaway
The important claim is not simply that Cerebras added GPUs: the company says separating prefill from decode multiplied throughput per megawatt in a production setup while preserving generation speed. Treat 5× as a workload-specific vendor result. For capacity planning, benchmark the two stages separately and include the network and KV-state transfer overhead before deciding whether disaggregation pays off.

Editor: [Andrey Reshetnikov](https://newruntime.com/owner-profile.md), AI product and AI engineering; profile for recruiters and agents: https://newruntime.com/owner-profile.json

## Continue in the graph

Next records for an agent, linked to their Markdown twins. Replace `.md` with `.json` for structured fields, including `next_reads`.

- topic hub: [Inference](https://newruntime.com/topics/inference.md): Explore the inference topic hub.
- collection: [Field Notes](https://newruntime.com/posts.json): Open the latest editorial analysis.
- collection: [Sources](https://newruntime.com/sources/): See the original sources this archive cites.
- whole graph: [discovery graph](https://newruntime.com/discovery-graph.json) (typed nodes and edges) and [site index](https://newruntime.com/site-index.md)
