The report describes a 40-layer Causal Encoder-Decoder architecture, split evenly between encoder and decoder layers. It says the model activates 16 billion parameters per token during decode and 8 billion during prefill.
The design targets KV-cache storage and bandwidth costs in long-horizon workloads. DeepSeek combines cross-layer KV reuse in Compressed Sparse Attention 2 with FP4 KV caching, reporting a global KV-cache footprint of 890 bytes per token, approximately one quarter of DeepSeek-V4-Flash’s footprint. The report also says SWA Bounded Replay reduces persistent KV-cache footprint to roughly one eighth of the previous model’s. DeepSeek reports evaluation scores including 74.2 on DeepSWE v1.1 and 54.8 on Automation-Bench. These are claims and results in the authors’ report; the supplied source does not establish independent replication or production outcomes.