The authors use DeepSeek OCR as a baseline and replace all attention layers in its decoder with R-SWA.
The authors say the design keeps the key-value cache constant during decoding, addressing memory use that grows with output length. Combined with DeepSeek OCR’s encoder compression, it can transcribe dozens of document pages in one forward pass under a standard maximum length of 32K.
The report describes R-SWA as a general-purpose parsing attention mechanism that could also be applied to speech recognition and translation. The code and model weights are publicly available, according to the paper. The source does not establish comparative accuracy or performance on independent benchmarks, and it does not report results for those other proposed applications.