The article says the approach cuts that cost to less than 1% of the original training cost and reports savings equivalent to trillions of tokens and hundreds of thousands of GPU-hours.
The reported 32-billion-parameter Gated KalmaNet-based hybrid matched its source Transformer on reasoning benchmarks and decoded up to twice as quickly with a 128,000-token context. The article also says state-space models outperformed sliding-window attention in its comparisons, including when the latter had twice the memory budget. These are results described in Amazon Science’s account; the report does not establish independent replication or production performance.