Reinforcement learning is often discussed as if it creates reasoning after pretraining ends. A new controlled study asks a more operational question: how strongly do pretraining choices determine the returns available from later RL compute?
The researchers use chess because the pipeline can be measured end to end. They pretrained language models from 5 million to 1 billion parameters on human games, supervised-fine-tuned them on synthetic reasoning traces, then applied RL to chess puzzles with verifiable rewards.
At a given RL compute level, post-RL performance was well predicted by pretraining loss. The slope of the RL reward curve also improved approximately linearly with the number of pretraining tokens. Better and longer pretraining did not merely raise the starting point; it changed how quickly the model benefited from RL.
RL does two different things
On easier puzzles, RL amplified correct moves the supervised policy already preferred. On harder puzzles, it surfaced correct moves that had very low probability after supervised fine-tuning.
That distinction matters. Some post-training gains are sharper selection among existing behaviors. Others look more like recovery of latent capabilities that the supervised policy almost never expressed.
The authors ran a transfer check with a 1-billion-parameter model pretrained on math-domain text. Longer-pretrained checkpoints again reached higher post-RL performance and improved faster, suggesting the predictive relationship is not limited to chess.
The study does not establish a universal LLM scaling law. Chess is a controlled environment, the tested models top out at 1 billion parameters, and the math transfer uses one model scale. But the design provides a way to reason about budget allocation across stages: measure the pretraining state, then estimate whether more RL compute is likely to have a steep or shallow return curve.
For teams training reasoning models, “pretraining versus RL” is the wrong split. Post-training efficiency is partly an inherited property of pretraining quality, data, and duration.