Observed facts: Prime-RL’s documentation defines IPO masking as |π_train − π_infer| > ε and contrasts it with IcePop’s ratio band. Observed facts: In the reported GLM-4.5-Air SWE RL ablation, IcePop’s trainer–inference KL rose to more than 20× baseline around step 1.2k while solve rate fell from about 0.8 to about 0.3; both IPO arms kept mismatch low through that window. These results come from a particular setup: a 100B MoE model, specific trainer and inference topology, batch and context settings, and SWE-Bench Verified evaluation.
Mechanism: The ratio treats tokens similarly regardless of baseline probability; an absolute difference instead responds to the amount of probability mass moved. In the two-outcome framing used by the documentation, that difference is a total-variation distance. The Why now: Prime-RL identifies IPO as its default loss and publishes configurations to reproduce the comparison, making this a practical design choice for teams running asynchronous RL rather than a purely theoretical proposal.
First implication: Teams using asynchronous policy training can compare probability-difference masking against ratio masking while tracking mismatch KL, solve rate, entropy, and masked-token fraction. Evidence boundary: The supplied document reports one controlled ablation on one model/task recipe; it does not establish general superiority across models, tasks, or training systems.
First watch condition: Whether independent runs across different tasks and trainer–inference configurations reproduce IPO’s lower mismatch without sacrificing task performance.