Yingru Li’s essay says that on samples drawn from the model, the score has zero mean. Mechanism: Centering of the score under the sampling distribution turns the expected gradient into a covariance; Cauchy–Schwarz bounds it by signal spread and Fisher spread. At token level, common logit movement cancels and only relative candidate sensitivity contributes. The essay also derives the bound |G_i| ≤ std(A)√F_i and says a large Fisher is necessary but not sufficient for a large gradient.
Why now: The essay distinguishes small post-training updates from sparse updates, a distinction needed to interpret claims about how training changes model parameters. First implication: Fisher identifies where large updates are possible, while reward alignment determines whether those updates occur. The essay reports that, in cited measurements, the top 1% of coordinates by Fisher contained 47.40% of the top 1% by gradient magnitude; that is a reported result, not a general guarantee.
Evidence boundary: This is a summary of one theoretical essay and its reported empirical result. The supplied source excerpt does not independently validate the measurement or establish that the mechanism generalizes across training setups. It is part one of a three-part series. First watch condition: Check whether the later parts of the series and independent evaluations clarify where the sparsity mechanism holds across algorithms, sampling distributions, and training data.