arXiv paper questions whether broader alignment improves cross-tokenizer distillation
A paper submitted to arXiv on 6 October 2026 examines cross-tokenizer on-policy distillation (OPD), in which a student learns from its own generations using teacher feedback. The authors study whether expanding alignment coverage improves learning across heterogeneous teacher–student pairs.
Across three pairs evaluated on mathematical reasoning and code generation, strict one-to-one alignment groups covered most student-generated tokens despite substantial vocabulary mismatch. On pre-distillation student responses, the shared vocabulary retained nearly all teacher and student probability mass at strictly aligned positions on average. Applying reverse KL to a student-selected top-16 subset of the shared vocabulary at each such position achieved accuracy comparable to full shared-vocabulary OPD and outperformed the evaluated cross-tokenizer baselines.
The paper also reports that adding mean-squared-error supervision on span log-probabilities in mismatch groups provided complete supervision coverage but reduced accuracy. At checkpoints trained with only the strict loss, span gradients showed weak or negative directional agreement with strict-loss gradients and grew in magnitude relative to them. The authors argue that these diagnostics may help explain the accuracy drop.
The result is limited to the paper’s evaluated model pairs and tasks. It suggests that increasing alignment coverage is not automatically beneficial when additional positions introduce less reliable or conflicting supervision. The paper is available at https://arxiv.org/abs/2610.08448.