Observed facts: The paper introduces CitePretrainBench with Wikipedia, Common Crawl, arXiv, and novel unseen documents, spanning short- and long-form citation tasks. On Qwen-2.5-7B and 3B, Active Indexing improved citation precision over Passive Indexing on every evaluated task, with gains up to 30.2 percentage points.
Mechanism: Continual pretraining binds facts to persistent document identifiers; instruction tuning teaches the model to emit identifiers as citations. Active Indexing adds synthetic source-to-fact and fact-to-source examples. Continual pretraining binds facts to persistent document identifiers; instruction tuning teaches the model to emit identifiers as citations. Active Indexing adds synthetic source-to-fact and fact-to-source examples.
Why now: The paper tests whether internalized source associations can complement or sometimes substitute for external retrieval when traceability is sought without a retrieval call per answer.
First implication: The first implication is that internal citations may offer a lower-latency fallback when retrieval is unavailable, while remaining limited to the indexed training corpus.
Evidence boundary: The evidence boundary is the authors’ benchmark and experiments on Qwen-2.5-7B and 3B; the reported gains do not establish reliable provenance for arbitrary answers or production use.
First watch condition: The first watch condition is whether independent evaluations test citation resolution and attribution errors on models and corpora beyond this benchmark.
A document identifier is only useful as evidence if it can be resolved against a maintained catalog and if the cited document actually supports the claim. The paper therefore describes a route to learned attribution, not a guarantee that citations provide faithful explanations of how a model produced an answer. External retrieval and internal citations can be complementary: one depends on current indexes, the other on what was represented during training.