The repository’s README identifies an attack path in which a base-model creator trains a backdoor before release, then uses black-box access to a downstream model fine-tuned on proprietary examples. Mechanism: The backdoor is planted during pre-release training; later fine-tuning incorporates the private dataset, and queries to the resulting model can elicit those examples. The repository provides scripts for backdoor and downstream training, evaluation code, and processed datasets.
Observed facts: The README describes experiments across four base models from 3B to 32B parameters and two downstream datasets, Dolly and Finance-alpaca. It reports up to 76.3% perfect extraction of 5,000 queries in practical settings, and 94.9% in more ideal settings. It also says a detection-based defense can be bypassed by an improved attack. These figures are the authors’ reported results, not independent validation in this record.
Why now: The code and data release lets other researchers inspect and attempt to reproduce a concrete supply-chain attack claim, shifting discussion from a hypothetical risk toward a testable research result. First implication: Teams fine-tuning open-weight checkpoints should treat base-model provenance and post-training behavior checks as part of protecting private training data, not assume that dataset access controls alone are sufficient.
Evidence boundary: The supplied public source is the project repository and its README; it documents controlled experiments and released artifacts, not attacks observed in deployed systems or the prevalence of vulnerable models in use.
First watch condition: Look for independent reproduction or evaluation of the extraction method on real-world fine-tuning pipelines and defenses.