{"type":"post","id":"code-and-data-released-at-thu-coai-backdoor-a427ad","slug":"code-and-data-released-at-thu-coai-backdoor-a427ad","title":"Code and data released at thu-coai/Backdoor-Data-Extraction .","description":"**Thesis:** Fine-tuning on a base model from an untrusted source can make private training examples vulnerable to extraction, so model provenance is part of data protection.","retrieval_nugget":"**Thesis:** Fine-tuning on a base model from an untrusted source can make private training examples vulnerable to extraction, so model provenance is part of data protection.","published_at":"2026-10-16","updated_at":"2026-10-16","record_date":"2026-10-16","date_kind":"published_at","topics":["ai-security","models","security"],"source_urls":["https://github.com/thu-coai/Backdoor-Data-Extraction"],"schema_version":"newruntime-agent-readable-v0.2","stable_id":"post:code-and-data-released-at-thu-coai-backdoor-a427ad","status":"published","visuals":[],"editorial_provenance":{"schema_version":"newruntime-editorial-copy-v1","content_status":"source_grounded_final","final_copy_sha256":"sha256:1964ec745d06462d77ba0cb8fb4d06d88479939a6f723fbdd9cc2f38833864d5","reviewed_at":"2026-10-16T03:00:13.440Z","source_evidence_count":1,"verified_claim_count":3,"site_analysis_schema_version":"newruntime-site-analysis-v1","site_object_kind":"field_note","observed_fact_count":3,"implication_count":1,"watch_condition_count":1,"related_record_count":0},"analysis":{"schema_version":"newruntime-site-analysis-v1","object_kind":"field_note","thesis":"Fine-tuning on a base model from an untrusted source can make private training examples vulnerable to extraction, so model provenance is part of data protection.","observed_facts":[{"text":"The repository README describes experiments across four base models ranging from 3B to 32B parameters and two downstream datasets: Dolly and Finance-alpaca.","source_urls":["https://github.com/thu-coai/Backdoor-Data-Extraction"]},{"text":"The README reports up to 76.3% perfect extraction of 5,000 queries in practical settings and 94.9% in more ideal settings.","source_urls":["https://github.com/thu-coai/Backdoor-Data-Extraction"]},{"text":"The README says a detection-based defense can be bypassed by an improved attack and provides training and evaluation scripts plus processed datasets.","source_urls":["https://github.com/thu-coai/Backdoor-Data-Extraction"]}],"mechanism":"The backdoor is planted during pre-release training; later fine-tuning incorporates the private dataset, and queries to the resulting model can elicit those examples.","why_now":"The code and data release lets other researchers inspect and attempt to reproduce a concrete supply-chain attack claim, shifting discussion from a hypothetical risk toward a testable research result.","implications":["Teams fine-tuning open-weight checkpoints should treat base-model provenance and post-training behavior checks as part of protecting private training data, not assume that dataset access controls alone are sufficient."],"evidence_boundary":"The supplied public source is the project repository and its README; it documents controlled experiments and released artifacts, not attacks observed in deployed systems or the prevalence of vulnerable models in use.","watch_conditions":["Look for independent reproduction or evaluation of the extraction method on real-world fine-tuning pipelines and defenses."],"related_records":[],"new_branch_reason":"No prior New Runtime record was supplied for this research release, so it starts a distinct record on reproducible model-supply-chain data-extraction evidence."},"editor":{"role":"editor","name":"Andrey Reshetnikov","profile":"https://newruntime.com/owner-profile.md","profile_json":"https://newruntime.com/owner-profile.json","same_as":["https://www.linkedin.com/in/reshetnikov1/","https://t.me/lapetuse"]},"routes":{"html":"https://newruntime.com/posts/code-and-data-released-at-thu-coai-backdoor-a427ad/","markdown":"https://newruntime.com/posts/code-and-data-released-at-thu-coai-backdoor-a427ad.md","json":"https://newruntime.com/posts/code-and-data-released-at-thu-coai-backdoor-a427ad.json"},"next_reads":[{"type":"topic","path":"/topics/ai-security/","url":"https://newruntime.com/topics/ai-security/","title":"Ai Security","reason":"Explore the ai-security topic hub.","media_type":"text/html","markdown_url":"https://newruntime.com/topics/ai-security.md","json_url":"https://newruntime.com/topics/ai-security.json"},{"type":"topic","path":"/topics/models/","url":"https://newruntime.com/topics/models/","title":"Models","reason":"Explore the models topic hub.","media_type":"text/html","markdown_url":"https://newruntime.com/topics/models.md","json_url":"https://newruntime.com/topics/models.json"},{"type":"topic","path":"/topics/security/","url":"https://newruntime.com/topics/security/","title":"Security","reason":"Explore the security topic hub.","media_type":"text/html","markdown_url":"https://newruntime.com/topics/security.md","json_url":"https://newruntime.com/topics/security.json"},{"type":"collection","path":"/posts/","url":"https://newruntime.com/posts/","title":"Field Notes","reason":"Open the latest editorial analysis.","media_type":"text/html","json_url":"https://newruntime.com/posts.json"},{"type":"collection","path":"/source-ledger.json","url":"https://newruntime.com/source-ledger.json","title":"Source Ledger","reason":"Inspect the public source evidence graph.","media_type":"application/json"}]}