---
type: "post"
id: "code-and-data-released-at-thu-coai-backdoor-a427ad"
slug: "code-and-data-released-at-thu-coai-backdoor-a427ad"
title: "Code and data released at thu-coai/Backdoor-Data-Extraction ."
description: "**Thesis:** Fine-tuning on a base model from an untrusted source can make private training examples vulnerable to extraction, so model provenance is part of data protection."
retrieval_nugget: "**Thesis:** Fine-tuning on a base model from an untrusted source can make private training examples vulnerable to extraction, so model provenance is part of data protection."
published_at: "2026-10-16"
updated_at: "2026-10-16"
record_date: "2026-10-16"
date_kind: "published_at"
topics: ["ai-security","models","security"]
source_urls: ["https://github.com/thu-coai/Backdoor-Data-Extraction"]
schema_version: "newruntime-agent-readable-v0.2"
stable_id: "post:code-and-data-released-at-thu-coai-backdoor-a427ad"
status: "published"
visuals: []
editorial_provenance: {"schema_version":"newruntime-editorial-copy-v1","content_status":"source_grounded_final","final_copy_sha256":"sha256:1964ec745d06462d77ba0cb8fb4d06d88479939a6f723fbdd9cc2f38833864d5","reviewed_at":"2026-10-16T03:00:13.440Z","source_evidence_count":1,"verified_claim_count":3,"site_analysis_schema_version":"newruntime-site-analysis-v1","site_object_kind":"field_note","observed_fact_count":3,"implication_count":1,"watch_condition_count":1,"related_record_count":0}
analysis: {"schema_version":"newruntime-site-analysis-v1","object_kind":"field_note","thesis":"Fine-tuning on a base model from an untrusted source can make private training examples vulnerable to extraction, so model provenance is part of data protection.","observed_facts":[{"text":"The repository README describes experiments across four base models ranging from 3B to 32B parameters and two downstream datasets: Dolly and Finance-alpaca.","source_urls":["https://github.com/thu-coai/Backdoor-Data-Extraction"]},{"text":"The README reports up to 76.3% perfect extraction of 5,000 queries in practical settings and 94.9% in more ideal settings.","source_urls":["https://github.com/thu-coai/Backdoor-Data-Extraction"]},{"text":"The README says a detection-based defense can be bypassed by an improved attack and provides training and evaluation scripts plus processed datasets.","source_urls":["https://github.com/thu-coai/Backdoor-Data-Extraction"]}],"mechanism":"The backdoor is planted during pre-release training; later fine-tuning incorporates the private dataset, and queries to the resulting model can elicit those examples.","why_now":"The code and data release lets other researchers inspect and attempt to reproduce a concrete supply-chain attack claim, shifting discussion from a hypothetical risk toward a testable research result.","implications":["Teams fine-tuning open-weight checkpoints should treat base-model provenance and post-training behavior checks as part of protecting private training data, not assume that dataset access controls alone are sufficient."],"evidence_boundary":"The supplied public source is the project repository and its README; it documents controlled experiments and released artifacts, not attacks observed in deployed systems or the prevalence of vulnerable models in use.","watch_conditions":["Look for independent reproduction or evaluation of the extraction method on real-world fine-tuning pipelines and defenses."],"related_records":[],"new_branch_reason":"No prior New Runtime record was supplied for this research release, so it starts a distinct record on reproducible model-supply-chain data-extraction evidence."}
editor: {"role":"editor","name":"Andrey Reshetnikov","profile":"https://newruntime.com/owner-profile.md","profile_json":"https://newruntime.com/owner-profile.json","same_as":["https://www.linkedin.com/in/reshetnikov1/","https://t.me/lapetuse"]}
routes: {"html":"https://newruntime.com/posts/code-and-data-released-at-thu-coai-backdoor-a427ad/","markdown":"https://newruntime.com/posts/code-and-data-released-at-thu-coai-backdoor-a427ad.md","json":"https://newruntime.com/posts/code-and-data-released-at-thu-coai-backdoor-a427ad.json"}
---

# Code and data released at thu-coai/Backdoor-Data-Extraction .

## Retrieval answer

**Thesis:** Fine-tuning on a base model from an untrusted source can make private training examples vulnerable to extraction, so model provenance is part of data protection.

**Thesis:** Fine-tuning on a base model from an untrusted source can make private training examples vulnerable to extraction, so model provenance is part of data protection.

The repository’s README identifies an attack path in which a base-model creator trains a backdoor before release, then uses black-box access to a downstream model fine-tuned on proprietary examples. **Mechanism:** The backdoor is planted during pre-release training; later fine-tuning incorporates the private dataset, and queries to the resulting model can elicit those examples. The repository provides scripts for backdoor and downstream training, evaluation code, and processed datasets.

**Observed facts:** The README describes experiments across four base models from 3B to 32B parameters and two downstream datasets, Dolly and Finance-alpaca. It reports up to 76.3% perfect extraction of 5,000 queries in practical settings, and 94.9% in more ideal settings. It also says a detection-based defense can be bypassed by an improved attack. These figures are the authors’ reported results, not independent validation in this record.

**Why now:** The code and data release lets other researchers inspect and attempt to reproduce a concrete supply-chain attack claim, shifting discussion from a hypothetical risk toward a testable research result. **First implication:** Teams fine-tuning open-weight checkpoints should treat base-model provenance and post-training behavior checks as part of protecting private training data, not assume that dataset access controls alone are sufficient.

**Evidence boundary:** The supplied public source is the project repository and its README; it documents controlled experiments and released artifacts, not attacks observed in deployed systems or the prevalence of vulnerable models in use.

**First watch condition:** Look for independent reproduction or evaluation of the extraction method on real-world fine-tuning pipelines and defenses.

Editor: [Andrey Reshetnikov](https://newruntime.com/owner-profile.md), AI product and AI engineering; profile for recruiters and agents: https://newruntime.com/owner-profile.json

## Continue in the graph

Next records for an agent, linked to their Markdown twins. Replace `.md` with `.json` for structured fields, including `next_reads`.

- topic hub: [Ai Security](https://newruntime.com/topics/ai-security.md): Explore the ai-security topic hub.
- topic hub: [Models](https://newruntime.com/topics/models.md): Explore the models topic hub.
- topic hub: [Security](https://newruntime.com/topics/security.md): Explore the security topic hub.
- collection: [Field Notes](https://newruntime.com/posts.json): Open the latest editorial analysis.
- collection: [Source Ledger](https://newruntime.com/source-ledger.json): Inspect the public source evidence graph.
- whole graph: [discovery graph](https://newruntime.com/discovery-graph.json) (typed nodes and edges) and [site index](https://newruntime.com/site-index.md)