The paper describes a method that reconstructs text from a single hidden representation at the last token, using a trainable adapter and a decoder LLM. For short texts of about 16 words, the authors report recovering roughly half the original words while preserving overall meaning. These results are the observed basis for the analysis, not a claim about every model or representation.
Thesis
A model’s hidden representation may retain enough information to reconstruct parts of its input, making representation access relevant to privacy risk.
Observed facts
Rep2Text reconstructs text from a single hidden representation at the last token using a trainable adapter and a decoder LLM.
1For short texts of about 16 words, the paper reports recovering roughly half the original words while preserving overall meaning.
1The paper says exact wording is harder to recover for longer text, while main ideas can still be recovered.
1
Mechanism
Rep2Text trains an adapter to map a single hidden representation into a form that a decoder language model can use to generate reconstructed text.
Why now
The paper gives a concrete reconstruction result, rather than treating hidden states only as opaque internal data.
Implications
Teams should not assume that a non-text representation is inherently safe to expose or retain; its sensitivity depends on access controls and the specific model and decoding conditions.
Evidence boundary
The supplied arXiv abstract reports the method and summarized findings, but does not specify an operational attack, deployment prevalence, or the risk across all models and data types.
Watch conditions
Further evaluations should test reconstruction across model families, input lengths, and realistic access constraints before translating this result into a quantified product risk.