---
type: "post"
id: "your-transformer-can-hold-two-thoughts-at-once-c49646"
slug: "your-transformer-can-hold-two-thoughts-at-once-c49646"
title: "Your Transformer Can Hold Two Thoughts at Once: Evidence of Linear Superposition in LLMs"
description: "The paper’s central claim is that Transformers can preserve separable information from two unrelated text streams when their input representations are linearly combined."
retrieval_nugget: "The paper’s central claim is that Transformers can preserve separable information from two unrelated text streams when their input representations are linearly combined."
published_at: "2026-09-25"
updated_at: "2026-09-25"
record_date: "2026-09-25"
date_kind: "published_at"
topics: []
source_urls: ["https://huggingface.co/papers/2609.29845"]
schema_version: "newruntime-agent-readable-v0.2"
stable_id: "post:your-transformer-can-hold-two-thoughts-at-once-c49646"
status: "published"
visuals: []
editorial_provenance: {"schema_version":"newruntime-editorial-copy-v1","content_status":"source_grounded_final","final_copy_sha256":"sha256:d296e18ddb066ee2009713b9aeb2605a74e0e84a83170c8a4e8f22021a1a0983","reviewed_at":"2026-09-25T16:13:54.298Z","source_evidence_count":1,"verified_claim_count":2,"site_analysis_schema_version":"newruntime-site-analysis-v1","site_object_kind":"field_note","observed_fact_count":2,"implication_count":2,"watch_condition_count":2,"related_record_count":0}
analysis: {"schema_version":"newruntime-site-analysis-v1","object_kind":"field_note","thesis":"Thesis: Superposition linearity may be an architectural property of Transformers that can be measured, altered by training, and exploited for multi-stream decoding.","observed_facts":[{"text":"Observed facts: the authors report a superposition of individual next-token distributions after combining inputs, and they report that the effect diminishes during pretraining.","source_urls":["https://huggingface.co/papers/2609.29845"]},{"text":"The authors report that lightweight fine-tuning can substantially restore linearity and introduce guided decoding to separate the superposed output into two continuations.","source_urls":["https://huggingface.co/papers/2609.29845"]}],"mechanism":"Mechanism: Linear mixing of two input representations is reported to produce a mixed next-token distribution, which guided decoding then attempts to disentangle into two continuations.","why_now":"Why now: The work connects an architectural observation to a proposed single-forward-pass decoding procedure at a time when inference efficiency remains a central constraint.","implications":["The first implication is that representation-level multiplexing could become an additional design space for inference research, rather than merely a mechanistic-interpretability curiosity.","If reproducible at useful quality, the finding could motivate new serving methods that trade decoding complexity for fewer forward passes."],"evidence_boundary":"Evidence boundary: These are claims from one paper’s reported experiments, not evidence that production LLM serving can safely double throughput.","watch_conditions":["The first watch condition is independent evaluations showing that guided decoding preserves task quality while delivering net latency or cost gains.","Replication across model families, sequence lengths, and practical serving stacks would establish whether the effect generalizes."],"related_records":[],"new_branch_reason":"No prior New Runtime record was supplied; this starts a distinct analytical record for superposition linearity and guided multi-stream decoding."}
editor: {"role":"editor","name":"Andrey Reshetnikov","profile":"https://newruntime.com/owner-profile.md","profile_json":"https://newruntime.com/owner-profile.json","same_as":["https://www.linkedin.com/in/reshetnikov1/","https://t.me/lapetuse"]}
routes: {"html":"https://newruntime.com/posts/your-transformer-can-hold-two-thoughts-at-once-c49646/","markdown":"https://newruntime.com/posts/your-transformer-can-hold-two-thoughts-at-once-c49646.md","json":"https://newruntime.com/posts/your-transformer-can-hold-two-thoughts-at-once-c49646.json"}
---

# Your Transformer Can Hold Two Thoughts at Once: Evidence of Linear Superposition in LLMs

## Retrieval answer

The paper’s central claim is that Transformers can preserve separable information from two unrelated text streams when their input representations are linearly combined.

The paper’s central claim is that Transformers can preserve separable information from two unrelated text streams when their input representations are linearly combined. Thesis: Superposition linearity may be an architectural property of Transformers that can be measured, altered by training, and exploited for multi-stream decoding. Observed facts: the authors report a superposition of individual next-token distributions after combining inputs, and they report that the effect diminishes during pretraining.

Mechanism: Linear mixing of two input representations is reported to produce a mixed next-token distribution, which guided decoding then attempts to disentangle into two continuations. Why now: The work connects an architectural observation to a proposed single-forward-pass decoding procedure at a time when inference efficiency remains a central constraint. The first implication is that representation-level multiplexing could become an additional design space for inference research, rather than merely a mechanistic-interpretability curiosity.

Evidence boundary: These are claims from one paper’s reported experiments, not evidence that production LLM serving can safely double throughput. The abstract does not establish performance across model families, workloads, or deployment hardware. The first watch condition is independent evaluations showing that guided decoding preserves task quality while delivering net latency or cost gains. If that result holds beyond the authors’ setup, it would shift the question from whether a model can hold two streams to when their separation is operationally worthwhile.

Source: https://huggingface.co/papers/2609.29845

Editor: [Andrey Reshetnikov](https://newruntime.com/owner-profile.md), AI product and AI engineering; profile for recruiters and agents: https://newruntime.com/owner-profile.json

## Continue in the graph

Next records for an agent, linked to their Markdown twins. Replace `.md` with `.json` for structured fields, including `next_reads`.

- collection: [Field Notes](https://newruntime.com/posts.json): Open the latest editorial analysis.
- collection: [Source Ledger](https://newruntime.com/source-ledger.json): Inspect the public source evidence graph.
- whole graph: [discovery graph](https://newruntime.com/discovery-graph.json) (typed nodes and edges) and [site index](https://newruntime.com/site-index.md)