---
id: "nr-researchers-release-recurrent-looped-transformer-with-hidden-state-feedback-cd2386"
slug: "researchers-release-recurrent-looped-transformer-with-hidden-state-feedback-cd2386"
title: "Researchers release Recurrent Looped Transformer with hidden-state feedback across tokens"
observed_at: "2026-09-18T13:40:45.45798+00:00"
record_date: "2026-09-18"
date_kind: "observed_at"
novelty: "new"
source_platform: "github.com"
topics: ["model-architecture","research"]
entities: ["Recurrent Looped Transformer"]
related_patterns: []
source_url: "https://github.com/yifanzhang-pro/recurrent-looped-tranformer"
source_urls: ["https://github.com/yifanzhang-pro/recurrent-looped-tranformer"]
basket: {"id":"0325e6b8-f527-4f9b-bfa3-3f30e57f380b","revision":1,"cluster_id":"cd2386ec-6891-481a-bf08-cf722355dbfe"}
type: "post"
description: "A new report describes the Recurrent Looped Transformer, which feeds decoder state to the next token, and shows faster parity learning than a matched Transformer in small experiments."
why_it_matters: "The results come from models of roughly 25 to 29 million parameters trained on six algorithmic tasks, so they do not establish gains for language modeling or at larger scale. The chunked RLT-2 variant is only proposed, with no training or throughput results. Architecture researchers can use the matched Transformer baseline and three-seed reporting to check whether the parity advantage survives replication. Replications on language modeling tasks, and any throughput results for the proposed chunked RLT-2 variant, would show whether the design holds up."
verification_level: "source-inspected"
signal_type: "field_note"
schema_version: "newruntime-agent-readable-v0.2"
stable_id: "post:researchers-release-recurrent-looped-transformer-with-hidden-state-feedback-cd2386"
retrieval_nugget: "A new report describes the Recurrent Looped Transformer, which feeds decoder state to the next token, and shows faster parity learning than a matched Transformer in small experiments. Yifan Zhang, Jichen Feng and Shihan Qin released a report and Apache-2.0 repository for the Recurrent Looped Transformer, dated September 12, 2026 and updated September 17. The decoder reads encoder-derived global KV"
status: "published"
visuals: []
editorial_provenance: {"schema_version":"newruntime-editorial-copy-v1","content_status":"source_grounded_final","final_copy_sha256":"sha256:80a40503b373691c8555cb345f578fea4a19f51779c2ed725d8aa8005c208e21","reviewed_at":"2026-09-24T13:28:46Z","source_evidence_count":1,"verified_claim_count":2,"site_analysis_schema_version":"newruntime-site-analysis-v1","site_object_kind":"field_note","observed_fact_count":3,"implication_count":1,"watch_condition_count":1,"related_record_count":0}
analysis: {"schema_version":"newruntime-site-analysis-v1","object_kind":"field_note","thesis":"The Recurrent Looped Transformer adds depth through recurrence across tokens rather than by stacking more layers per token.","observed_facts":[{"text":"RLT-1 passes the decoder's final hidden state to the next token with that token's causal encoder representation.","source_urls":["https://github.com/yifanzhang-pro/recurrent-looped-tranformer"]},{"text":"At step 500, RLT-1 6+2 reached 99.44 ± 0.98% parity accuracy versus 48.48 ± 0.53% for Transformer 8.","source_urls":["https://github.com/yifanzhang-pro/recurrent-looped-tranformer"]},{"text":"RLT-1 has 26.10 to 28.73M parameters, while Transformer 8 has 25.31M.","source_urls":["https://github.com/yifanzhang-pro/recurrent-looped-tranformer"]}],"mechanism":"RLT-1 passes the decoder's final hidden state to the next token together with that token's causal encoder representation, so the recurrent path grows with sequence length while the blocks evaluated per token stay fixed.","why_now":"Looped and recurrent designs are being revisited as a way to get more computation per parameter without widening or deepening the model.","implications":["Architecture researchers can use the matched Transformer baseline and three-seed reporting to check whether the parity advantage survives replication."],"evidence_boundary":"The results come from models of roughly 25 to 29 million parameters trained on six algorithmic tasks, so they do not establish gains for language modeling or at larger scale.","watch_conditions":["Replications on language modeling tasks, and any throughput results for the proposed chunked RLT-2 variant, would show whether the design holds up."],"related_records":[],"new_branch_reason":"The report offers a concrete recurrent architecture with matched ablations, distinct from general looped-transformer commentary."}
routes: {"html":"https://newruntime.com/posts/researchers-release-recurrent-looped-transformer-with-hidden-state-feedback-cd2386/","markdown":"https://newruntime.com/posts/researchers-release-recurrent-looped-transformer-with-hidden-state-feedback-cd2386.md","json":"https://newruntime.com/posts/researchers-release-recurrent-looped-transformer-with-hidden-state-feedback-cd2386.json"}
editor: {"role":"editor","name":"Andrey Reshetnikov","profile":"https://newruntime.com/owner-profile.md","profile_json":"https://newruntime.com/owner-profile.json","same_as":["https://www.linkedin.com/in/reshetnikov1/","https://t.me/lapetuse"]}
---

# Researchers release Recurrent Looped Transformer with hidden-state feedback across tokens

## Retrieval answer

A new report describes the Recurrent Looped Transformer, which feeds decoder state to the next token, and shows faster parity learning than a matched Transformer in small experiments. Yifan Zhang, Jichen Feng and Shihan Qin released a report and Apache-2.0 repository for the Recurrent Looped Transformer, dated September 12, 2026 and updated September 17. The decoder reads encoder-derived global KV

Yifan Zhang, Jichen Feng and Shihan Qin released a report and Apache-2.0 repository for the Recurrent Looped Transformer, dated September 12, 2026 and updated September 17. The decoder reads encoder-derived global KV memory and keeps a sliding-window attention cache at every layer.

The Recurrent Looped Transformer adds depth through recurrence across tokens rather than by stacking more layers per token. RLT-1 passes the decoder's final hidden state to the next token together with that token's causal encoder representation, so the recurrent path grows with sequence length while the blocks evaluated per token stay fixed. Looped and recurrent designs are being revisited as a way to get more computation per parameter without widening or deepening the model.

The September 17 snapshot covers 108 runs of 2,000 optimizer steps across six algorithmic tasks and three seeds. At step 500, RLT-1 6+2 reached 99.44 ± 0.98% parity accuracy, compared with 48.48 ± 0.53% for an eight-layer Transformer. On standard S5, every model stayed below 3%.

The results come from models of roughly 25 to 29 million parameters trained on six algorithmic tasks, so they do not establish gains for language modeling or at larger scale. The chunked RLT-2 variant is only proposed, with no training or throughput results. Architecture researchers can use the matched Transformer baseline and three-seed reporting to check whether the parity advantage survives replication. Replications on language modeling tasks, and any throughput results for the proposed chunked RLT-2 variant, would show whether the design holds up.

Editor: [Andrey Reshetnikov](https://newruntime.com/owner-profile.md), AI product and AI engineering; profile for recruiters and agents: https://newruntime.com/owner-profile.json

## Continue in the graph

Next records for an agent, linked to their Markdown twins. Replace `.md` with `.json` for structured fields, including `next_reads`.

- topic hub: [Model Architecture](https://newruntime.com/topics/model-architecture.md): Explore the model-architecture topic hub.
- topic hub: [Research](https://newruntime.com/topics/research.md): Explore the research topic hub.
- collection: [Field Notes](https://newruntime.com/posts.json): Open the latest editorial analysis.
- collection: [Source Ledger](https://newruntime.com/source-ledger.json): Inspect the public source evidence graph.
- whole graph: [discovery graph](https://newruntime.com/discovery-graph.json) (typed nodes and edges) and [site index](https://newruntime.com/site-index.md)