{"type":"analysis","id":"analysis:research-llm-hidden-states-can-be-swapped-for-symbolic-tensor-product-formulas-with","slug":"research-llm-hidden-states-can-be-swapped-for-symbolic-tensor-product-formulas-with","title":"LLM hidden states can be swapped for symbolic tensor-product formulas with little loss of behavior, but this is an interpretability result,","description":"McCoy, Soulos, Linzen and Smolensky report that a closed-form tensor product representation (TPR) can stand in for GPT-OSS-20b's input-token representations at a cost of at most 2.36 points of accuracy, and that edits made through that formula steer the model about 90% of the time. The result is strong evidence that neural networks learn implicit structure, but it does not show a market or archite","published_at":"2026-10-23T05:00:00.000Z","updated_at":"2026-10-23T05:00:00.000Z","record_date":"2026-10-23","topics":["architecture","tools","research"],"source_urls":["https://arxiv.org/html/2608.29530v1","https://github.com/tommccoy1/discover","https://arxiv.org/abs/1812.08718","https://arxiv.org/abs/2605.09967","https://arxiv.org/abs/2608.29530"],"language":"en","routes":{"html":"https://newruntime.com/analysis/research-llm-hidden-states-can-be-swapped-for-symbolic-tensor-product-formulas-with/","markdown":"https://newruntime.com/analysis/research-llm-hidden-states-can-be-swapped-for-symbolic-tensor-product-formulas-with.md","json":"https://newruntime.com/analysis/research-llm-hidden-states-can-be-swapped-for-symbolic-tensor-product-formulas-with.json"},"author":{"role":"author","name":"Andrey Reshetnikov","profile":"https://newruntime.com/owner-profile.md","profile_json":"https://newruntime.com/owner-profile.json","same_as":["https://www.linkedin.com/in/reshetnikov1/","https://t.me/lapetuse"]},"editorial_provenance":{"desk":"authorial","target_ref":"deep_research:5b9e17ce-bfaf-4431-a36f-767e761577bf","authorial_article_id":"5b9e17ce-bfaf-4431-a36f-767e761577bf"},"next_reads":[{"type":"topic","path":"/topics/architecture/","url":"https://newruntime.com/topics/architecture/","title":"Architecture","reason":"Explore the architecture topic hub.","media_type":"text/html","markdown_url":"https://newruntime.com/topics/architecture.md","json_url":"https://newruntime.com/topics/architecture.json"},{"type":"topic","path":"/topics/tools/","url":"https://newruntime.com/topics/tools/","title":"Tools","reason":"Explore the tools topic hub.","media_type":"text/html","markdown_url":"https://newruntime.com/topics/tools.md","json_url":"https://newruntime.com/topics/tools.json"},{"type":"topic","path":"/topics/research/","url":"https://newruntime.com/topics/research/","title":"Research","reason":"Explore the research topic hub.","media_type":"text/html","markdown_url":"https://newruntime.com/topics/research.md","json_url":"https://newruntime.com/topics/research.json"},{"type":"collection","path":"/analysis/","url":"https://newruntime.com/analysis/","title":"Analysis","reason":"Open the latest authorial analysis.","media_type":"text/html","json_url":"https://newruntime.com/analysis.json"},{"type":"collection","path":"/source-ledger.json","url":"https://newruntime.com/source-ledger.json","title":"Source Ledger","reason":"Inspect the public source evidence graph.","media_type":"application/json"}]}