{"type":"analysis","id":"analysis:research-openai-shows-public-chat-data-can-track-production-misbehavior-rates-and-that","slug":"research-openai-shows-public-chat-data-can-track-production-misbehavior-rates-and-that","title":"OpenAI shows public chat data can track production misbehavior rates and that trait-targeted RL generalizes, but only on its own models and","description":"In June 2026 OpenAI published two alignment research posts. One finds that replaying public WildChat prompts predicts real-world chat failure rates within a few-fold. The other finds that mixing 5% \"beneficial trait\" data into RL improves 44 of 53 out-of-distribution alignment evals. Both are lab-internal results with no released data or code. Neither holds up well yet for agentic use.","published_at":"2026-10-24T22:00:00.000Z","updated_at":"2026-10-24T22:00:00.000Z","record_date":"2026-10-24","topics":["evals","openai","design","research"],"source_urls":["https://alignment.openai.com/validating-public-evals","https://arxiv.org/abs/2607.07184","https://alignment.openai.com/beneficial-rl","https://arxiv.org/html/2606.24014","https://arxiv.org/abs/2606.24014"],"language":"en","routes":{"html":"https://newruntime.com/analysis/research-openai-shows-public-chat-data-can-track-production-misbehavior-rates-and-that/","markdown":"https://newruntime.com/analysis/research-openai-shows-public-chat-data-can-track-production-misbehavior-rates-and-that.md","json":"https://newruntime.com/analysis/research-openai-shows-public-chat-data-can-track-production-misbehavior-rates-and-that.json"},"author":{"role":"author","name":"Andrey Reshetnikov","profile":"https://newruntime.com/owner-profile.md","profile_json":"https://newruntime.com/owner-profile.json","same_as":["https://www.linkedin.com/in/reshetnikov1/","https://t.me/lapetuse"]},"editorial_provenance":{"desk":"authorial","target_ref":"deep_research:0b92d474-e411-4809-bdcc-20013794480d","authorial_article_id":"0b92d474-e411-4809-bdcc-20013794480d"},"next_reads":[{"type":"topic","path":"/topics/evals/","url":"https://newruntime.com/topics/evals/","title":"Evals","reason":"Explore the evals topic hub.","media_type":"text/html","markdown_url":"https://newruntime.com/topics/evals.md","json_url":"https://newruntime.com/topics/evals.json"},{"type":"topic","path":"/topics/openai/","url":"https://newruntime.com/topics/openai/","title":"Openai","reason":"Explore the openai topic hub.","media_type":"text/html","markdown_url":"https://newruntime.com/topics/openai.md","json_url":"https://newruntime.com/topics/openai.json"},{"type":"topic","path":"/topics/design/","url":"https://newruntime.com/topics/design/","title":"Design","reason":"Explore the design topic hub.","media_type":"text/html","markdown_url":"https://newruntime.com/topics/design.md","json_url":"https://newruntime.com/topics/design.json"},{"type":"collection","path":"/analysis/","url":"https://newruntime.com/analysis/","title":"Analysis","reason":"Open the latest authorial analysis.","media_type":"text/html","json_url":"https://newruntime.com/analysis.json"},{"type":"collection","path":"/source-ledger.json","url":"https://newruntime.com/source-ledger.json","title":"Source Ledger","reason":"Inspect the public source evidence graph.","media_type":"application/json"}]}