{"type":"post","id":"blind-german-writing-test-kolibri-loses-all-136-e80a2a","slug":"blind-german-writing-test-kolibri-loses-all-136-e80a2a","title":"Blind German-writing test: Kolibri loses all 136 comparisons to Claude Sonnet 5.5","description":"A practitioner’s blind test found Aleph Alpha’s Kolibri far behind Claude Sonnet 5.5 on a narrowly defined German-writing workload: 0 wins in 136 pairwise verdicts and lower judge scores.","retrieval_nugget":"A practitioner’s blind test found Aleph Alpha’s Kolibri far behind Claude Sonnet 5.5 on a narrowly defined German-writing workload: 0 wins in 136 pairwise verdicts and lower judge scores. The more durable lesson for AI builders is methodological: a model substitution needs task-level acceptance criteria, controlled inputs, and inspection beyond a headline benchmark.","published_at":"2026-10-28","updated_at":"2026-10-28","record_date":"2026-10-28","date_kind":"published_at","topics":["claude","gemini"],"entities":["Aleph Alpha","Kolibri","Claude Sonnet 5.5","Dewfall","GPT-5.5","Gemini 3.1 Pro","Hugging Face","OpenRouter"],"related_patterns":["verification-bandwidth-is-the-scarce-resource"],"novelty_label":"new_fact","what_changed":"A follow-up test found Kolibri lost all 136 blind German Bible-writing comparisons to Claude Sonnet 5.5, with lower judge scores and more serious errors.","prior_refs":["memory:0e621bcc-09e8-8c7c-a733-91d142f500c3"],"source_url":"https://tej.as/blog/kolibri-vs-claude-german-llm-benchmark","source_urls":["https://tej.as/blog/kolibri-vs-claude-german-llm-benchmark"],"schema_version":"newruntime-agent-readable-v0.2","stable_id":"post:blind-german-writing-test-kolibri-loses-all-136-e80a2a","status":"published","visuals":[],"editor":{"role":"editor","name":"Andrey Reshetnikov","profile":"https://newruntime.com/owner-profile.md","profile_json":"https://newruntime.com/owner-profile.json","same_as":["https://www.linkedin.com/in/reshetnikov1/","https://t.me/lapetuse"]},"routes":{"html":"https://newruntime.com/posts/blind-german-writing-test-kolibri-loses-all-136-e80a2a/","markdown":"https://newruntime.com/posts/blind-german-writing-test-kolibri-loses-all-136-e80a2a.md","json":"https://newruntime.com/posts/blind-german-writing-test-kolibri-loses-all-136-e80a2a.json"},"next_reads":[{"type":"topic","path":"/topics/claude/","url":"https://newruntime.com/topics/claude/","title":"Claude","reason":"Explore the claude topic hub.","media_type":"text/html","markdown_url":"https://newruntime.com/topics/claude.md","json_url":"https://newruntime.com/topics/claude.json"},{"type":"topic","path":"/topics/gemini/","url":"https://newruntime.com/topics/gemini/","title":"Gemini","reason":"Explore the gemini topic hub.","media_type":"text/html","markdown_url":"https://newruntime.com/topics/gemini.md","json_url":"https://newruntime.com/topics/gemini.json"},{"type":"collection","path":"/posts/","url":"https://newruntime.com/posts/","title":"Field Notes","reason":"Open the latest editorial analysis.","media_type":"text/html","json_url":"https://newruntime.com/posts.json"},{"type":"collection","path":"/source-ledger.json","url":"https://newruntime.com/source-ledger.json","title":"Source Ledger","reason":"Inspect the public source evidence graph.","media_type":"application/json"}]}