{"type":"post","id":"steal-our-eval-setup-for-chat-agents-c59267","slug":"steal-our-eval-setup-for-chat-agents-c59267","title":"Steal our eval setup for chat agents","description":"Langfuse has published the evaluation workflow behind its documentation chatbot, showing how production traces can become reviewed cases, datasets and experiments.","retrieval_nugget":"Langfuse has published the evaluation workflow behind its documentation chatbot, showing how production traces can become reviewed cases, datasets and experiments. For teams building tool-using assistants, the useful point is that evaluation need not start from scratch: a working loop can be adapted, while app-specific tests remain necessary.","published_at":"2026-11-03","updated_at":"2026-11-03","record_date":"2026-11-03","date_kind":"published_at","topics":["mcp"],"entities":["Langfuse","Vercel AI SDK","Inkeep","Langfuse Docs MCP","OpenAI"],"related_patterns":["verification-bandwidth-is-the-scarce-resource"],"novelty_label":"new_fact","what_changed":"The post adds Langfuse’s specific docs-agent improvement workflow: capture production conversation signals, turn them into repeatable cases, then evaluate them.","prior_refs":[],"source_url":"https://langfuse.com/blog/2026-07-16-steal-our-eval-setup","source_urls":["https://langfuse.com/blog/2026-07-16-steal-our-eval-setup"],"schema_version":"newruntime-agent-readable-v0.2","stable_id":"post:steal-our-eval-setup-for-chat-agents-c59267","status":"published","visuals":[],"editor":{"role":"editor","name":"Andrey Reshetnikov","profile":"https://newruntime.com/owner-profile.md","profile_json":"https://newruntime.com/owner-profile.json","same_as":["https://www.linkedin.com/in/reshetnikov1/","https://t.me/lapetuse"]},"routes":{"html":"https://newruntime.com/posts/steal-our-eval-setup-for-chat-agents-c59267/","markdown":"https://newruntime.com/posts/steal-our-eval-setup-for-chat-agents-c59267.md","json":"https://newruntime.com/posts/steal-our-eval-setup-for-chat-agents-c59267.json"},"markdown_url":"https://newruntime.com/posts/steal-our-eval-setup-for-chat-agents-c59267.md","next_reads":[{"type":"topic","path":"/topics/mcp/","url":"https://newruntime.com/topics/mcp/","title":"Mcp","reason":"Explore the mcp topic hub.","media_type":"text/html","markdown_url":"https://newruntime.com/topics/mcp.md","json_url":"https://newruntime.com/topics/mcp.json"},{"type":"collection","path":"/posts/","url":"https://newruntime.com/posts/","title":"Field Notes","reason":"Open the latest editorial analysis.","media_type":"text/html","json_url":"https://newruntime.com/posts.json"},{"type":"collection","path":"/sources/","url":"https://newruntime.com/sources/","title":"Sources","reason":"See the original sources this archive cites.","media_type":"text/html"}]}