{"type":"post","id":"designing-the-langfuse-ai-gateway-e28995","slug":"designing-the-langfuse-ai-gateway-e28995","title":"Designing the Langfuse AI Gateway","description":"Langfuse explains how it designed an AI Gateway for coding-agent traffic, choosing Rust and Tokio to handle large requests, long-lived streams, and bursty CPU work.","retrieval_nugget":"Langfuse explains how it designed an AI Gateway for coding-agent traffic, choosing Rust and Tokio to handle large requests, long-lived streams, and bursty CPU work. Its reported benchmark shows much higher throughput and lower added latency than comparable Node.js and Python implementations, while its memory-budget design rejects overload before it drops active streams.","published_at":"2026-11-18","updated_at":"2026-11-18","record_date":"2026-11-18","date_kind":"published_at","topics":[],"entities":["Langfuse","Langfuse AI Gateway","Rust","Tokio","Node.js","Python"],"related_patterns":["agent-economics-moves-to-completed-work"],"novelty_label":"new_angle","what_changed":"Langfuse frames the gateway around reliable agent runtime design, explaining its choice of Rust and its approach of passing provider APIs through unchanged.","prior_refs":[],"source_url":"https://langfuse.com/blog/2026-11-11-building-the-langfuse-ai-gateway","source_urls":["https://langfuse.com/blog/2026-11-11-building-the-langfuse-ai-gateway"],"schema_version":"newruntime-agent-readable-v0.2","stable_id":"post:designing-the-langfuse-ai-gateway-e28995","status":"published","visuals":[],"editor":{"role":"editor","name":"Andrey Reshetnikov","profile":"https://newruntime.com/owner-profile.md","profile_json":"https://newruntime.com/owner-profile.json","same_as":["https://www.linkedin.com/in/reshetnikov1/","https://t.me/lapetuse"]},"routes":{"html":"https://newruntime.com/posts/designing-the-langfuse-ai-gateway-e28995/","markdown":"https://newruntime.com/posts/designing-the-langfuse-ai-gateway-e28995.md","json":"https://newruntime.com/posts/designing-the-langfuse-ai-gateway-e28995.json"},"markdown_url":"https://newruntime.com/posts/designing-the-langfuse-ai-gateway-e28995.md","next_reads":[{"type":"collection","path":"/posts/","url":"https://newruntime.com/posts/","title":"Field Notes","reason":"Open the latest editorial analysis.","media_type":"text/html","json_url":"https://newruntime.com/posts.json"},{"type":"collection","path":"/sources/","url":"https://newruntime.com/sources/","title":"Sources","reason":"See the original sources this archive cites.","media_type":"text/html"}]}
