{"type":"post","id":"endpoint-candidates-evaluate-and-promote-inference-endpoints-analysis-3de079","slug":"endpoint-candidates-evaluate-and-promote-inference-endpoints-analysis-3de079","title":"Endpoint Candidates: evaluate and promote inference endpoints","description":"Thesis: Endpoint Candidates make inference deployment comparison a staged production workflow rather than a separately assembled traffic-mirroring project.","retrieval_nugget":"Thesis: Endpoint Candidates make inference deployment comparison a staged production workflow rather than a separately assembled traffic-mirroring project.","published_at":"2026-10-12","updated_at":"2026-10-12","record_date":"2026-10-12","date_kind":"published_at","topics":[],"source_urls":["https://modal.com/blog/endpoint-candidates"],"schema_version":"newruntime-agent-readable-v0.2","stable_id":"post:endpoint-candidates-evaluate-and-promote-inference-endpoints-analysis-3de079","status":"published","visuals":[],"editorial_provenance":{"schema_version":"newruntime-editorial-copy-v1","content_status":"source_grounded_final","final_copy_sha256":"sha256:973af01a89a97be09528653c40acddbd8fce5ac7ebe060f3bbb6a1e0f2ba9ca5","reviewed_at":"2026-10-12T20:05:19.856Z","source_evidence_count":1,"verified_claim_count":3,"site_analysis_schema_version":"newruntime-site-analysis-v1","site_object_kind":"field_note","observed_fact_count":3,"implication_count":3,"watch_condition_count":1,"related_record_count":0},"analysis":{"schema_version":"newruntime-site-analysis-v1","object_kind":"field_note","thesis":"Endpoint Candidates make inference deployment comparison a staged production workflow rather than a separately assembled traffic-mirroring project.","observed_facts":[{"text":"Modal says Endpoint Candidates are generally available for all Modal workspaces.","source_urls":["https://modal.com/blog/endpoint-candidates"]},{"text":"Candidates mirror a sample of production traffic while production handles the original requests; each candidate has its own URL for evaluations.","source_urls":["https://modal.com/blog/endpoint-candidates"]},{"text":"Modal says the Pingora-based proxy cancels mirrored requests when bounded buffers or the process-level retained-body budget are exceeded, while the original request continues.","source_urls":["https://modal.com/blog/endpoint-candidates"]}],"mechanism":"A candidate receives mirrored requests independently through bounded per-candidate buffers and a process-level cap on retained request-body data; overload cancels mirrors rather than delaying production.","why_now":"Modal presents Candidates as a place to test inference optimizations, including changes to models, before promoting a candidate through the dashboard to the existing Endpoint URL.","implications":["Teams can evaluate candidate deployments against sampled live traffic without rolling their own mirroring and deployment infrastructure.","Mirroring is best-effort, so an evaluation may lose samples under pressure even as production remains unaffected.","Standard compute pricing applies, including compute used for mirrored requests."],"evidence_boundary":"The announcement documents availability, workflow, proxy safeguards, and pricing treatment; it does not report measured quality, latency, throughput, or cost gains.","watch_conditions":["Watch whether teams can turn mirrored evaluations into repeatable, measured promotion decisions without dropped samples or compute costs erasing the benefit."],"related_records":[],"new_branch_reason":"No prior New Runtime record for this release was supplied; this opens a distinct record on staged inference deployment evaluation and its best-effort traffic-mirroring boundary."},"editor":{"role":"editor","name":"Andrey Reshetnikov","profile":"https://newruntime.com/owner-profile.md","profile_json":"https://newruntime.com/owner-profile.json","same_as":["https://www.linkedin.com/in/reshetnikov1/","https://t.me/lapetuse"]},"routes":{"html":"https://newruntime.com/posts/endpoint-candidates-evaluate-and-promote-inference-endpoints-analysis-3de079/","markdown":"https://newruntime.com/posts/endpoint-candidates-evaluate-and-promote-inference-endpoints-analysis-3de079.md","json":"https://newruntime.com/posts/endpoint-candidates-evaluate-and-promote-inference-endpoints-analysis-3de079.json"},"next_reads":[{"type":"collection","path":"/posts/","url":"https://newruntime.com/posts/","title":"Field Notes","reason":"Open the latest editorial analysis.","media_type":"text/html","json_url":"https://newruntime.com/posts.json"},{"type":"collection","path":"/source-ledger.json","url":"https://newruntime.com/source-ledger.json","title":"Source Ledger","reason":"Inspect the public source evidence graph.","media_type":"application/json"}]}