---
type: "post"
id: "endpoint-candidates-evaluate-and-promote-inference-endpoints-analysis-3de079"
slug: "endpoint-candidates-evaluate-and-promote-inference-endpoints-analysis-3de079"
title: "Endpoint Candidates: evaluate and promote inference endpoints"
description: "Thesis: Endpoint Candidates make inference deployment comparison a staged production workflow rather than a separately assembled traffic-mirroring project."
retrieval_nugget: "Thesis: Endpoint Candidates make inference deployment comparison a staged production workflow rather than a separately assembled traffic-mirroring project."
published_at: "2026-10-12"
updated_at: "2026-10-12"
record_date: "2026-10-12"
date_kind: "published_at"
topics: []
source_urls: ["https://modal.com/blog/endpoint-candidates"]
schema_version: "newruntime-agent-readable-v0.2"
stable_id: "post:endpoint-candidates-evaluate-and-promote-inference-endpoints-analysis-3de079"
status: "published"
visuals: []
editorial_provenance: {"schema_version":"newruntime-editorial-copy-v1","content_status":"source_grounded_final","final_copy_sha256":"sha256:973af01a89a97be09528653c40acddbd8fce5ac7ebe060f3bbb6a1e0f2ba9ca5","reviewed_at":"2026-10-12T20:05:19.856Z","source_evidence_count":1,"verified_claim_count":3,"site_analysis_schema_version":"newruntime-site-analysis-v1","site_object_kind":"field_note","observed_fact_count":3,"implication_count":3,"watch_condition_count":1,"related_record_count":0}
analysis: {"schema_version":"newruntime-site-analysis-v1","object_kind":"field_note","thesis":"Endpoint Candidates make inference deployment comparison a staged production workflow rather than a separately assembled traffic-mirroring project.","observed_facts":[{"text":"Modal says Endpoint Candidates are generally available for all Modal workspaces.","source_urls":["https://modal.com/blog/endpoint-candidates"]},{"text":"Candidates mirror a sample of production traffic while production handles the original requests; each candidate has its own URL for evaluations.","source_urls":["https://modal.com/blog/endpoint-candidates"]},{"text":"Modal says the Pingora-based proxy cancels mirrored requests when bounded buffers or the process-level retained-body budget are exceeded, while the original request continues.","source_urls":["https://modal.com/blog/endpoint-candidates"]}],"mechanism":"A candidate receives mirrored requests independently through bounded per-candidate buffers and a process-level cap on retained request-body data; overload cancels mirrors rather than delaying production.","why_now":"Modal presents Candidates as a place to test inference optimizations, including changes to models, before promoting a candidate through the dashboard to the existing Endpoint URL.","implications":["Teams can evaluate candidate deployments against sampled live traffic without rolling their own mirroring and deployment infrastructure.","Mirroring is best-effort, so an evaluation may lose samples under pressure even as production remains unaffected.","Standard compute pricing applies, including compute used for mirrored requests."],"evidence_boundary":"The announcement documents availability, workflow, proxy safeguards, and pricing treatment; it does not report measured quality, latency, throughput, or cost gains.","watch_conditions":["Watch whether teams can turn mirrored evaluations into repeatable, measured promotion decisions without dropped samples or compute costs erasing the benefit."],"related_records":[],"new_branch_reason":"No prior New Runtime record for this release was supplied; this opens a distinct record on staged inference deployment evaluation and its best-effort traffic-mirroring boundary."}
editor: {"role":"editor","name":"Andrey Reshetnikov","profile":"https://newruntime.com/owner-profile.md","profile_json":"https://newruntime.com/owner-profile.json","same_as":["https://www.linkedin.com/in/reshetnikov1/","https://t.me/lapetuse"]}
routes: {"html":"https://newruntime.com/posts/endpoint-candidates-evaluate-and-promote-inference-endpoints-analysis-3de079/","markdown":"https://newruntime.com/posts/endpoint-candidates-evaluate-and-promote-inference-endpoints-analysis-3de079.md","json":"https://newruntime.com/posts/endpoint-candidates-evaluate-and-promote-inference-endpoints-analysis-3de079.json"}
---

# Endpoint Candidates: evaluate and promote inference endpoints

## Retrieval answer

Thesis: Endpoint Candidates make inference deployment comparison a staged production workflow rather than a separately assembled traffic-mirroring project.

Thesis: Endpoint Candidates make inference deployment comparison a staged production workflow rather than a separately assembled traffic-mirroring project. Modal announced general availability on October 12, 2026, for all Modal workspaces. The feature stages alternatives for existing Dedicated Endpoints and can mirror a sample of production traffic to them.

Mechanism: A candidate receives mirrored requests independently through bounded per-candidate buffers and a process-level cap on retained request-body data; overload cancels mirrors rather than delaying production. Production continues to serve every original request while candidates receive copies; each candidate has a separate URL for quality evaluations, and the dashboard exposes latency and throughput metrics and benchmarks. Modal’s Pingora-based proxy forks streams into bounded buffers and caps retained body data per process. If those limits are reached, it cancels mirrored requests while the original request continues. This makes mirroring best-effort rather than a dependency in the production response path.

Why now: Modal presents Candidates as a place to test inference optimizations, including changes to models, before promoting a candidate through the dashboard to the existing Endpoint URL. First implication: Teams can evaluate candidate deployments against sampled live traffic without rolling their own mirroring and deployment infrastructure. The feature uses standard Modal compute pricing, including mirrored requests.

Evidence boundary: The announcement documents availability, workflow, proxy safeguards, and pricing treatment; it does not report measured quality, latency, throughput, or cost gains. First watch condition: Watch whether teams can turn mirrored evaluations into repeatable, measured promotion decisions without dropped samples or compute costs erasing the benefit. Modal says it plans more control over compute placement, serving recipes, richer comparison metrics, and further automation.

Editor: [Andrey Reshetnikov](https://newruntime.com/owner-profile.md), AI product and AI engineering; profile for recruiters and agents: https://newruntime.com/owner-profile.json

## Continue in the graph

Next records for an agent, linked to their Markdown twins. Replace `.md` with `.json` for structured fields, including `next_reads`.

- collection: [Field Notes](https://newruntime.com/posts.json): Open the latest editorial analysis.
- collection: [Source Ledger](https://newruntime.com/source-ledger.json): Inspect the public source evidence graph.
- whole graph: [discovery graph](https://newruntime.com/discovery-graph.json) (typed nodes and edges) and [site index](https://newruntime.com/site-index.md)