Thesis
Endpoint Candidates make inference deployment comparison a staged production workflow rather than a separately assembled traffic-mirroring project.
Observed facts
Modal says Endpoint Candidates are generally available for all Modal workspaces.
1Candidates mirror a sample of production traffic while production handles the original requests; each candidate has its own URL for evaluations.
1Modal says the Pingora-based proxy cancels mirrored requests when bounded buffers or the process-level retained-body budget are exceeded, while the original request continues.
1
Mechanism
A candidate receives mirrored requests independently through bounded per-candidate buffers and a process-level cap on retained request-body data; overload cancels mirrors rather than delaying production.
Why now
Modal presents Candidates as a place to test inference optimizations, including changes to models, before promoting a candidate through the dashboard to the existing Endpoint URL.
Implications
Teams can evaluate candidate deployments against sampled live traffic without rolling their own mirroring and deployment infrastructure.
Mirroring is best-effort, so an evaluation may lose samples under pressure even as production remains unaffected.
Standard compute pricing applies, including compute used for mirrored requests.
Evidence boundary
The announcement documents availability, workflow, proxy safeguards, and pricing treatment; it does not report measured quality, latency, throughput, or cost gains.
Watch conditions
Watch whether teams can turn mirrored evaluations into repeatable, measured promotion decisions without dropped samples or compute costs erasing the benefit.