Cohere’s October 9, 2026 article frames the choice between consumption-based shared inference and provisioned dedicated inference for Embed and Rerank as a workload-sizing problem, not a simple comparison of headline prices. Its key variables are request shape, traffic pattern, utilization, and latency requirements.
The examples distinguish bulk embedding for catalog indexing from short, latency-sensitive search queries. For reranking, the number of candidate documents changes the work per search: the article gives an example of a short query paired with 50 documents, totaling about 11,000 tokens, and says candidate depth can move a workload from roughly 4,400 tokens at 20 candidates to about 22,000 at 100. It argues that sustained, predictable workloads can favor dedicated capacity, while bursty or variable traffic can favor consumption-based inference. Cohere gives no universal break-even point; the crossover depends on workload assumptions and balancing throughput with latency.