---
type: analysis
slug: rerank-5-costs-about-a-tenth-of-a-cent-per-query-and-our-bet-is-that-cohere
title: "Rerank 5 costs about a tenth of a cent per query, and our bet is that Cohere earns its margin in Vault, not the API"
description: "Rerank 5 lists at $0.02 to $0.05 per million tokens. Cohere sells it through the per-token API and through dedicated Model Vault deployments, and NVIDIA says Cohere plans to add a confidential bare-metal option to Vault. We think the money is in the dedicated tier. That is our hypothesis, not anything Cohere has said, and there is a simpler explanation that fits the evidence just as well."
published_at: 2026-10-23T14:00:00.000Z
topics: ["trends"]
source_urls: ["https://cohere.com/blog/rerank-5","https://blogs.nvidia.com/blog/confidential-computing-model-builders-infrastructure","https://cohere.com/blog/rethinking-confidence-in-reasoning-models","https://t.me/qwgai/1521"]
author: "Andrey Reshetnikov"
author_profile: https://newruntime.com/owner-profile.md
---
# Rerank 5 costs about a tenth of a cent per query, and our bet is that Cohere earns its margin in Vault, not the API

Rerank 5 lists at $0.02 to $0.05 per million tokens. Cohere sells it through the per-token API and through dedicated Model Vault deployments, and NVIDIA says Cohere plans to add a confidential bare-metal option to Vault. We think the money is in the dedicated tier. That is our hypothesis, not anything Cohere has said, and there is a simpler explanation that fits the evidence just as well.

## Summary

Cohere released Rerank 5 in two tiers: Pro at $0.05 per million tokens and Fast at $0.02. It also added Context Focus, an option that returns the relevant passages inside each document in the same rerank call. The models are available through the per-token API and through Model Vault, which Cohere describes as a dedicated deployment with no shared resources. NVIDIA separately said Cohere plans to integrate Confidential Bare Metal into Vault. Our hypothesis is that per-token reranking makes very little money and the dedicated, isolated tier carries the margin. The competing explanation is that open-weight rerankers force the low price and Vault simply serves the customers who need isolation.

## What happened

Rerank 5 is generally available through the Cohere API, Model Vault, Microsoft Foundry, Amazon SageMaker and Cohere North. Cohere says Fast handles about 187 documents of 1,000 tokens per second on one NVIDIA A100 under its test conditions, 2.9 times Pro's throughput. Cohere also says Pro ranked first on 39 of 46 composite datasets and scored 82.2 on BRIGHT. All of these are Cohere's own numbers. Context Focus is optional. Instead of only ordering documents or chunks, it returns the passages within them that matter, so the downstream generator receives less text. In a separate announcement, NVIDIA introduced Confidential Bare Metal next to its confidential containers and VMs. It said confidential-computing support is extending to Grace Blackwell NVL72 and Rubin systems, and that Cohere plans to integrate the option into Model Vault. NVIDIA gave no availability date for the Cohere integration, and nobody outside NVIDIA has verified the security properties. Cohere also published unrelated research, CALIBER, on training reasoning models to report separate confidence estimates before and after reasoning. Those results come from benchmarks only, and this take does not rely on them.

## What the archive adds

According to the dossier, Cohere's pricing page (d3) describes Model Vault as a dedicated, fully managed deployment with no shared resources or multi-tenancy overhead, and positions North as the enterprise platform for search and agents. Rerank 5 therefore reaches customers in two ways: metered API calls, and dedicated capacity that the customer reserves whether or not it is busy. Our evidence does not include Vault's prices. Earlier Cohere posts in our archive on North 2 (d1) and on managing agent sprawl (d2) show the company has been selling governed enterprise deployments for some time, so a confidential Vault tier would extend an existing line rather than open a new one. A practitioner RAG guide in our archive, written in Russian, describes retrieve, rerank, generate as the baseline design. It treats cross-encoder reranking as a standard stage and argues that most of the quality work happens earlier, in corpus preparation, metadata, chunking and hybrid search. That is useful context: rerankers are an expected part of the stack, not a feature that sets one vendor apart.

## Trend signal

The narrow pattern supported by the evidence is that one model now ships through several channels with different billing. Rerank 5 is available on Cohere's API, two hyperscaler marketplaces, Cohere's own dedicated Vault and North (primary, d3), and a confidential hardware option is announced for Vault. NVIDIA's announcement shows the infrastructure side of the same shift. It is pushing hardware-rooted isolation and attestation down to bare metal on its newest systems so that model builders can run on third-party hardware without giving up control of their weights or customer data. Whether this amounts to an industry-wide move of margin from tokens to deployment cannot be shown from three vendor posts. What we can say is that Cohere is now positioned to charge for the deployment, not only for the call.

## Why now

1) List prices of $0.02 and $0.05 per million tokens (primary) mean reranking costs fractions of a cent per query, so API revenue scales slowly. 2) Cohere already sells Model Vault as a dedicated deployment with no multi-tenancy (d3) and ships Rerank 5 there (primary), which gives it a capacity-based channel next to the per-token API. 3) NVIDIA is extending confidential computing to bare metal and its newest systems, and names Cohere as planning a Vault integration (primary), which provides the hardware basis for a premium isolated tier. 4) Cohere's continuing push on governed enterprise platforms, North 2 and agent-sprawl management (d1, d2), gives it a natural place to bundle retrieval with deployment.

## Connections to other events

Start with the number that holds up. Take a query that reranks 50 candidates of 1,000 tokens each, about 50,000 tokens. At list price that costs roughly $0.0025 on Pro and $0.001 on Fast. A million such queries a month comes to about $2,500 on Pro or $1,000 on Fast. Next, compare that with the generator. Reranking 50,000 tokens on Pro costs $0.0025. If Context Focus cuts N tokens from the prompt and the generator charges p dollars per million input tokens, the savings are N×p/1,000,000. Break-even is N = 2,500/p. At a hypothetical generator input price of $1 per million tokens, Context Focus pays for the whole Pro rerank once it removes about 2,500 tokens per query. At higher generator prices it pays off sooner. The evidence does not say whether Context Focus is billed separately, so treat this as an upper bound on its value. Now the hypothesis. If reranking costs this little per query, a customer's reranking spend on the API stays small even at large volume. Dedicated Vault capacity is different: the customer pays for it whether it is used or not. A confidential tier on attested hardware would presumably add a premium on top. We therefore suspect that, for Cohere, Rerank 5 works mainly as a reason to adopt North or Vault and that the dedicated tier holds the margin. We drop the earlier GPU-revenue estimate because we have neither Cohere's serving costs nor Vault prices. The counter-reading is just as consistent with the facts. Open-weight rerankers such as the BGE family can be self-hosted, which may be enough on its own to hold API prices near $0.02 to $0.05. In that case Vault and the confidential tier serve buyers who need isolation regardless, with no cross-subsidy. Three pieces of evidence would tell the two apart. First, Vault or confidential-tier prices compared with API cost at equal volume. Second, whether Cohere promotes Rerank 5 mainly as an entry point to Vault and North. Third, whether Cohere cuts the API price further when open models improve.

## Why it matters

For builders, the reranker is rarely the cost that matters. At a tenth to a quarter of a cent per 50-document query, the decision should rest on recall and nDCG on your own corpus and on how many generator tokens the reranker removes, not on per-token price. For buyers in regulated sectors, the choice is about deployment, not about the model. A dedicated Vault instance is a fixed cost. It makes sense when volume is steady and high, or when a compliance requirement rules out shared infrastructure. Ask for Vault pricing at your realistic utilization and compare it with API cost at your actual query volume. Before paying extra for a confidential tier, confirm that your auditors accept hardware attestation as a control. Whichever reading is right, the practical advice is the same: price the whole retrieval pipeline, not the reranker line item.

## What happens next

- **Cohere publicly documents a confidential-computing option for Model Vault, built on NVIDIA confidential computing, that lists Rerank 5 as deployable.** (6 months, 45%). Watch for: Cohere docs or its pricing page describe a confidential Vault option, with attestation documentation, that names Rerank 5.
- **Cohere publishes Vault or confidential-tier pricing detailed enough to compare with per-token API cost at a stated volume.** (12 months, 30%). Watch for: Per-instance or per-hour Vault prices appear on cohere.com/pricing or in public documentation, not only through 'contact sales'.
- **Cohere lowers Rerank 5 API list prices, or adds a cheaper reranker tier, within a year. That would point to open-weight price pressure rather than a deliberate entry-point price.** (12 months, 25%). Watch for: A Cohere pricing change or new reranker SKU priced below $0.02 per million tokens.

## Limits and uncertainty

All throughput and benchmark figures come from Cohere. The per-query arithmetic assumes 50 candidates of 1,000 tokens each, and real pipelines vary widely. The generator price in the break-even calculation is hypothetical. We do not know how Context Focus is billed, how many tokens it actually saves, or how it affects answer quality. Our evidence has no Vault prices, so the idea that Vault carries the margin is a hypothesis about how the business is structured, not a calculated figure. The counter-reading, that open-weight competition alone explains the price, fits the facts equally well. NVIDIA says Cohere "plans" the confidential integration. There is no date and no independent security review. We left out CALIBER because nothing in the evidence connects it to Rerank 5 or to Vault pricing.

## Skill to build now

Build a retrieval cost-and-quality harness that measures generator tokens saved, not just reranker spend. First exercise: take 200 real queries with labeled relevant passages and run three configurations: no reranker, Rerank 5 Fast with Context Focus, and a self-hosted open-weight reranker. For each, log recall@10, nDCG@10, tokens sent to the generator, and total cost per query including generation at your model's actual input price. Then find the monthly query volume at which dedicated capacity would cost less than API billing, once you have a real quote.

## Reader takeaway

Before you choose a reranker, put three numbers in one spreadsheet: reranker cost per query, generator tokens saved per query at your model's input price, and the fixed cost of a dedicated deployment at your realistic utilization. Then run Rerank 5 Fast with Context Focus, one self-hosted open-weight reranker and your current setup on about 200 labeled queries from your own corpus. Only move to Vault or a confidential tier if a written compliance requirement or steady high volume justifies paying for idle capacity.

## Open question

Is Rerank 5's low price a deliberate entry point to Vault and North, or simply the going rate set by open-weight rerankers, with Vault serving isolation buyers either way? And will regulated enterprises pay a lasting premium for hardware attestation when they could self-host an open reranker inside their own VPC?

## Sources

- [Cohere Rerank 5 | More Relevant Context for RAG and Agents](https://cohere.com/blog/rerank-5)

- [NVIDIA Confidential Computing Expands, Enabling Frontier Model Builders to Securely Scale Across Global Infrastructure](https://blogs.nvidia.com/blog/confidential-computing-model-builders-infrastructure)

- [Rethinking Confidence in Reasoning Models | Cohere](https://cohere.com/blog/rethinking-confidence-in-reasoning-models)

- [t.me](https://t.me/qwgai/1521)

- [Pricing | Secure and Scalable Enterprise AI](https://cohere.com/pricing)

- [Standard Vault Pricing](https://docs.cohere.com/docs/model-vault/standard/pricing)

- [BRIGHT: A Realistic and Challenging Benchmark for Reasoning-Intensive Retrieval](https://brightbenchmark.github.io/)

- [Introducing zerank-2: The Most Accurate Multilingual Instruction-Following Reranker](https://www.zeroentropy.dev/articles/zerank-2-advanced-instruction-following-multilingual-reranker)

- [Cohere Rerank](https://docs.cohere.com/docs/rerank)

- [Encryption & Key Management](https://docs.cohere.com/docs/model-vault/encrypted/encryption-key-management)

- [CALIBER: Calibrating Confidence Before and After Reasoning in Language Models](https://arxiv.org/abs/2606.24281)

Author: [Andrey Reshetnikov](https://newruntime.com/owner-profile.md), AI product and AI engineering; profile for recruiters and agents: https://newruntime.com/owner-profile.json

## Continue in the graph

Next records for an agent, linked to their Markdown twins. Replace `.md` with `.json` for structured fields, including `next_reads`.

- topic hub: [Trends](https://newruntime.com/topics/trends.md): Explore the trends topic hub.
- collection: [Analysis](https://newruntime.com/analysis.json): Open the latest authorial analysis.
- collection: [Source Ledger](https://newruntime.com/source-ledger.json): Inspect the public source evidence graph.
- whole graph: [discovery graph](https://newruntime.com/discovery-graph.json) (typed nodes and edges) and [site index](https://newruntime.com/site-index.md)