---
type: analysis
slug: cohere-s-confidential-inference-moves-trust-from-the-model-vendor-to-two
title: "Cohere's confidential inference moves trust from the model vendor to two hardware anchors, and nobody has said whether you can verify without them"
description: "Cohere's attested Model Vault beta, NVIDIA's Confidential Bare Metal and Europe's sovereign-AI projects all ask buyers to trust their model vendor less. The trust moves to NVIDIA's GPU attestation and to the CPU enclave stack underneath it. The published material doesn't say whether customers can check that evidence without a vendor-run service."
published_at: 2026-10-24T12:00:00.000Z
topics: ["trends"]
source_urls: ["https://cohere.com/blog/introducing-confidential-computing-on-cohere-model-vault","https://cohere.com/blog/inside-cohere-model-vault-building-verifiable-confidential-ai-inference","https://blogs.nvidia.com/blog/confidential-computing-model-builders-infrastructure","https://blogs.nvidia.com/blog/european-public-sector-open-models","https://www.testingcatalog.com/meta-prepares-muse-confidential-vm-for-private-ai-work"]
author: "Andrey Reshetnikov"
author_profile: https://newruntime.com/owner-profile.md
---
# Cohere's confidential inference moves trust from the model vendor to two hardware anchors, and nobody has said whether you can verify without them

Cohere's attested Model Vault beta, NVIDIA's Confidential Bare Metal and Europe's sovereign-AI projects all ask buyers to trust their model vendor less. The trust moves to NVIDIA's GPU attestation and to the CPU enclave stack underneath it. The published material doesn't say whether customers can check that evidence without a vendor-run service.

## Summary

On October 22, 2026, Cohere opened a public beta of Confidential Computing for Model Vault, its dedicated inference service. It runs on NVIDIA H100 GPUs with NVIDIA Confidential Computing inside Azure confidential VMs. Before any request leaves, the client checks signed attestation evidence, then encrypts the request with OHTTP to a gateway inside the attested environment. A day earlier, NVIDIA announced Confidential Bare Metal, which Cohere says it plans to add to Model Vault. NVIDIA also published a roundup of European public-sector projects built around keeping sensitive data under the organization's own control.

## What happened

According to Cohere, Model Vault's serving stack runs inside a confidential virtual machine. NVIDIA Confidential Computing extends the protection to GPU memory and encrypts traffic between the CPU and the GPU. Before sending a payload, the customer's client checks signed evidence about the hardware, firmware and software configuration against an approved policy. If the software measurements have changed, or the GPU is not in confidential mode, the client does not send the request. Payloads are encrypted with Oblivious HTTP to a public key tied to the attested environment. Cohere says only the OHTTP gateway inside that environment can decrypt them. Customers keep the existing Model Vault APIs and request formats.

Today the deployment is Azure plus H100 and nothing else. Cohere describes Confidential Bare Metal as a plan, not a shipped option. Cohere publishes no pricing, regions, performance overhead figures, supported-model list, independent audit or GA date. It says the feature does not replace application security, compliance, contracts or data-residency rules, and it does not claim to have removed every trusted component.

NVIDIA's October 21 post adds Bare Metal to its existing Confidential Containers for Kubernetes and Confidential VMs. It says confidential computing is expanding to Grace Blackwell NVL72 and Rubin systems and will integrate with DSX OS. In the same post, NVIDIA names Google Cloud's A4X Max on GB300 NVL72 and Cohere's Model Vault as partners. These are partner statements published by NVIDIA, not deployments Cohere runs today. NVIDIA's separate European post lists projects in Poland, the UK, Germany, France and Greece. One of them is a sovereign cloud for classified work in Germany, built by secunet, NVIDIA, Red Hat and deepset.

## What the archive adds

The archive shows other companies moving the same way. TestingCatalog reports that Meta is preparing a Muse Confidential VM for private AI work, which suggests consumer assistants are also starting to claim that even the operator can't see your data.

Cohere's documentation describes Model Vault as single-tenant inference with an Encrypted Vault option that protects data in use inside hardware-backed trusted execution environments (d1). It also says the OHTTP keys are generated inside the TEE and never shared (d2). That point matters: it means Cohere is designing itself out of the plaintext path, not just promising to stay out of it.

Cohere's partnership with the Government of Canada treats sovereign AI, meaning control over where sensitive data is processed, as a public-sector requirement (d3). That is the procurement pressure behind this feature.

The material available to us contains no audit and no vulnerability disclosure for this deployment, and Cohere has published no overhead numbers of its own. Every security and performance claim here comes from the vendors.

## Trend signal

Privacy promises in enterprise AI are moving from contracts to evidence a machine can check. A buyer used to get a data-processing agreement and a no-training clause. Cohere's client now refuses to send data unless the platform proves what it is running. Meta is reportedly preparing a version of the same idea for consumer chat. NVIDIA is selling it at every deployment layer: containers, VMs, bare metal, and its reference OS for AI factories. Europe's sovereignty projects are the demand behind it.

Together the stories show the trust anchor moving. It used to be the model vendor's word. Now it is a hardware root of trust plus an attestation policy. On closer reading that root has two parts. On the GPU side, every announcement in this cluster depends on NVIDIA. On the CPU side, the confidential VM rests on a CPU TEE: on Azure that is AMD SEV-SNP or Intel TDX, depending on the VM family, and Cohere does not say which. That stack also comes with its own firmware and attestation path. So the trend is not "trust NVIDIA instead of Cohere". It is "trust NVIDIA plus a CPU-enclave vendor and a cloud, instead of Cohere".

## Why now

Four things are driving this now:

1. **Public-sector buyers want control over where sensitive data is processed.** Cohere's Canada partnership (d3) and the projects in NVIDIA's European roundup both show this.
2. **NVIDIA has turned GPU confidential computing into a product line** spanning containers, VMs, bare metal and its reference OS. Model vendors get something ready-made to build on, alongside the CPU confidential VMs that clouds already offer.
3. **Hosted vendors need an answer to self-hosted open models on sensitive data.** Single-tenant vaults with keys generated inside the TEE (d1, d2) let Cohere argue that hosted can be as private as on-prem.
4. **Peers are moving the same way.** TestingCatalog reports Meta's planned Muse Confidential VM.

## Connections to other events

Here is the platform dependency in concrete terms. Cohere says some trusted components remain. Our read is that they fall into three groups:

1. **The CPU TEE and the cloud.** This is the Azure confidential VM, its CPU vendor's firmware and how its evidence is verified.
2. **The GPU.** This is NVIDIA's firmware, the confidential mode itself, and NVIDIA's attestation evidence.
3. **Cohere's own code.** This is the OHTTP gateway build and the software measurements a customer is told to approve.

Attestation does not remove trust. It narrows trust to whoever signs the evidence and whoever decides which measurement counts as "good".

Three consequences follow, and all of them are inference.

**NVIDIA's leverage grows with portability.** If Cohere ships Bare Metal and the partner clouds NVIDIA names deliver, one GPU attestation story would cover several providers. Moving between clouds gets easier, while the shared dependency on one GPU vendor's signing and mode rules gets bigger. The CPU side is less concentrated: a model vendor can choose between enclave technologies. That makes NVIDIA's GPU anchor the more concentrated of the two, though not the only one.

**Availability now depends on update discipline.** A client that blocks requests on a failed check will stop sending traffic if a firmware or software update changes measurements before your policy is updated. That could happen on the GPU side or the CPU side.

**The sovereignty question.** A European government might get strong assurance that the cloud operator can't read its data, while both hardware roots that make the assurance checkable belong to US vendors. Whether that conflicts with sovereignty depends on one detail the sources leave out: can a customer verify the evidence fully offline, against roots it holds itself, or does verification go through a vendor-run service? The answer decides whether the dependency is about supply or about runtime control.

## Why it matters

**For builders:** treat the attestation policy as production configuration. Give it an owner, put it under version control, and roll changes out deliberately. Ask Cohere how measurement changes are announced, and how far in advance, on both the CPU-TEE side and the GPU side. Ask what happens to in-flight traffic when the policy and the platform disagree.

**For buyers in regulated industries:** they now have a stronger case for using a hosted model on sensitive data. Procurement should still list every component that remains trusted: the CPU TEE and its firmware, the cloud's verification path, NVIDIA GPU firmware and attestation, and the vendor's gateway build. Then check whether each one is acceptable for the buyer's jurisdiction.

**Budget for performance testing.** Cohere has published no overhead figures, so run your own encrypted-versus-standard comparison on your real batch sizes and prompt lengths before you commit capacity.

**Competitively:** vendors that sell only on contractual privacy are likely to lose sensitive workloads to vendors whose privacy claim the client can check. NVIDIA gains in either case.

## What happens next

- **Cohere documents a confidential Model Vault deployment target other than Azure H100, either Blackwell-class GPUs or NVIDIA Confidential Bare Metal.** (6 months, 40%). Watch for: A Cohere docs or blog page naming GB200/GB300, Rubin or Confidential Bare Metal as a supported encrypted-vault target.
- **Cohere publishes GA status or its own latency/throughput overhead figures for confidential Model Vault.** (6 months, 40%). Watch for: Cohere docs listing GA, regions, or a benchmark comparing encrypted and standard vault performance.
- **A public disclosure (CVE, vendor bulletin or academic paper) describes a weakness in NVIDIA GPU confidential mode or in a CPU TEE used for confidential GPU VMs that forces an attestation-policy or firmware change for deployed confidential inference.** (12 months, 35%). Watch for: An NVIDIA, AMD, Intel or Azure security bulletin referencing confidential GPU/VM attestation, followed by updated required measurements.

## Limits and uncertainty

Every security property here is a vendor claim. There is no independent audit of Model Vault, and Cohere has published no overhead, pricing or supported-model data. We could not check the third-party overhead measurements for H100 confidential mode against primary sources, so we don't quote a number. Measure overhead yourself.

We don't know which CPU TEE Cohere's Azure VMs use. We also don't know whether verification goes through NVIDIA- or Microsoft-hosted attestation services or can run fully offline, and that is the detail that decides how strong either dependency is.

Our argument that policy breaks under firmware updates is inference. Cohere may already use staged measurement allowlists it hasn't documented.

The Google A4X Max and Blackwell or Rubin claims come from NVIDIA's partner announcement. They are not shipped Model Vault options.

Finally, the sovereignty point only applies to buyers whose threat model includes jurisdiction over hardware vendors. Many enterprises only need to keep the cloud operator and the model vendor out, and this design may be enough for them.

## Skill to build now

Practise writing attestation policy as code, with both hardware anchors made explicit. First exercise: pick one sensitive LLM workflow and draw its trust chain. Your client, the cloud's CPU TEE and its verification path, NVIDIA GPU firmware and attestation, and the vendor's gateway build each get a row. For every row, record three things: who can change that component's measurement, how you would find out, and whether you can verify it without calling a vendor-hosted service. If you have access to the beta, add a test that loads a wrong policy and asserts your client fails closed.

## Reader takeaway

If you are evaluating the confidential Model Vault beta, or any attested inference service, get three answers in writing:

1. A list of every component that remains trusted, with the CPU TEE named.
2. The process for changing attestation measurements on both the GPU and CPU sides.
3. Whether the evidence can be verified offline against roots you hold.

Then run two tests. First, load a deliberately mismatched policy and confirm your application fails closed instead of falling back to an unprotected endpoint. Second, run an encrypted-versus-standard benchmark on your own traffic.

## Open question

Confidential AI evidence chains back to two US hardware vendors' roots, NVIDIA for the GPU and AMD or Intel for the CPU. If a European government can verify that evidence offline against roots it holds, is the result sovereign enough? Or does sovereignty require attestation roots that are held locally, something none of these announcements offers?

## Sources

- [Confidential Computing on Cohere Model Vault | Cohere](https://cohere.com/blog/introducing-confidential-computing-on-cohere-model-vault)

- [Cohere Model Vault: Verifiable Confidential AI Inference](https://cohere.com/blog/inside-cohere-model-vault-building-verifiable-confidential-ai-inference)

- [NVIDIA Confidential Computing Expands, Enabling Frontier Model Builders to Securely Scale Across Global Infrastructure](https://blogs.nvidia.com/blog/confidential-computing-model-builders-infrastructure)

- [NVIDIA Open Models Power a New Wave of European Public Sector Projects](https://blogs.nvidia.com/blog/european-public-sector-open-models)

- [Meta prepares Muse Confidential VM for private AI work](https://www.testingcatalog.com/meta-prepares-muse-confidential-vm-for-private-ai-work)

- [Model Vault Overview | Cohere](https://docs.cohere.com/v1/docs/model-vault)

- [Encryption & Key Management | Cohere](https://docs.cohere.com/docs/model-vault/encrypted/encryption-key-management)

- [Cohere Deepens Partnership with Government of Canada](https://cohere.com/blog/cohere-partnership-canada-government)

- [Hardware-backed security at global scale: Confidential G4 with NVIDIA Blackwell is GA](https://cloud.google.com/blog/topics/ai-infrastructure/confidential-g4-gpus-are-ga)

- [GPU TEE Cloud — H100/H200/B300 Confidential AI](https://phala.com/gpu-tee)

- [Confidential AI](https://www.fortanix.com/platform/confidential-ai)

- [Confidential LLM Inference: Performance and Cost Across CPU and GPU TEEs](https://arxiv.org/abs/2509.18886)

- [Secure AI Compatibility Matrix](https://www.nvidia.com/en-us/data-center/solutions/confidential-computing/secure-ai-compatibility-matrix/)

- [Confidential Computing on NVIDIA Hopper GPUs: A Performance Benchmark Study](https://arxiv.org/abs/2409.03992)

Author: [Andrey Reshetnikov](https://newruntime.com/owner-profile.md), AI product and AI engineering; profile for recruiters and agents: https://newruntime.com/owner-profile.json

## Continue in the graph

Next records for an agent, linked to their Markdown twins. Replace `.md` with `.json` for structured fields, including `next_reads`.

- topic hub: [Trends](https://newruntime.com/topics/trends.md): Explore the trends topic hub.
- collection: [Analysis](https://newruntime.com/analysis.json): Open the latest authorial analysis.
- collection: [Source Ledger](https://newruntime.com/source-ledger.json): Inspect the public source evidence graph.
- whole graph: [discovery graph](https://newruntime.com/discovery-graph.json) (typed nodes and edges) and [site index](https://newruntime.com/site-index.md)