Attested inference stops the vendor reading your prompts, not the vendor cutting you off, and both sovereignty routes run on NVIDIA's root of trust

Summary

On October 22, 2026, Cohere opened a public beta of Confidential Computing for Model Vault, its dedicated inference service, running on NVIDIA H100 GPUs on Azure. Before sending anything, the client checks signed attestation evidence. Payloads are then encrypted with OHTTP to a gateway inside the attested environment. NVIDIA also announced Confidential Bare Metal and said confidential-computing support is expanding to Blackwell and Rubin systems. In an October 21 post, it listed European public-sector projects that run open models on infrastructure the organisations control.

What happened

Cohere's two posts describe the design. The Model Vault serving stack runs inside a confidential VM. NVIDIA Confidential Computing protects GPU memory and encrypts traffic between the CPU and GPU. The customer's client checks signed evidence about hardware, firmware and software against an approved policy. If software measurements have changed, or the GPU is not in confidential mode, the request is never sent. Payloads are encrypted to a public key bound to the attested environment, and Cohere says only an OHTTP gateway inside that environment can decrypt them. Customers keep the existing Model Vault APIs. Cohere has not published pricing, regions, performance overhead, supported models, a GA date or an independent audit. It says the feature does not replace application security, compliance, contractual controls or data-residency requirements, and it does not claim to remove every trusted component.

NVIDIA's confidential-computing post adds Confidential Bare Metal alongside its existing Confidential Containers and Confidential VMs. It says support is expanding to Grace Blackwell NVL72, Rubin and DSX OS. Google Cloud is extending support to A4X Max (GB300 NVL72) instances, and Cohere plans to integrate the bare-metal option into Model Vault. NVIDIA's separate October 21 post describes European public-sector work that takes the other route: open models on infrastructure the organisations control. Examples include Poland's eHealth imaging platform on DGX BasePOD and MONAI; an agentic system for more than 400 Polish labour offices using PLLuM, GPT OSS and Inkling; synthetic-persona datasets for the UK and Germany; and a sovereign cloud for classified work being developed by secunet, NVIDIA, Red Hat and deepset. NVIDIA does not say all of these are in production.

What the archive adds

According to Cohere's security-model documentation (d1), Model Vault Encrypted is meant to close the "data-in-use gap": prompts and responses should stay inaccessible to both Cohere and the infrastructure operator while inference runs. That is the commercial point. Until now, a hosted-model vendor could offer a buyer a contractual promise and not much more. On the demand side, the European Commission describes a €180 million sovereign-cloud procurement for EU institutions and reports strong interest from public administrations (d2). That is the kind of buyer both routes in this cluster are courting. The archive holds no independent audits, CVEs or overhead measurements for Cohere's design. Everything about how the system works comes from Cohere and NVIDIA.

Trend signal

Buyers with sensitive data now have two ways to handle distrust of AI vendors. The first is to bring the model in-house: open weights on owned or sovereign infrastructure, as in NVIDIA's European examples. The second is to keep using the vendor's model while making it technically unable to see the data, through hardware isolation, attestation and payload encryption, as Cohere does. Cohere's design depends on NVIDIA Confidential Computing extending protection to GPU memory and CPU-GPU traffic. NVIDIA's announcement pushes that capability onto Blackwell, Rubin and bare-metal deployments. Both routes are maturing in the same month, so buyers can choose between them. What they share is the layer underneath. Both run on NVIDIA hardware, and in the confidential route the attestation chain starts with NVIDIA's GPU.

Why now

1) GPU memory is now inside the protected boundary. Cohere's design relies on NVIDIA Confidential Computing protecting GPU memory and CPU-GPU traffic, and NVIDIA says support is extending to Blackwell, Rubin and bare metal. 2) Procurement is being driven by sovereignty: the European Commission describes a €180M sovereign-cloud procurement and strong public-administration interest (d2). 3) Closed-model vendors need an answer to "self-host or nothing". Cohere's docs frame Model Vault Encrypted as closing the data-in-use gap, so that even Cohere cannot see prompts (d1).

Connections to other events

Here is our read of who gains and who loses. These are inferences from vendor announcements, not measured outcomes.

NVIDIA is well placed on either route. A ministry that self-hosts open models runs them on NVIDIA systems such as DGX BasePOD. An enterprise that picks Cohere's confidential vault gets a guarantee anchored in NVIDIA's GPU confidential mode and attestation. Confidential Bare Metal stretches that anchoring role to third-party and enterprise infrastructure. That does not make NVIDIA a guaranteed winner: its position depends on its attestation stack holding up under scrutiny.

Cohere gains an answer to the sovereignty objection that does not require handing over weights. A regulated buyer can use a closed model with a cryptographic check, rather than only a contract, that Cohere cannot read prompts. If the design holds, that was previously something only self-hosting could offer.

Azure and Google Cloud gain a confidential GPU product to sell, with A4X Max named for Google. Cohere has published no pricing, so whether confidential instances carry a premium is not established here.

Vendors whose only privacy story is a contractual pledge are likely to lose ground in procurement once a competitor can show signed measurements. The case for self-hosting open models also narrows, though it does not disappear. Attestation weakens the argument that self-hosting is the only way to keep data from the vendor. It leaves untouched what self-hosting still uniquely provides: independence from the vendor's jurisdiction and from the vendor's decision to change terms or end service.

Why it matters

For buyers, attestation narrows trust; it does not remove it. Under Cohere's design you still trust NVIDIA's root of trust, Azure's confidential VM stack, the OHTTP gateway, and whichever software measurements you approve. That beats a bare contract, but it is a different arrangement from owning the model. Separate your two risks: the vendor reading your data, and the vendor (or the jurisdiction it operates under) cutting off your access. Confidential inference addresses the first and not the second. Cohere itself says it does not replace data-residency or contractual controls. For builders, the attestation policy is becoming the critical artefact: which measurements to accept, and how to roll them forward when the vendor ships an update without breaking production. Vendors without an attestation story should expect the question to appear in RFPs for regulated workloads.

What happens next

• Cohere publishes either GA pricing or a general-availability date for Model Vault confidential computing. (6 months, 50%). Watch for: A Cohere pricing page, docs page or blog post that states an encrypted-vault price or a GA date. Coin-flip odds: public betas often reach GA within two quarters, but Cohere has given no timeline.
• Cohere announces Model Vault confidential computing on NVIDIA Confidential Bare Metal or Blackwell-class hardware. (12 months, 45%). Watch for: A Cohere announcement or docs update naming GB200/GB300 or a Bare Metal target for encrypted vaults. NVIDIA's post already says Cohere plans this integration, but hardware rollouts often slip past a year.
• Cohere publishes or links to a named third-party security assessment of the Model Vault attestation and OHTTP design. (12 months, 30%). Watch for: A report from a named security firm, or a Cohere post linking to an external review. Probability is low because neither announcement mentions any planned audit.

Limits and uncertainty

Everything here comes from vendor announcements. Cohere has published no independent audit, overhead figure, pricing, region list or GA date. Overhead and pricing figures from other confidential-GPU providers would be useful for comparison, but they are not in the material we could verify, so we do not quote any. TEE designs have a history of side-channel and firmware problems. One serious vulnerability in GPU confidential mode would undercut the whole confidential route and reorder the winners above. We infer that procurement will reward attestation; the evidence does not show it, and the EU procurement (d2) is not stated to require attestation. Not all of the European projects are confirmed to be in production.

Skill to build now

Learn to write and maintain remote-attestation policies. First exercise: using the evidence description in Cohere's security-model docs (d1), list the fields you would pin (GPU confidential-mode state, firmware versions, software measurements). Then write down what your client should do when the vendor pushes a new measurement: block, alert, or accept only from a signed allow-list. Test your choice against a simulated mismatch.

Reader takeaway

If you are evaluating hosted LLMs for regulated data, add one RFP requirement this quarter: "Provide signed remote-attestation evidence, the reference measurements we should pin, and your process for changing them on updates." Score vendors who cannot answer below those who can. Keep a separate line for exit and jurisdiction risk, because attestation does not address either one.

Open question

If attestation lets a closed-model vendor prove it cannot read your data, are jurisdiction and exit risk alone enough to justify the operational cost of self-hosting open models for most regulated buyers?