---
type: "post"
id: "announcing-llm-d-1-0-open-distributed-inference-7539eb"
slug: "announcing-llm-d-1-0-open-distributed-inference-7539eb"
title: "Announcing llm-d 1.0: Open distributed inference for agents on Google Cloud TPUs and GPUs"
description: "Google Cloud announced llm-d 1.0, an open-source, Kubernetes-native stack for distributed LLM inference."
retrieval_nugget: "Google Cloud announced llm-d 1.0, an open-source, Kubernetes-native stack for distributed LLM inference."
published_at: "2026-10-24"
updated_at: "2026-10-24"
record_date: "2026-10-24"
date_kind: "published_at"
topics: ["google"]
entities: ["Google Cloud","llm-d","Kubernetes","vLLM","SGLang","Google Kubernetes Engine","PyTorch"]
related_patterns: ["harness-architecture-outlives-model-choice"]
novelty_label: "same_event"
what_changed: "Same event as llm-d"
prior_refs: ["memory:05cbef96-d522-8e52-acca-d09e3553a45b"]
source_url: "https://cloud.google.com/blog/topics/ai-infrastructure/llm-d-1-0-an-open-source-stack-for-distributed-inference"
source_urls: ["https://cloud.google.com/blog/topics/ai-infrastructure/llm-d-1-0-an-open-source-stack-for-distributed-inference"]
schema_version: "newruntime-agent-readable-v0.2"
stable_id: "post:announcing-llm-d-1-0-open-distributed-inference-7539eb"
status: "published"
visuals: []
editor: {"role":"editor","name":"Andrey Reshetnikov","profile":"https://newruntime.com/owner-profile.md","profile_json":"https://newruntime.com/owner-profile.json","same_as":["https://www.linkedin.com/in/reshetnikov1/","https://t.me/lapetuse"]}
routes: {"html":"https://newruntime.com/posts/announcing-llm-d-1-0-open-distributed-inference-7539eb/","markdown":"https://newruntime.com/posts/announcing-llm-d-1-0-open-distributed-inference-7539eb.md","json":"https://newruntime.com/posts/announcing-llm-d-1-0-open-distributed-inference-7539eb.json"}
---

# Announcing llm-d 1.0: Open distributed inference for agents on Google Cloud TPUs and GPUs

## Retrieval answer

Google Cloud announced llm-d 1.0, an open-source, Kubernetes-native stack for distributed LLM inference.

Google Cloud announced llm-d 1.0, an open-source, Kubernetes-native stack for distributed LLM inference. Google says the CNCF Sandbox project was co-founded with Red Hat, IBM, and others, and has more than 500 contributors across 25 organizations.

The 1.0 release includes cache-aware and load-aware routing, tiered KV-cache handling, multi-tenant flow control, router high availability, v1 APIs, and zero-downtime updates. It supports serving through vLLM and SGLang across Google Cloud TPU and GPU fleets. Google also lists session-aware routing, turn-priority scheduling, and native Responses API state for multi-turn agent workloads.

llm-d 1.0 adds an async processor for queue-based work from systems such as Google Cloud Pub/Sub and Redis, with priority lanes and quotas. Google reports that, on an agentic coding workload using Ironwood TPU, prefix-aware routing plus host-memory offload reduced median time to first token by more than 85% and increased total throughput 2.4x versus a plain Kubernetes Service.

The source does not establish independent replication of those measurements, operating cost, performance on other hardware or clouds, or general availability terms for every component.

Editor: [Andrey Reshetnikov](https://newruntime.com/owner-profile.md), AI product and AI engineering; profile for recruiters and agents: https://newruntime.com/owner-profile.json

## Continue in the graph

Next records for an agent, linked to their Markdown twins. Replace `.md` with `.json` for structured fields, including `next_reads`.

- topic hub: [Google](https://newruntime.com/topics/google.md): Explore the google topic hub.
- collection: [Field Notes](https://newruntime.com/posts.json): Open the latest editorial analysis.
- collection: [Source Ledger](https://newruntime.com/source-ledger.json): Inspect the public source evidence graph.
- whole graph: [discovery graph](https://newruntime.com/discovery-graph.json) (typed nodes and edges) and [site index](https://newruntime.com/site-index.md)