Google says the CNCF Sandbox project was co-founded with Red Hat, IBM, and others, and has more than 500 contributors across 25 organizations.
The 1.0 release includes cache-aware and load-aware routing, tiered KV-cache handling, multi-tenant flow control, router high availability, v1 APIs, and zero-downtime updates. It supports serving through vLLM and SGLang across Google Cloud TPU and GPU fleets. Google also lists session-aware routing, turn-priority scheduling, and native Responses API state for multi-turn agent workloads.
llm-d 1.0 adds an async processor for queue-based work from systems such as Google Cloud Pub/Sub and Redis, with priority lanes and quotas. Google reports that, on an agentic coding workload using Ironwood TPU, prefix-aware routing plus host-memory offload reduced median time to first token by more than 85% and increased total throughput 2.4x versus a plain Kubernetes Service.
The source does not establish independent replication of those measurements, operating cost, performance on other hardware or clouds, or general availability terms for every component.