llm-d 1.0 Ships the Serving Layer DeepSeek Withheld. The Sponsors Sell the Managed Version.
Photo: Forkast
When DeepSeek open-sourced its production inference engine , the repository named three components a complete serving stack needs and does not ship: a KV cache system, prefill/decode disaggregation, and load balancing across data-parallel replicas. Those three pieces separate a fast engine from a running service. On October 20, at the PyTorch Conference, Google, Red Hat, and IBM shipped them — free, vendor-neutral, and already running in production. The project is llm-d 1.0 , a CNCF Sandbox stack built by more than 500 contributors across 25-plus organizations, founded in 2025 by Red Hat, IBM, Google Cloud, CoreWeave, and NVIDIA.
Source: Forkast
