PyTorch Foundation Welcomes vLLM and DeepSpeed as Hosted Projects
PyTorch is one of AI’s top open source projects. Now, its ruling foundation, the PyTorch Foundation has announced a major expansion. The group recently announced it would be an umbrella foundation for other open source AI programs. The first of these top-level projects are vLLM and DeepSpeed.
Virtual Large Language Model (vLLM) is an open source library focused on making large language model (LLM) inference and serving significantly faster and more efficiently. Originally developed at University of California Berkeley, vLLM is designed to address the speed and memory challenges of running large AI models in production environments.
PagedAttention Algorithm
VLLM’s most interesting feature, to me at least, is PagedAttention Algorithm. Inspired by virtual memory in operating systems, PagedAttention optimizes memory usage by managing attention keys and values in non-contiguous blocks, reducing memory waste and enabling larger batch sizes.
In a statement, Simon Mo, Project Co-Lead at vLLM, said, “We’re excited that vLLM is one of the first Platform Projects joining the PyTorch Foundation. VLLM is built on top of PyTorch with deep integration, such as Torch Compile and multi-hardware support. We look forward to further collaborating with the ecosystem that powers innovations in open source and vendor-neutral technologies for AI.”
DeepSpeed is a distributed training library that simplifies scaling AI workloads. It does this by providing optimization techniques such as Zero Redundancy Optimizer (ZeRO), 3D parallelism, and inference acceleration, enabling the efficient training of extremely large models. It is widely used in both academic research and production environments.
Like vLLM, DeepSpeed has its roots in PyTorch. As Olatunji Ruwase, DeepSpeed’s Project Lead, said in a statement, He’s “delighted to become a hosted Platform project in the PyTorch Foundation. “From inception, DeepSpeed has built on PyTorch, with critical dependencies on features such as Module, Tensor, Distributed, and Compiler. We are eager to leverage this closer integration with the PyTorch ecosystem to achieve our goal of providing open and democratized access to state-of-the-art AI technologies for all.”
Platforms and Verticals
Looking ahead, under its new umbrella structure, the PyTorch Foundation will oversee two categories of projects. The first is platform projects. These are domain-agnostic programs support the entire AI lifecycle, including training, inference, model optimization, deployment, and agentic systems. The other, vertical projects, as the name suggests, are tools tailored for specific industries and applications, such as bioinformatics, geospatial intelligence, and protein folding.
Projects accepted into the foundation benefit from vendor-neutral governance, strategic support, increased visibility, and access to a global community of contributors. The foundation distinguishes between “ecosystem projects,” which remain independently governed, and “foundation-hosted projects,” which adopt the foundation’s open governance model and receive comprehensive operational support. Both vLLM and DeepSpeed are foundation-hosted projects.
Opening the Umbrella
Since its inception at the Linux Foundation two and a half years ago, the PyTorch Foundation has grown to include over 30 member organizations and 120 ecosystem projects. These newest moves are a natural evolution of its rapid growth and global momentum.
“This is an exciting new chapter for the PyTorch Foundation and the broader open source AI ecosystem,” said Matt White, the PyTorch Foundation’s Executive Director. “By transitioning to an umbrella foundation, we’re not only formalizing the momentum we’ve built across the PyTorch ecosystem; we’re creating space for new projects and innovators to thrive within a vendor-neutral, open governance environment.”
For more information on joining the PyTorch Foundation or submitting a project, visit the official website and stay tuned for further updates and opportunities to participate.
