# VLLM > Mediated Wiki article. Canonical URL: https://mediated.wiki/source/VLLM > Markdown URL: https://mediated.wiki/source/VLLM.md > Source: https://en.wikipedia.org/wiki/VLLM > Source revision: 1354268161 > License: Creative Commons Attribution-ShareAlike 4.0 International (https://creativecommons.org/licenses/by-sa/4.0/) {{lowercase title}} {{Short description|Open-source software for large language model inference}} {{Use mdy dates|date=April 2026}} {{Infobox software | name = vLLM | logo = vLLM.svg | author = Sky Computing Lab
[Cal Berkeley](/source/University_of_California%2C_Berkeley) | developer = vLLM contributors | released = 2023 | programming language = [Python](/source/Python_(programming_language)), [CUDA](/source/CUDA), [C++](/source/C%2B%2B) | genre = [Large language model](/source/Large_language_model) [inference engine](/source/inference_engine) | license = [Apache License 2.0](/source/Apache_License_2.0) | website = {{URL|https://vllm.ai}} | repo = {{URL|https://github.com/vllm-project/vllm}} }} '''vLLM''' is an open-source software framework for inference and serving of [large language model](/source/large_language_model)s and related [multimodal model](/source/multimodal_model)s. Originally developed at the [University of California, Berkeley](/source/University_of_California%2C_Berkeley)'s Sky Computing Lab,{{Cite web |title=vLLM - A High-Throughput and Memory-Efficient Inference and Serving Engine for LLMs |url=https://sky.cs.berkeley.edu/project/vllm/ | publisher=UC Berkeley, Sky Computing Lab}} the project is centered on ''[PagedAttention](/source/PagedAttention)'', a [memory-management](/source/memory_management) method for [transformer](/source/Transformer_(deep_learning)) [key–value cache](/source/Transformer_(deep_learning))s, and supports features such as continuous batching, [distributed](/source/distributed_computing) inference, [quantization](/source/Large_language_model), and [OpenAI](/source/OpenAI)-compatible [APIs](/source/application_programming_interface).{{cite web |title=GitHub - vllm-project/vllm: A high-throughput and memory-efficient inference and serving engine for LLMs |url=https://github.com/vllm-project/vllm |website=GitHub |publisher=GitHub, Inc. |access-date=April 22, 2026}}{{cite conference |last1=Kwon |first1=Woosuk |last2=Li |first2=Zhuohan |last3=Zhuang |first3=Siyuan |last4=Sheng |first4=Ying |last5=Zheng |first5=Lianmin |last6=Yu |first6=Cody Hao |last7=Gonzalez |first7=Joseph E. |last8=Zhang |first8=Hao |last9=Stoica |first9=Ion |title=Efficient Memory Management for Large Language Model Serving with PagedAttention |conference=Proceedings of the ACM SIGOPS 29th Symposium on Operating Systems Principles |year=2023 |url=https://arxiv.org/abs/2309.06180 |access-date=April 22, 2026}}{{cite web |title=vLLM |url=https://pytorch.org/projects/vllm/ |website=PyTorch |publisher=PyTorch Foundation |access-date=April 22, 2026}} According to a project [maintainer](/source/software_maintainer), the "v" in vLLM originally referred to "virtual", inspired by [virtual memory](/source/virtual_memory).{{cite web |title=vLLM full name |url=https://github.com/vllm-project/vllm/issues/835 |website=GitHub |publisher=GitHub, Inc. |date=August 23, 2023 |access-date=April 22, 2026}} == History == vLLM was introduced in 2023 by researchers affiliated with the Sky Computing Lab at UC Berkeley. Its core ideas were described in the 2023 paper ''Efficient Memory Management for Large Language Model Serving with [PagedAttention](/source/PagedAttention)'',{{cite arXiv |last1=Kwon |first1=Woosuk |last2=Li |first2=Zhuohan |last3=Zhuang |first3=Siyuan |last4=Sheng |first4=Ying |last5=Zheng |first5=Lianmin |last6=Yu |first6=Cody Hao |last7=Gonzalez |first7=Joseph E. |last8=Zhang |first8=Hao |last9=Stoica |first9=Ion |eprint=2309.06180 |title=Efficient Memory Management for Large Language Model Serving with PagedAttention |class=cs.LG |date=2023-09-12}} which presented the system as a [high-throughput](/source/High-throughput_computing) and [memory](/source/Memory_(computer))-efficient serving engine for [large language model](/source/Large_language_model)s. In 2025, the [PyTorch](/source/PyTorch) Foundation announced that vLLM had become a Foundation-hosted project. PyTorch's project page states that the [University of California, Berkeley](/source/University_of_California%2C_Berkeley) contributed vLLM to the [Linux Foundation](/source/Linux_Foundation) in July 2024.{{cite web |title=PyTorch Foundation Welcomes vLLM as a Hosted Project |url=https://pytorch.org/blog/pytorch-foundation-welcomes-vllm/ |website=PyTorch |publisher=PyTorch Foundation |date=May 7, 2025 |access-date=April 22, 2026}} In January 2026, ''[TechCrunch](/source/TechCrunch)'' reported that the creators of vLLM had launched the startup Inferact to commercialize the project, raising $150 million in seed funding.{{cite web |last=Temkin |first=Marina |title=Inference startup Inferact lands $150M to commercialize vLLM |url=https://techcrunch.com/2026/01/22/inference-startup-inferact-lands-150m-to-commercialize-vllm/ |website=TechCrunch |date=January 22, 2026 |access-date=April 22, 2026}} == Architecture == According to its 2023 paper, vLLM was designed to improve the efficiency of [large language model](/source/large_language_model) serving by reducing memory waste in the [key–value cache](/source/Transformer_(deep_learning)) used during [transformer](/source/Transformer_(deep_learning)) inference. The paper introduced ''[PagedAttention](/source/PagedAttention)'', an algorithm inspired by [virtual memory](/source/virtual_memory) and [paging](/source/paging) techniques in [operating system](/source/operating_system)s, and described vLLM as using block-level memory management and request scheduling to increase [throughput](/source/throughput) while maintaining similar [latency](/source/Latency_(engineering)). The project documentation and repository describe support for continuous batching, chunked prefill, [speculative decoding](/source/speculative_decoding), prefix caching, [quantization](/source/Large_language_model), and multiple forms of [distributed](/source/distributed_computing) inference and serving. [PyTorch](/source/PyTorch) has described vLLM as a high-throughput, memory-efficient inference and serving engine that supports a range of hardware back ends, including [NVIDIA](/source/Nvidia) and [AMD](/source/Advanced_Micro_Devices) [GPUs](/source/graphics_processing_unit), [Google TPUs](/source/Tensor_Processing_Unit), [AWS](/source/AWS) Trainium, and [Intel](/source/Intel) processors. == See also == * [SGLang](/source/SGLang) * [TensorRT-LLM](/source/TensorRT) * [llama.cpp](/source/llama.cpp) * [OpenVINO](/source/OpenVINO) * [Open Neural Network Exchange](/source/Open_Neural_Network_Exchange) * [Comparison of deep learning software](/source/Comparison_of_deep_learning_software) * [Comparison of machine learning software](/source/Comparison_of_machine_learning_software) * [List of software developed at universities](/source/List_of_software_developed_at_universities) * [Lists of open-source artificial intelligence software](/source/Lists_of_open-source_artificial_intelligence_software) == References == {{reflist}} == External links == * [https://catalog.ngc.nvidia.com/orgs/nvidia/containers/vllm?version=26.03.post1-py3 vLLM on NVIDIA NGC] * [https://pytorch.org/projects/vllm/ vLLM project page at PyTorch] {{Template:Large language models}} Category:Large language models Category:Natural language processing software Category:Free software programmed in Python Category:2020s software --- Adapted from the Wikipedia article [VLLM](https://en.wikipedia.org/wiki/VLLM) by Wikipedia contributors ([contributor history](https://en.wikipedia.org/wiki/VLLM?action=history)). Available under [Creative Commons Attribution-ShareAlike 4.0 International](https://creativecommons.org/licenses/by-sa/4.0/). Changes may have been made.