Research paper🇺🇸
2022 paper introducing exact, memory-efficient attention on GPUs, now used in many training and inference stacks.
Efficient Memory Management for Large Language Model Serving with PagedAttention (vLLM) research paper summary (October 2026)
SOSP 2023 paper introducing PagedAttention and the vLLM serving system for running large language models.
What is Efficient Memory Management for Large Language Model Serving with PagedAttention (vLLM)?
This 2023 paper from UC Berkeley researchers introduces PagedAttention, a way to manage the memory of large language models during inference, and vLLM, the serving system built on it. It is the reference paper behind one of the most used open source LLM inference engines.
The paper, presented at SOSP 2023, shows that the key-value cache of each request wastes memory through fragmentation and duplication, which limits batch size. PagedAttention borrows paging from operating systems and stores the cache in fixed blocks that can be shared.
The code lives on in vLLM, which the README describes as maintained by a community of academic institutions and companies with more than 2,000 contributors. The repository had 92,862 GitHub stars and was updated on September 28, 2026.
Efficient Memory Management for Large Language Model Serving with PagedAttention (vLLM) AI features and use cases
- Explains why self-hosted LLMs get cheaperThe paper shows how better cache memory management lets one GPU serve more requests at once. IT and platform teams can use it to understand where inference costs come from before sizing hardware for internal chat or document tools.
- Leads to a production serving engineThe linked vLLM repository turns the method into an OpenAI-compatible API server with batching, quantization and support for many model families. Teams can run open-weight models on their own infrastructure instead of sending data to external APIs.
- Supports shared prompts efficientlyBlock sharing lets parallel sampling and requests with the same prefix reuse cache memory. Applications that send the same long system prompt with every request save memory and time.
What sets Efficient Memory Management for Large Language Model Serving with PagedAttention (vLLM) apart
- Peer-reviewed at SOSP 2023, a major operating systems conference.
- The accompanying vLLM repository is among the most active open source AI projects.
- Method applies to many model architectures, not a single model.
Where Efficient Memory Management for Large Language Model Serving with PagedAttention (vLLM) is strongest
Available internationally
The paper is freely available on arXiv, and the accompanying code runs on your own hardware in any country.
Questions to ask before you buy Efficient Memory Management for Large Language Model Serving with PagedAttention (vLLM)
The paper describes the 2023 design; vLLM has changed a lot since, so use the current documentation for deployment decisions. Running vLLM yourself means your team handles GPUs, updates and security, while data stays on your infrastructure.
Efficient Memory Management for Large Language Model Serving with PagedAttention (vLLM) reviews
No reviews yet. Be the first to share how your team uses it.
Every review shows its verification level. We read each one before it goes live. How we check reviews
Questions about Efficient Memory Management for Large Language Model Serving with PagedAttention (vLLM)
Is the code free to use in a company?
Yes. vLLM is under the Apache 2.0 license. The models you serve with it have their own licenses.
Who maintains vLLM today?
The README says vLLM started at UC Berkeley's Sky Computing Lab and is now maintained by a community of academic institutions and companies with more than 2,000 contributors.
Where was the paper published?
At SOSP 2023, the ACM Symposium on Operating Systems Principles. The arXiv version dates from September 12, 2023.
Efficient Memory Management for Large Language Model Serving with PagedAttention (vLLM) update historyno updates yet
No changes recorded for Efficient Memory Management for Large Language Model Serving with PagedAttention (vLLM) yet.
Changes the vendor made and confirmed user reports, each checked against the vendor's own pages. We review every tool monthly.
Alternatives to Efficient Memory Management for Large Language Model Serving with PagedAttention (vLLM)
Tools in the same category (Language models) with the most overlapping markets.
Research paper🇨🇳
Paper from Beijing (Tsinghua and ModelBest) on small language models with 1.2 and 2.4 billion parameters and scalable training strategies.