KV Cache & Continuous Batching
The two ideas that separate a naive inference loop from what production LLM serving actually does.
Because attention is causal (see The Transformer Architecture), the key and value vectors for every already-generated token never change as generation continues — only the query for the newest token needs to look back at them. Without caching, generating token N would require recomputing K/V for all N-1 prior tokens from scratch, at every single step, which is wasteful and gets worse the longer the output. The KV cache stores those K/V vectors once and reuses them for every subsequent step.
# without a KV cache: O(n^2) redundant work across a full generation
# with a KV cache: each new token computes K/V once and appends to the cache
kv_cache = {}
for step in range(max_new_tokens):
k, v = compute_kv(new_token)
kv_cache["k"].append(k); kv_cache["v"].append(v)
logits = attend(query(new_token), kv_cache["k"], kv_cache["v"])
new_token = sample(logits)
# per sequence, per token:
kv_bytes_per_token = 2 * num_layers * num_kv_heads * head_dim * bytes_per_element
# ^2 for K and V
# example: a 32-layer model, 8 KV heads, head_dim=128, FP16 (2 bytes)
per_token = 2 * 32 * 8 * 128 * 2 # = 131,072 bytes ≈ 128 KB / token
# a single 8,000-token conversation: ~1 GB of KV cache, for ONE request
Naive batching waits for every request in a batch to finish before starting the next batch — so a batch with one long request and several short ones leaves the GPU idle on the short requests' slots until the long one finishes. Continuous batching (also called in-flight batching) instead evicts a finished sequence from the batch and immediately admits a new waiting request into its slot, every generation step, keeping the GPU near full utilization instead of bottlenecked by the slowest request in a static batch.
Before PagedAttention, each sequence's KV cache was typically allocated as one contiguous memory block sized for the maximum possible sequence length — wasting most of that memory for the many requests that finish well short of the max. PagedAttention (introduced by vLLM) borrows the idea of OS virtual memory paging: KV cache is allocated in small fixed-size blocks on demand, non-contiguously, with a block table mapping logical sequence positions to physical blocks. This nearly eliminates internal fragmentation and is a large part of why vLLM achieves substantially higher throughput than a naive implementation at the same GPU memory budget.