#vLLM

7 articles tagged with vLLM

High-Throughput LLM Serving: Continuous Batching & PagedAttention Internals in vLLM

An enterprise LLM infrastructure engineering guide to high-throughput model serving. We dissect continuous iteration-level batching, PagedAttention virtual memory management, prefill-decode disaggregation, and maximizing tokens/sec/dollar.

9/2/202624 min read

Speculative Decoding in 2026: Medusa & Eagle Multi-Head Speculation for 3x Faster LLM Inference

A comprehensive systems engineering guide to speculative decoding. We evaluate draft-model speculation, Medusa multi-head tree attention, Eagle contextual recurrent drafting, and achieving 3.2x tokens/sec throughput with mathematically identical outputs.

9/2/202624 min read

Open Weight Models Explained: Recipe Books You Can Bake at Home (Llama, DeepSeek) (2026)

9/12/202615 min read

SGLang vs vLLM: RadixAttention & Structured JSON Generation (2026)

9/12/202617 min read

vLLM PagedAttention v3 vs TensorRT-LLM: KV Cache Optimization Deep Dive (2026)

9/12/202618 min read

Open-Weight LLM Benchmark 2026: Moonshot Kimi K3 vs DeepSeek V4 vs Meta Llama 3.3

9/12/20265 min read

EAGLE-2 & Speculative Decoding in vLLM: Tree Attention Verification & 3.5x Speedups (2026)

9/12/20265 min read