LLM Inference in Production
A practical route through the serving stack. Start with the map of where inference cost and latency actually come from, take the quantization lever apart, speed up decoding with speculative drafts and measure it yourself, confront what long context really delivers, and finish at the 2026 state of the art. Every step names its tradeoffs.
Intermediate5 pieces~1.4 hours total
1
Article·18 min read
LLM Inference Optimization: The Engineering Behind Fast, Cheap AI
The map of the territory: where latency and cost actually go when serving LLMs.
2
Article·19 min read
Quantization Deep Dive: FP8 Training, FP4, and the Outlier Problem
The biggest single cost lever: what low-bit formats buy you, and where they break.
3
Article·17 min read
Speculative Decoding in vLLM: A Practical Guide to Faster LLM Inference
Hands-on: measure the decoding speedup yourself — including the batch-size catch.
4
Article·16 min read
Effective Context Length: Why 1M-Token Windows Fall Short, and When RAG Still Wins
The context window on the spec sheet vs. the context the model actually uses.
5
Article·11 min read
DeepSeek DSpark: What Semi-Autoregressive Speculative Decoding Actually Changes
Where speculative decoding moved in 2026, and which vendor numbers to treat with care.