Making models fit. Low-bit weight and activation formats, quantization-aware training versus post-training quantization, what each bit width costs in quality, and the memory arithmetic behind running large models on small hardware.
Fine-tuning 4-bit models is a different problem from QLoRA over a BF16 base: what precision to train the adapter in, what to merge into, and what you deploy.
Self-hosting a frontier open MoE like DeepSeek V4 or Kimi K3 means holding all 1.6-2.8T parameters in VRAM, not the active count. The real GPU bill and when the API wins.
The KV cache is the real bottleneck in LLM serving: 70-90% of GPU memory at long context. PagedAttention, prefix caching, MLA, and quantization explained.
A technical guide to LLM quantization: FP8 training, NVFP4 and MXFP4, W4A4 inference, the outlier problem, and where low-bit precision quietly breaks accuracy.
A hands-on guide to running LLMs locally in 2026: install Ollama, verify the API, then build llama.cpp and serve with vLLM, with the VRAM and bandwidth math behind each step.
Learn LoRA fine-tuning step by step: the math behind low-rank adaptation, QLoRA quantization, Unsloth training, hyperparameter selection, and practical code for consumer GPUs.