What happens after pretraining. Supervised fine-tuning, LoRA and parameter-efficient methods, preference optimization from RLHF through DPO and its successors, reinforcement learning with verifiable rewards, and synthetic data.
Fine-tuning 4-bit models is a different problem from QLoRA over a BF16 base: what precision to train the adapter in, what to merge into, and what you deploy.
Thinking Machines Inkling debuted at Artificial Analysis index 41, behind Kimi K3's 57. For an open-weights fine-tuning base, rank is the wrong axis. What to measure instead.
Synthetic data for post-training: when distillation helps, when self-generated data triggers model collapse, and how to detect the narrowing before it ships.
Test-time compute scaling explained: best-of-N, self-consistency, and verifier-guided search, where each saturates, and when more inference compute is wasted.
Explore how reasoning models like o1, o3, and DeepSeek-R1 use inference-time compute scaling and chain-of-thought to solve problems standard LLMs cannot.
Runnable comparison of preference optimization after DPO: IPO, KTO, and SimPO. What each objective changes, when reference-free wins, and how to switch in TRL.
Hands-on RLVR tutorial: build a verifiable-reward loop with GRPO in TRL, write a math verifier, and learn the DAPO and GSPO fixes that keep it stable.
Learn LoRA fine-tuning step by step: the math behind low-rank adaptation, QLoRA quantization, Unsloth training, hyperparameter selection, and practical code for consumer GPUs.
Latent reasoning lets LLMs think in hidden states instead of tokens. A research review of Coconut, Abstract CoT, and SWITCH, and why it is not production-ready yet.
Explore how self-correcting AI architectures using RLHF, GRPO, constitutional AI, and self-play feedback loops are driving progress toward AGI through recursive self-improvement.