Post-Training & Alignment

What happens after pretraining. Supervised fine-tuning, LoRA and parameter-efficient methods, preference optimization from RLHF through DPO and its successors, reinforcement learning with verifiable rewards, and synthetic data.

Articles

Post-Training19 min read

Reasoning Models: How LLMs Learned to Think Before They Speak

Explore how reasoning models like o1, o3, and DeepSeek-R1 use inference-time compute scaling and chain-of-thought to solve problems standard LLMs cannot.

Roei ZAPR 6, 2026

Tutorials

Advanced2 to 3 hours

RLVR in Practice: Build a Verifiable-Reward Loop with GRPO

Hands-on RLVR tutorial: build a verifiable-reward loop with GRPO in TRL, write a math verifier, and learn the DAPO and GSPO fixes that keep it stable.

Prerequisites

Familiarity with PyTorchthe Hugging Face stack (transformerspeftdatasets)a single 24GB+ GPUand a working mental model of supervised fine-tuning. No prior RL experience required.

Intermediate2-3 hours

Fine-Tuning Transformer Models with Low-Rank Adaptation (LoRA)

Learn LoRA fine-tuning step by step: the math behind low-rank adaptation, QLoRA quantization, Unsloth training, hyperparameter selection, and practical code for consumer GPUs.

Prerequisites

Python proficiencyPyTorch basicsUnderstanding of transformer architecture

Research

Neural Symbiosis: The Path to AGI via Recurrent Feedback Loops

Explore how self-correcting AI architectures using RLHF, GRPO, constitutional AI, and self-play feedback loops are driving progress toward AGI through recursive self-improvement.

Roei ZJUN 2, 2026

Post-Training

Key Terms

Alignment Best-of-N Sampling Constitutional AI DAPO (Decoupled Clip and Dynamic Sampling Policy Optimization)DPO (Direct Preference Optimization)Distillation DoRA (Weight-Decomposed Low-Rank Adaptation)Fine-tuning GRPO (Group Relative Policy Optimization)GSPO (Group Sequence Policy Optimization)IPO (Identity Preference Optimization)Inference-Time Compute KTO (Kahneman-Tversky Optimization)LoRA (Low-Rank Adaptation)Model Collapse PRM (Process Reward Model)QLoRA (Quantized LoRA)RLHF (Reinforcement Learning from Human Feedback)RLVR (Reinforcement Learning with Verifiable Rewards)Reasoning Models Reward Hacking SFT (Supervised Fine-Tuning)Self-Consistency SimPO (Simple Preference Optimization)Synthetic Data