A large language model that handles Thai reasonably well will still, more often than not, know nothing about your organization's specialized knowledge — not Thai government regulations, not the jargon of your industry, not your internal documents. This article covers the most direct fix: Continue Pretraining (CPT), from the equations all the way to code that runs end to end on free Colab in about 15 minutes.
Last chapter we used Continue Pretraining to put knowledge into the model's weights. But a model that knows is not a model that answers — a base model has exactly one job: continuing text. This chapter teaches SFT (Supervised Fine-Tuning) with LoRA: a technique that trains only about 1.7% of the parameters yet changes the behavior of the entire model — finished in ~15 minutes on free Colab. What you get back is an adapter file of roughly 40 MB that becomes the backbone of the rest of the series.
In chapter 2 we taught the model by "imitating the answer key" one token at a time. But the qualities that make an AI assistant genuinely usable — correct, polite, not making things up, not sliding into English — have no answer key to imitate, and cannot be written directly as a loss function. This chapter is the most classical answer to that problem: RLHF (Reinforcement Learning from Human Feedback) with PPO. We will train a real reward model from Thai preference pairs, then write the PPO loop ourselves from scratch in about 120 lines, and close with my favorite experiment in the series: taking off the KL leash and watching the model cheat the reward live. This is deliberately the heaviest chapter of the series, because chapter 4 (DPO) and chapter 5 (GRPO) both start from this chapter's equations and each choose a different piece to delete.
In the last chapter we did RLHF with PPO, and you saw how many moving parts it has — a separate reward model to train, four models resident in VRAM at once, a dozen-plus PPO hyperparameters to tune, and if the reward model is off, the policy finds a shortcut that games the score. In this chapter we'll accomplish the same thing with an ordinary supervised training loop — no reward model, no RL. And crucially, this is not an approximation. We'll prove algebraically that those two pieces genuinely cancel out.
The last chapter ended on DPO's gap: it can only rank answers someone prepared in a file. And chapter 3 paid PPO's full price: 4 models in VRAM, plus an entire value network to train on the side. This chapter combines the good halves of both — the model samples its own answers to learn from, genuine RL — but deletes the value network wholesale, using a statistical observation so simple it's almost irritating nobody framed it sooner: if you sample several answers to the same problem, the group's mean reward is already the baseline the value network was trying to estimate. And if the problem can be graded by code, we need no human preference data at all — zero pairs, zero baht.
Every time a user messages your chatbot, you attach the same system prompt, hundreds of tokens long — every request, for the lifetime of the system, paid again and again with no end. This chapter moves that block of knowledge from the prompt into the model's weights, using a technique called Context Distillation in its on-policy version (OPCD). The most beautiful part: the teacher and the student are the exact same model — the only thing that differs is who gets to see the prompt.
In chapter 6 we distilled a context into the weights of the same model. In this chapter we distill an entire model into a smaller one. The two chapters are a deliberate pair: context distillation changes what the model knows without having to tell it anymore — model distillation changes the size of the model while trying not to change what it can do. And the heart of this chapter cuts hard against intuition: the most valuable thing a teacher can hand its student is not the right answer, but the way the teacher is wrong — and the dial called temperature is what makes that visible.