[LLM 2/10] SFT + LoRA: Teaching a Model to Be an Assistant by Training 1.69% of Its Parameters

Last chapter we used Continue Pretraining to put knowledge into the model's weights. But a model that knows is not a model that answers — a base model has exactly one job: continuing text. This chapter teaches SFT (Supervised Fine-Tuning) with LoRA: a technique that trains only about 1.7% of the parameters yet changes the behavior of the entire model — finished in ~15 minutes on free Colab. What you get back is an adapter file of roughly 40 MB that becomes the backbone of the rest of the series.

Open in Colab02_sft_lora.ipynb

Thai LLM seriesPart 2 of 10

1. 1Continue Pretraining
2. 2SFT and LoRA(you are here)
3. 3RLHF and PPO
4. 4DPO: Direct Preference Optimization
5. 5GRPO
6. 6Context Distillation
7. 7Model Distillation
8. 8Guardrails
9. 9Benchmarking
10. 10Deployment

1. The Problem

Take a true base model like Qwen3-0.6B-Base from the last chapter and type in (in Thai) "Could you recommend some Thai dishes, please?" What comes back is usually not an answer but a continuation — it may compose three more questions, keep going as a travel article, or switch into English partway through, because the only thing it was ever trained on is "on the internet, what usually follows text like this?"

The ability to answer — take an instruction, respond to the point, then stop — does not come with pretraining. It comes from SFT: continued training on (instruction, good answer) pairs, thousands to millions of them. Every instruct model you have ever used has been through this stage.

But the moment you try it yourself, you hit two problems stacked on top of each other:

Layer one — the cost of full fine-tuning. Train every parameter and you get an entire new model (~1.2 GB per task for a 0.6B model). An organization with ten tasks — document summaries, letter drafting, customer replies, complaint triage — has to store ten copies. And moving every single weight with a high learning rate is a recipe for erasing the knowledge you just added in chapter 1 (remember the learning-rate box?).

Layer two — Thai. Even the post-trained Qwen3-0.6B shows the symptom we will see all series long: ask in Thai, and the answer drifts into English mid-sentence (this is where the series' th_ratio metric comes from), because the SFT data it saw was overwhelmingly English.

This chapter fixes both layers at once: SFT on Thai instruction data, done through LoRA instead of full fine-tuning.

2. What We're Going to Do

We take Qwen3-0.6B and train it on 4,000 Thai instruction-answer pairs, using the exact same loss as chapter 1 plus two new pieces:

1. Completion mask — compute the loss only on the answer tokens, never the question (section 3.1 explains the comically broken failure you get if you skip this)
2. LoRA (Low-Rank Adaptation) — freeze all the original weights and instead train two small matrices laid over each layer

The core idea of this chapter

You are not training the model's weights — you are training a low-rank "correction" laid on top of the original weights.

This is why the adapter is only ~40 MB, not 1.2 GB, why you can keep twenty adapters and swap them on a single base (twenty tasks = 0.8 GB, not 24 GB), and why the reference model in chapter 4 (DPO) will cost zero additional bytes of VRAM — switch the adapter off and you get the starting model back exactly.

3. The Equations

3.1 The SFT loss and the completion mask

$$\mathcal{L}_{\text{SFT}}(\theta) = -\mathbb{E}_{(x,y)}\left[\sum_{t=1}^{|y|} m_t \log p_\theta(y_t \mid x, y_{<t})\right]$$

• $(x, y)$ = one training pair — $x$ is the instruction part (chat template included) and $y$ is the token sequence of the example the model actually sees during training
• $y_t$ = the token at position $t$, and $y_{<t}$ = every token before it
• $p_\theta$ = the probability predicted by the model with parameters $\theta$
• $m_t \in \{0,1\}$ = the completion mask — 1 only on answer-side tokens, and 0 on prompt tokens

Remove $m_t$ (i.e. set $m_t \equiv 1$ everywhere) and this equation instantly becomes the CPT objective from chapter 1. SFT is CPT on text staged as a conversation, plus one mask — nothing more.

But that one mask is half the battle, because $m_t \equiv 1$ is the easiest default to fall into (many pipelines, SFTTrainer included, will silently train this way if the collator isn't wired correctly). And in typical Thai instruction data, prompt-side tokens make up around 60% of each example — meaning most of your gradient is busy teaching the model to write the user's questions, not to answer them. The side effect this causes will make its appearance in section 9.

3.2 LoRA: training the correction, not the weights

Instead of updating the weight matrix $W_0$ directly, LoRA pins $W_0$ in place and learns a difference that is the product of two small matrices:

$$W' = W_0 + \Delta W = W_0 + \frac{\alpha}{r}\,B A$$

• $W_0 \in \mathbb{R}^{d\times k}$ = the layer's original weights, frozen — they receive no gradient at all
• $B \in \mathbb{R}^{d\times r}$ and $A \in \mathbb{R}^{r\times k}$ = the two adapter matrices we actually train
• $r \ll \min(d, k)$ = the rank of the correction — LoRA's main dial (this chapter uses $r = 16$)
• $\alpha$ = a scale multiplier — the product $BA$ is always multiplied by $\alpha/r$ (here $\alpha = 32$, so $\alpha/r = 2$)

Two details in this definition matter more than they look:

$B$ is initialized to all zeros, so $\Delta W = 0$ at the first step — training starts from exactly the base model, with no period where random weights disturb it. ($A$ is random Gaussian — set both to zero and both gradients stay zero forever, since each is multiplied by the other's zeros.)

The divisor $r$ in $\alpha/r$ makes the update magnitude independent of the rank — double $r$ and the sum $BA$ has twice as many terms, but it gets divided right back. So you can sweep $r$ without re-tuning the learning rate every time.

And when training finishes, you have two choices: merge ($W' = W_0 + \frac{\alpha}{r}BA$, giving a single model with no added latency) or keep it separate — the second is the road this series takes, because a detachable adapter is what chapter 4 uses to build a reference model for free.

3.3 The fraction of parameters trained — a number to check, not to recite

For a single $d \times k$ matrix, the adapter has $rd + rk$ parameters, a fraction of

$$\frac{r(d+k)}{dk}$$

• $d, k$ = the dimensions of the original weight matrix
• $r$ = the adapter's rank

Plug in the real values for Qwen3-0.6B (hidden 1024, intermediate 3072, 28 layers, adapters on all 7 matrices: q, k, v, o, gate, up, down) and you get 10,092,544 trainable parameters on a base of 596,049,920 = 1.69%.

1.69% is not "under 1%" — a lesson in checking numbers

Nearly every LoRA article says it "trains under 1% of the parameters." That number is true at 7B scale and up, but not for small models, because adapter parameters grow as $r(d+k)$ — linear in the hidden size — while base parameters grow as $dk$ — quadratic. The smaller the model, the larger a fraction the adapter becomes.

Notice too that 1.69% is lower than the per-matrix fraction (~2.1–2.3%) — because the denominator includes roughly 156 million embedding parameters we attach no adapter to. All of this can be checked with plain arithmetic, and in section 7 the notebook has peft print the real value for you to see with your own eyes — trust the print, not a blog (this one included).

The full lesson is in the course

This post is roughly the first 30% of the chapter. The rest — environment setup, data preparation, the main code, measured results and the wrap-up — is in the free LLM Finetuning course. Sign in with Google to read it.

Read the full lesson in the course →