105. Model Weights 4 - Fine-tuning, LoRA and What Actually Changes

The lesson

Fine-tuning updates the weights using your own examples. Full fine-tuning adjusts every parameter, needs the memory to hold gradients and optimizer state alongside the weights, and produces a complete new model. For a 7-billion-parameter model that is several times the inference memory, which is why most practitioners never do it on their own hardware.

LoRA avoids that by freezing the original weights and training a small pair of low-rank matrices inserted beside chosen layers. The adapter is typically a few million parameters rather than billions, so it trains on modest hardware, and it can be stored as a small file and applied or removed at load. The base model is unchanged, which also means one base can serve many adapters.

QLoRA goes further: the frozen base is loaded in 4-bit, and the adapters train in higher precision on top. That is what makes fine-tuning a large model possible on a single consumer card. The quality cost is small in practice for many tasks, and the technique is widely used for exactly that reason.

Fine-tuning is the wrong tool more often than people expect. It teaches style, format and task shape well; it is a poor way to add facts, which change and which the model will still misremember. For project-specific knowledge, retrieval or a well-filled context is usually cheaper, immediate and easier to correct. Training also freezes a snapshot: the moment your data changes, the model is stale in a way you cannot patch.

If you do fine-tune, treat it like any other engineering change. Hold out evaluation data before training, not after. Record the base model's exact version and hash, because adapters do not transfer reliably across base revisions. Check the licence of the base weights for derivative and commercial terms. And compare against a strong prompting baseline on the same evaluation set, because a disappointing adapter that beat nothing is a common and expensive result.

Check yourself