Fine-Tuning
Adapting a pre-trained model's weights on your specific data for customised behaviour.
Simple explanation
Fine-tuning means training the model further on your data so it learns your style, domain or format. Unlike RAG (which adds knowledge at runtime), fine-tuning bakes behaviour into the weights.
Engineering explanation
Methods: full fine-tuning (all parameters — expensive), LoRA (low-rank adaptation — 0.1% of params), QLoRA (quantised LoRA — 4-bit, single-GPU). Fine-tune for: style, domain behaviour, format, private languages. DON'T fine-tune for: knowledge updates (use RAG), one-off tasks (use prompting), < 500 examples (use few-shot). Risk: catastrophic forgetting (general capability drops).
Example
Fine-tune Llama 3.1 8B on 1000 examples of your brand voice → model consistently writes in your style. Combine with RAG for knowledge.
When to use it
When prompting can't reliably achieve the style/behaviour/format you need, and you have 1k+ examples. Not for knowledge.
Common misconception
Fine-tuning is NOT a replacement for RAG. Fine-tune for style, RAG for knowledge. They're complementary. Most teams should start with RAG + prompting.