Fine-Tuning Fundamentals
LoRA, QLoRA, full fine-tuning, when to fine-tune vs RAG, and the economics of customisation.
What is fine-tuning?
Fine-tuning adapts a pre-trained model's weights on your specific data. Full fine-tuning updates all parameters (expensive — needs many GPUs, lots of data). Parameter-Efficient Fine-Tuning (PEFT) methods like LoRA (Low-Rank Adaptation) and QLoRA (Quantised LoRA) update a tiny fraction of parameters (0.1-1%) — cheaper, faster, and often nearly as effective.
The critical distinction from RAG: RAG injects knowledge at inference time (updatable, no weight changes). Fine-tuning bakes behaviour/style/knowledge into the weights (faster inference, lower per-call cost, but slow to update). They are complementary, not alternatives.
Why fine-tuning matters (and when it doesn't)
Fine-tune when: (1) you need a specific output style (brand voice, format) that prompting can't reliably achieve, (2) you need domain-specialised behaviour (medical, legal, code) that base models handle poorly, (3) you need lower latency/cost at inference (a fine-tuned 7B can outperform a prompted 70B on a narrow task). DON'T fine-tune for: knowledge updates (use RAG), one-off tasks (use prompting), or when you have < 500 examples (use few-shot).
Fine-tuning pipeline
Training data (input-output pairs) → base model → PEFT adapter (LoRA) → trained adapter → merge or serve alongside base → eval → deploy. For QLoRA: quantise base model to 4-bit, train LoRA adapters, serve with the quantised base + adapter. Eval is critical — fine-tuning can regress on general capability while improving on the target task.
LoRA fine-tuning with PEFT
from peft import LoraConfig, get_peft_model
from transformers import AutoModelForCausalLM, TrainingArguments, Trainer
from datasets import Dataset
import torch
# 1. Load base model (quantised for QLoRA)
base_model = "meta-llama/Llama-3.1-8B-Instruct"
model = AutoModelForCausalLM.from_pretrained(
base_model,
load_in_4bit=True, # QLoRA: 4-bit quantisation
device_map="auto",
)
# 2. LoRA config — only train these adapter weights
lora_config = LoraConfig(
r=16, # rank — higher = more capacity, more params
lora_alpha=32,
target_modules=["q_proj", "v_proj", "k_proj", "o_proj"],
lora_dropout=0.05,
task_type="CAUSAL_LM",
)
model = get_peft_model(model, lora_config)
model.print_trainable_parameters()
# trainable: 8M / 8B = 0.1% of parameters
# 3. Training data: input-output pairs for your task
dataset = Dataset.from_json("training_data.jsonl")
# 4. Train
training_args = TrainingArguments(
output_dir="./lora-adapter",
num_train_epochs=3,
per_device_train_batch_size=4,
learning_rate=2e-4,
save_steps=100,
eval_steps=100,
)
trainer = Trainer(model=model, args=training_args, train_dataset=dataset)
trainer.train()
# 5. Eval: compare fine-tuned vs base on your eval set
# CRITICAL: ensure the fine-tuned model didn't regress on general tasks
eval_results = run_eval(fine_tuned_model, golden_set)
assert eval_results.faithfulness >= baseline.faithfulness - 0.05
# 6. Save the adapter (small — MB not GB)
model.save_pretrained("./lora-adapter")
# Serve: base model + this adapterExperiment: fine-tune vs RAG vs prompting
See which customisation approach wins for each scenario.
What to observe
Fine-tuning is NOT a replacement for RAG — they solve different problems. Fine-tune for: style, domain behaviour, format, private languages. Use RAG for: knowledge, frequently-updated content. Combine them: fine-tune the model's behaviour, RAG for the knowledge. ALWAYS eval after fine-tuning — models can regress on general capability.
Production fine-tuning
Production fine-tuning: LoRA/QLoRA (not full FT unless you have 100k+ examples and budget), eval gate comparing fine-tuned vs base on BOTH target task AND general capability (avoid catastrophic forgetting), version adapters, serve via vLLM with adapter hot-loading, monitor for drift, and a rollback path to the base model. Training data quality matters more than quantity — 1k curated examples beats 10k noisy ones.
Challenge
Your fine-tuned model nails the target task (95% accuracy) but general capability dropped 20% (catastrophic forgetting). How do you get the target improvement without the regression? (Hint: LoRA rank, regularisation, mixing general data into training, or DPO.)
Production checklist
Production checklist
0 of 10 checked
Knowledge check
Your product docs update weekly. Users want the assistant to answer from them. Fine-tune or RAG?
Complete
You can now decide when and how to fine-tune. Fine-tuning is a powerful tool — but RAG is the right answer 80% of the time.
Mark this chapter as complete
Track your progress and unlock the next chapter.