Supervised Fine-tuning: Data and Recipe

Updated Nov 16, 2026 · 4 min read
On this page

    A team fine-tunes an open model on 20,000 support conversations exported from their helpdesk. The result is worse than the base model: it apologises constantly, invents order numbers, and sometimes continues the user's message instead of answering. Nothing was wrong with the training code. The data contained every agent's bad habits, the prompt formatting didn't match the model's chat template, and the loss was trained on the user's words as well as the answers. Supervised fine-tuning (SFT) is mostly a data problem, and interviewers want to hear that.

    How it works

    SFT continues training a model on examples of input → desired output, with the ordinary next-token loss. The model learns to imitate the outputs: their format, tone, length and the skill they show.

    One example in chat format. The loss is computed only on the assistant's answer; the system and user parts are context.
    One example in chat format. The loss is computed only on the assistant's answer; the system and user parts are context.

    The data

    • Format: each example is a conversation (system, user, assistant turns) or a prompt–completion pair, rendered with the exact chat template of the base model (its special tokens and role markers). A mismatch silently lowers quality at inference time.
    • Quality over quantity: a few hundred to a few thousand clean, consistent examples often beat tens of thousands of noisy ones. Every answer should be one you would be happy to see in production.
    • Coverage: the mix should match the real traffic: easy and hard cases, edge cases, refusals where needed, and multi-turn examples if the product is multi-turn.
    • Sources: expert-written answers, filtered production logs, or answers generated by a stronger model and reviewed (check that the stronger model's licence allows it).

    The training

    • Loss masking: compute the loss only on the assistant tokens (in Hugging Face, prompt tokens get the label -100). Otherwise the model also learns to write user messages.
    • Method: LoRA or QLoRA for most work; full fine-tuning when you have the compute and need bigger changes.
    • Typical settings: 1–3 epochs; a learning rate around 1e-5 for full fine-tuning and around 1e-4 to 2e-4 for LoRA; warm-up and cosine decay; sequence packing to use the GPU well. Treat these as starting points and tune on the eval set.
    • Watch the eval loss and task metrics, not just the training loss: training loss keeps falling while the model memorises.
    Most iterations are data fixes: read the failures, fix or add examples, train again.
    Most iterations are data fixes: read the failures, fix or add examples, train again.

    Trade-offs and costs

    • More epochs fit the format better but overfit sooner and cause more forgetting of general ability.
    • Narrow data gives a strong specialist that may get worse at everything else (see catastrophic forgetting).
    • Imitation copies everything: the model learns the mistakes, verbosity and made-up details in the examples as faithfully as the good parts.
    • Cost is usually small compared with data collection and review time.

    In production

    • Version everything: data set, base model, hyperparameters, and the resulting adapter or checkpoint.
    • Hold out an eval set of real cases, plus a general-ability set, plus safety prompts. Compare against the base model with a good prompt.
    • Keep a data pipeline, not a one-off export: new failure cases from production become new training examples.
    • Re-run the fine-tune when the base model is upgraded; the old adapter won't transfer.

    When to reach for it

    • Prompting gets the behaviour right only some of the time, and you have or can create good examples.
    • You want a consistent output format or schema, a house style, or a narrow skill (classification, extraction, your tool-calling format).
    • You want a smaller model to match a larger one on one task (often combined with distillation).
    • Not for adding knowledge that changes; use retrieval.

    Example

    The support team starts again. They pick 2,500 conversations rated highly by customers, have two senior agents edit the answers into the house style, and remove anything with order numbers or personal data. They add 300 examples of polite refusals and hand-offs to a human. They render everything with the model's chat template, mask the user turns, and train a LoRA for 2 epochs. On a held-out set of 200 real tickets graded by agents, the fine-tuned model beats the prompted base model on tone and correctness, and a general-ability check shows no meaningful drop.

    Common mistakes

    • Training on raw logs without filtering, so the model learns the bad answers too.
    • Wrong chat template or missing end-of-turn tokens, so the model never stops or rambles.
    • No loss masking, so the model learns to generate the prompt.
    • Too many epochs on a small data set, then wondering why it forgot general skills.
    • Evaluating on training examples, or with no baseline.

    Say this in the interview

    "SFT trains the model with next-token loss on input–output examples so it imitates the outputs. The hard part is the data: a few thousand clean, consistent, representative examples beat a large noisy dump. I'd use the base model's exact chat template, mask the loss to the assistant tokens, train a LoRA for one to three epochs with a small learning rate, and evaluate on a held-out set of real cases plus a general-ability set, compared against the prompted base model."

    Interview questions on this

    1. How much data do you need? Depends on the task; for format or a narrow skill, hundreds to a few thousand high-quality examples is often enough. Measure with a learning curve.
    2. Why mask the prompt tokens? So the loss teaches the model to answer, not to predict user text.
    3. Your fine-tuned model got worse at general questions. Why? Forgetting from narrow data or too many epochs; mix in general data, use LoRA, lower epochs.
    4. Can you use a stronger model to write the training data? Yes, with review and licence checks; this is a form of distillation.
    5. What's the difference between SFT and preference tuning? SFT imitates good answers; preference tuning learns from comparisons of better and worse answers.

    Used in

    • AI Engineering: Fine-tune, RAG or Prompt?; LoRA and QLoRA; Preference Tuning: RLHF and DPO; Evaluating a Fine-tuned Model and Catastrophic Forgetting; Synthetic Data for LLMs; Labeling and Preference Data.
    • AI System Design: Design a Fine-tuning Platform.