ML System Design Cheat Sheet

Free with account Updated Nov 16, 2026 · 3 min read
On this page

    The night before an ML system design interview, you don't need another long walkthrough. You need the shape of a good answer, the phrases that show judgement, and the traps to avoid. This page is that, on one screen.

    The plan

    Seven steps in about 45 minutes: a quick pass through all of them, then deep dives wherever the interviewer points.
    Seven steps in about 45 minutes: a quick pass through all of them, then deep dives wherever the interviewer points.

    Say this in the first minute:

    "I'll frame the problem and the label, then data and features, a baseline and how I'd improve on it, training, serving, offline and online evaluation, and monitoring. I'll go through all of them quickly, then we can go deep wherever you like."

    1. Frame (5 min)

    • Goal + guardrail: "More satisfying watch time, without more reports."
    • ML objective: "Predict P(watch ≥ 50% | user, video, context)" — the event, the unit and the window.
    • Decision: how the probability becomes an action (rank, threshold, expected-value rule).
    • Say what you're not optimising and why (clicks reward clickbait).

    2. Data and features (8 min)

    • One row = ? One impression, one transaction, one user–item pair.
    • Labels: source, delay, bias (position, selection, noise). Name a human-reviewed sample for truth.
    • Features in four groups: user, item, context, user × item cross. Give two concrete ones per group.
    • Leakage check: "Would I know this at prediction time?" Log features at serving.

    3. Model ladder (10 min)

    • Weak: heuristic or logistic regression; a baseline and a fallback.
    • Good: gradient-boosted trees on hand-made features.
    • Strong: neural model with embeddings, multi-task heads, or a two-tower retrieval model, when there's enough data to justify it.
    • Say why each step is worth its cost.

    4. Training (5 min)

    • Split by time. Never random for future prediction.
    • Imbalance: downsample negatives, then correct probabilities and calibrate.
    • Cadence: online, daily or weekly, chosen by how fast the data changes.
    • Gates: data validation, beat production on the same test set per segment, shadow, canary, rollback.

    5. Serving (5 min)

    • Batch or online, and which parts of each.
    • Funnel sizes: millions → ~1,000 (retrieval) → ~100 (ranking) → page (re-rank).
    • Latency budget per stage, and a fallback for each failure.

    6. Evaluation (7 min)

    Pick the offline metric that matches the label and the decision, and an online metric that matches the business goal, plus guardrails.
    Pick the offline metric that matches the label and the decision, and an online metric that matches the business goal, plus guardrails.
    • Offline metric that matches the label (table above).
    • A/B test: randomisation unit, one primary metric, guardrails, duration (at least a week or two), sample size.
    • Interference: social and marketplace products need cluster or switchback designs.

    7. Monitoring (5 min)

    • Four layers: system, inputs, predictions, outcomes, by segment.
    • Drift: data drift (inputs move) vs concept drift (meaning moves).
    • Feedback loops: position bias, exploration slot, random holdout, long-term holdout.

    Numbers worth remembering

    These are rough, interview-scale numbers to sanity-check your design (always state them as assumptions):

    QuantityRough value
    Requests a second from daily requestsper day ÷ 86,400 (≈ ÷ 100,000), peak ≈ 2–3× average
    64-float embedding256 bytes; 100 M items ≈ 26 GB
    Product-quantised vector~32–64 bytes
    Tree model, CPU~10,000–50,000 scorings a second per core (example)
    Retrieval stagetens of ms for ~1,000 candidates
    Users per arm to detect a 1% relative lift on a 5% rate~3 million
    Downsampling correctionp = p′ / (p′ + (1 − p′) / w)