The ML System Design Answer Framework

Updated Nov 16, 2026 · 5 min read
On this page

    Most candidates who fail an ML system design round know enough ML. They fail because the answer has no shape: ten minutes on clarifying questions, twenty on a model, and the interviewer's "how would you evaluate this?" arrives with two minutes left. A fixed framework solves that. You stop deciding what to talk about next and spend your energy on the content.

    This page gives you a seven-step framework that works for ranking, recommendation, classification and prediction problems alike, with what to say and roughly how long to spend at each step.

    How it works

    The seven steps in order. Monitoring feeds the next model, so the last step loops back to the first.
    The seven steps in order. Monitoring feeds the next model, so the last step loops back to the first.

    1. Clarify and frame (about 5 minutes)

    Ask a few questions that change the design, then commit to a framing out loud.

    • Goal: what does the business want more of, and what must not get worse? ("More watch time, without more people leaving.")
    • Scale: users, items, requests per second, latency budget.
    • Surface: where the prediction is shown, and what the user can do with it.

    Then turn the goal into an ML objective and a label: "I'll predict the probability that a user watches at least half of a video, and rank by that, mixed with a small weight on likes." Say what you are not optimising and why.

    2. Data and features (about 8 minutes)

    • Training examples: one row per what? (One impression, one transaction, one user–item pair.)
    • Labels: where they come from (clicks, purchases, human review, user reports), how long you wait for them, and their known biases.
    • Features: group them so the list is easy to follow: user features, item features, context features (time, device, location), and cross features (this user's history with this creator).
    • Leakage check: for each feature, ask "would I know this at prediction time?"

    3. Model ladder (about 10 minutes)

    Climb in three rungs, and say why each step is worth its cost:

    • Weak (baseline): a heuristic or a simple model, such as "most popular in your country" or logistic regression on a dozen features. It gives you a number to beat and a fallback when the real model fails.
    • Good: gradient-boosted trees on rich hand-made features. Fast to train, easy to debug, strong on tabular data.
    • Strong: a neural model that learns from raw IDs and sequences (embeddings, a two-tower model for retrieval, a multi-task ranker for several outcomes at once), when there is enough data to justify it.

    4. Training (part of 10 minutes with serving)

    How you split data (by time, never randomly, when the future must be predicted from the past), how you handle class imbalance, how often you retrain, and how a new model gets from training to production.

    5. Serving (part of 10 minutes with training)

    Batch (precompute predictions every few hours) or online (score at request time)? If you rank, show the funnel: cheap retrieval narrows millions of items to hundreds, a heavier model ranks those, and business rules apply last. Give a latency budget per stage.

    6. Evaluation (about 7 minutes)

    • Offline: a metric you can compute on held-out data before launch, chosen to match the label (AUC, precision at a fixed recall, NDCG, mean absolute error).
    • Online: the A/B test. One primary metric that matches the business goal, plus guardrail metrics that must not get worse.

    7. Monitoring and the loop (about 5 minutes)

    What you watch (prediction distribution, feature freshness, label rates, online metric), what triggers retraining or rollback, and how the model's own decisions bias tomorrow's training data.

    The board you end with

    Two halves: an offline pipeline that turns logs into a registered model, and an online path that serves it in milliseconds. Logged outcomes close the loop.
    Two halves: an offline pipeline that turns logs into a registered model, and an online path that serves it in milliseconds. Logged outcomes close the loop.

    Whatever the problem, the final board has the same two halves. The offline half runs on a schedule: logs become labelled training data, a model is trained and evaluated, and the winner is stored in a model registry. The online half runs on every request: the client calls the API gateway, retrieval narrows the candidates, the ranking model scores them with features from a feature store, and business rules make the final cut. Every impression and its outcome are logged, which closes the loop.

    Trade-offs and costs

    • Rigid versus adaptive. The framework is a default, not a script. If the interviewer says "assume the data exists, focus on the model", skip ahead. Following the plan against the interviewer's wishes is worse than having no plan.
    • Breadth first, then depth. A pass through all seven steps in about 25 minutes, then deep dives where the interviewer points. If you go deep on step 3 first, you may never reach step 6.
    • Time per step is a guide. Trust and safety problems spend longer on labels; ranking problems spend longer on serving; pricing problems spend longer on evaluation, because A/B tests on prices are hard.

    When to reach for it

    Use the full framework for any open prompt of the form "Design X", where X makes a decision with a model. For narrower prompts ("how would you evaluate this ranking model?" or "our fraud model's precision dropped, what do you do?"), jump straight to the matching step but still say one sentence about the steps around it, so the interviewer sees you know where it fits.

    Example

    Prompt: "Design a system that flags fraudulent payments."

    1. Frame: block fraud without blocking good customers. Predict the probability that a payment will be charged back within 60 days. Three actions: approve, send to review, block.
    2. Data: one row per payment; label = chargeback or confirmed fraud, available after up to 60 days. Features: amount, merchant, card age, device, velocity (payments from this card in the last hour), distance from the usual location. Leakage check: "refund issued" is only known later, so it is out.
    3. Model ladder: rules (amount over a limit from a new device) → gradient-boosted trees → trees plus graph features linking cards, devices and accounts.
    4. Training: split by time; fraud is about 0.1% of payments, so downsample good payments and reweight; retrain weekly because fraudsters adapt.
    5. Serving: online, inside the payment flow, under 50 ms; features from a feature store with streaming counters.
    6. Evaluation: offline, precision–recall curve and recall at a fixed false-positive rate; online, fraud losses and good-customer decline rate in a holdout.
    7. Monitoring: score distribution, review queue size, chargeback rate by segment; a small random sample of payments is approved regardless of score, so the model still sees what it would have blocked.

    Notice that every step is two or three sentences. That is the breadth pass. The interviewer will choose where to go deep.

    Common mistakes

    1. Skipping step 1. Without an objective, every later choice is arbitrary.
    2. Feature lists with no structure. Thirty features in a random order are hard to follow. Group them.
    3. A ladder with one rung. Naming only the strongest model hides your judgement about cost and benefit.
    4. Random splits on time-ordered data. The model sees the future in training and looks better offline than it will online.
    5. No guardrail metric. "Click-through went up" is not a win if complaints went up too.
    6. Leaving monitoring for the last 30 seconds. Budget five real minutes for it.

    Say this in the interview

    "I'll go in seven steps: frame the problem and the label, data and features, a baseline and how I'd improve on it, training, serving, offline and online evaluation, and monitoring. I'll do a quick pass through all of them, then we can go deep wherever you like."

    Used in