How ML System Design Interviews Work
On this page
You know how to draw a load balancer, a cache and a sharded database. Then the interviewer says: "Design the system that decides which videos to show on the home screen." You draw the same boxes, and twenty minutes later you have a perfectly scalable service that returns the wrong videos. The round was never about the boxes.
An ML system design round asks a different question from a classic system design round. Classic system design asks will it stay up and stay fast? ML system design asks will it make the right decision, can you prove it, and will it keep making it next month? This page explains what the round looks like, what is scored, and how to spend your 45 minutes.
How it works
What you are asked
The prompt is almost always a product feature that needs a decision made millions of times a day, where no hand-written rule is good enough:
- Ranking and recommendation: which videos, posts, products, friends or notifications to show, and in what order.
- Classification: is this message spam, this post harmful, this payment fraud, this account fake.
- Prediction of a number: how long will this trip take, what price will clear the market.
- Search: which results answer this query, which completions to suggest.
The prompt is vague on purpose. "Design a video recommendation system" does not say what "good" means. Turning it into something measurable is the first thing being tested.
What is different from system design

You still draw a serving path, and you still talk about latency and scale. But most of the points come from four things a classic round barely touches:
- Framing: turning a business goal ("people enjoy the app") into a prediction target a model can learn ("probability the user watches more than half of this video").
- Data and labels: where training examples come from, how labels are defined, how long you wait for them, and what biases they carry.
- Evaluation: an offline metric you can compute before launch, an online metric you watch in an A/B test, and a reason the two should agree.
- The loop: a deployed model changes what users see, which changes the data the next model trains on. Monitoring and retraining are part of the design, not an afterthought.
What is scored
Interviewers usually score along the same axes, whatever the company calls them:
- Problem framing: did you pick a sensible ML objective and say why?
- Data: do your labels match the goal? Did you spot leakage, delay and bias?
- Modelling: did you start simple, then add complexity only where it pays?
- Evaluation: offline and online metrics, and how you would run the experiment.
- Production: serving latency, freshness, retraining, monitoring, failure modes.
- Communication: a clear structure the interviewer can follow and steer.
Notice that "knows the newest architecture" is not on the list. A well-chosen gradient-boosted tree with good features and a correct evaluation beats a fashionable model with leaky labels every time.
Trade-offs and costs
The round has a fixed budget: about 45 minutes, sometimes 60. Everything you add costs time somewhere else.

- Breadth versus depth. Touch every stage briefly, then let the interviewer pick one or two to go deep. Covering only modelling in depth is the most common way to fail.
- Simple versus impressive. A baseline first (a heuristic or logistic regression) shows judgement and gives you something to compare against. Jumping straight to a large neural network looks like you can't tell what matters.
- Generic versus specific. "We'll use features" scores nothing. "Watch-time ratio on this creator's last 20 videos, computed hourly" scores well.
When to reach for it
You will meet this round if you apply for:
- Machine learning engineer and applied scientist roles, where it is usually one or two of the onsite rounds.
- Software engineer roles on ranking, ads, search, feeds, trust and safety, or marketplace teams.
- Senior and staff roles where you will own an ML-backed product area, even if you won't train models yourself.
If the job is building LLM products (chat assistants, retrieval-augmented generation, agents), you are more likely to get an AI system design round; see the AI System Design track. The two overlap in evaluation and monitoring, so the fundamentals here still help.
Example
A short example of the difference in practice.
Prompt: "Design the system that ranks notifications for a social app."
A system-design-only answer draws an event queue, a notification service, a push provider and a database of sent notifications. It is correct, and it scores poorly, because it never says which notifications to send.
An ML system design answer starts with the goal: more people come back to the app, without more of them turning notifications off. It frames the model as predicting, for each candidate notification, the probability the user opens it and the probability they disable notifications. It defines labels from open events within one hour, picks features (who sent it, how many unread notifications the user already has, time since last session), starts with logistic regression, moves to a gradient-boosted model, and decides to send only when the expected value clears a threshold. It evaluates offline with AUC and online with an A/B test on daily active users and opt-out rate, and it monitors open rate by notification type. The queue and the push provider are still there, drawn in thirty seconds.
Common mistakes
- Starting with the model. Naming an architecture in the first two minutes, before you know what is being predicted.
- Labels that don't match the goal. Optimising clicks when the business wants satisfaction gives you clickbait.
- Leakage. Using a feature that is only known after the outcome, such as "was refunded" to predict fraud at checkout.
- One metric. Only AUC, with no online metric and no guardrail (opt-outs, complaints, latency).
- No loop. Forgetting that the model's own decisions shape tomorrow's training data.
- Ignoring serving. A model that needs 300 ms per item cannot score 1,000 candidates in a 200 ms page load.
Say this in the interview
"Before I pick a model, let me pin down what we're optimising and how we'll know it worked. Then I'll go through data and labels, a simple baseline and how I'd improve on it, training and serving, the offline and online metrics, and how we monitor and retrain. Tell me where you'd like me to go deeper."
That one paragraph tells the interviewer you have a plan, and invites them to steer.
Used in
- Next: The ML System Design Answer Framework turns the stages above into a step-by-step script.
- Fundamentals: Framing a Product Problem as an ML Problem, Labels and Training Data, Online Metrics and A/B Testing
- Problems to practise first: Design a Spam Email Classifier, Design a Video Recommendation System