ML System Design Cheat Sheet
On this page
The night before an ML system design interview, you don't need another long walkthrough. You need the shape of a good answer, the phrases that show judgement, and the traps to avoid. This page is that, on one screen.
The plan

Say this in the first minute:
"I'll frame the problem and the label, then data and features, a baseline and how I'd improve on it, training, serving, offline and online evaluation, and monitoring. I'll go through all of them quickly, then we can go deep wherever you like."
1. Frame (5 min)
- Goal + guardrail: "More satisfying watch time, without more reports."
- ML objective: "Predict P(watch ≥ 50% | user, video, context)" — the event, the unit and the window.
- Decision: how the probability becomes an action (rank, threshold, expected-value rule).
- Say what you're not optimising and why (clicks reward clickbait).
2. Data and features (8 min)
- One row = ? One impression, one transaction, one user–item pair.
- Labels: source, delay, bias (position, selection, noise). Name a human-reviewed sample for truth.
- Features in four groups: user, item, context, user × item cross. Give two concrete ones per group.
- Leakage check: "Would I know this at prediction time?" Log features at serving.
3. Model ladder (10 min)
- Weak: heuristic or logistic regression; a baseline and a fallback.
- Good: gradient-boosted trees on hand-made features.
- Strong: neural model with embeddings, multi-task heads, or a two-tower retrieval model, when there's enough data to justify it.
- Say why each step is worth its cost.
4. Training (5 min)
- Split by time. Never random for future prediction.
- Imbalance: downsample negatives, then correct probabilities and calibrate.
- Cadence: online, daily or weekly, chosen by how fast the data changes.
- Gates: data validation, beat production on the same test set per segment, shadow, canary, rollback.
5. Serving (5 min)
- Batch or online, and which parts of each.
- Funnel sizes: millions → ~1,000 (retrieval) → ~100 (ranking) → page (re-rank).
- Latency budget per stage, and a fallback for each failure.
6. Evaluation (7 min)

- Offline metric that matches the label (table above).
- A/B test: randomisation unit, one primary metric, guardrails, duration (at least a week or two), sample size.
- Interference: social and marketplace products need cluster or switchback designs.
7. Monitoring (5 min)
- Four layers: system, inputs, predictions, outcomes, by segment.
- Drift: data drift (inputs move) vs concept drift (meaning moves).
- Feedback loops: position bias, exploration slot, random holdout, long-term holdout.
Numbers worth remembering
These are rough, interview-scale numbers to sanity-check your design (always state them as assumptions):
| Quantity | Rough value |
|---|---|
| Requests a second from daily requests | per day ÷ 86,400 (≈ ÷ 100,000), peak ≈ 2–3× average |
| 64-float embedding | 256 bytes; 100 M items ≈ 26 GB |
| Product-quantised vector | ~32–64 bytes |
| Tree model, CPU | ~10,000–50,000 scorings a second per core (example) |
| Retrieval stage | tens of ms for ~1,000 candidates |
| Users per arm to detect a 1% relative lift on a 5% rate | ~3 million |
| Downsampling correction | p = p′ / (p′ + (1 − p′) / w) |
Try it yourself first
You have the full problem. Sketch your answer, then sign up free to read the complete walkthrough.
- The complete design, flow by flow, with pictures
- Deep dives into the hard parts
- What mid, senior and staff candidates are expected to say
- Full company question lists
Free forever. No card needed.