Numbers Every AI Engineer Should Know
On this page
"How much GPU memory does that need?" "How long will the answer take?" "What does a million users cost?" Interviewers ask these to see whether you can reason from a few known numbers instead of guessing. You don't need exact figures. You need the right order of magnitude and the formula behind it, so you can say "about 140 GB, so at least two 80 GB GPUs, realistically four". These are the numbers worth knowing, as of late 2026. Hardware and prices change; the formulas don't.
How to use these in an interview: state the assumption, do the arithmetic out loud, round generously, and say how you'd confirm it ("then I'd load-test"). Numbers here are rules of thumb, not vendor specifications.
Five formulas

Tokens and text
| Number | Value | What it means |
|---|---|---|
| English text per token | about 4 characters, about 0.75 words | 1,000 tokens ≈ 750 words ≈ 1.5 pages |
| Other languages | often more tokens per word than English | the same prompt can cost more in Chinese, Japanese or Hindi; measure with the real tokenizer |
| Typical context window | 128k tokens common; some models offer 1M or more | fitting is not the same as using well: long prompts are slower and costlier |
| Human reading speed | roughly 4–7 tokens per second | streaming at 20+ tokens/s feels instant to a reader |
Try it yourself first
You have the full problem. Sketch your answer, then sign up free to read the complete walkthrough.
- The complete design, flow by flow, with pictures
- Deep dives into the hard parts
- What mid, senior and staff candidates are expected to say
- Full company question lists
Free forever. No card needed.