What AI Engineering Interviews Test
On this page
Two candidates prepare for "AI Engineer" interviews. One spends a month on transformer maths and gets asked how to cut a RAG feature's cost in half. The other practises prompt tricks and gets asked to size GPU memory for serving a 70B model. Both prepared hard, for the wrong job. "AI engineering" covers several different roles, and the interview loop follows the role. This page maps the roles, the usual rounds, and what interviewers listen for, so you can study the right parts of this track.
How it works
Three roles behind one title

- AI Application Engineer: builds product features on top of LLMs: prompts, structured output, RAG, evals, guardrails, and the latency and cost budget of each feature. Often a backend or full-stack engineer who has specialised.
- Agent Engineer: builds systems where the model takes multi-step actions with tools: agent loops, tool design, memory and state, reliability, safety and agent evaluation.
- AI Infra Engineer: builds and runs the platform: model serving (batching, KV cache, quantization), GPU clusters, training and fine-tuning pipelines, cost per token and capacity planning.
Read the job description: words like RAG, evals and product point to the first; tools, workflows and autonomy to the second; serving, GPUs, CUDA, distributed training and throughput to the third. Many roles mix two of them.
The usual rounds

- Screen with a recruiter or hiring manager: your experience, what you built with LLMs, why this role.
- Coding: a standard coding problem, or ML coding such as implementing attention, top-k/top-p sampling, a BPE tokenizer, or a KV cache in Python or NumPy. Infra roles may add performance or systems questions.
- AI system design: design something end to end, for example a document Q&A assistant, a support agent, or an inference service. See the AI System Design track for full walkthroughs.
- Technical deep dive on a past project: what you built, what you measured, what failed and what you'd change.
- Behavioural: ownership, handling ambiguity, collaboration.
Practical questions often appear inside these rounds: "How would you reduce hallucinations?", "Your RAG answers are wrong; how do you debug?", "How many GPUs to serve this model?". The Practice section has worked answers.
What strong answers have in common

- Numbers: token counts, latency budgets (time to first token, tokens per second), cost per request, GPU memory. Rough is fine; absent is not.
- Evals: how you'd know it works and how you'd catch regressions.
- Failure modes: hallucination, prompt injection, agent loops, overload, stale data.
- Trade-offs: why this choice, what you give up, and when you'd switch.
How to use this track
- Everyone: Start Here, then LLM Foundations (tokens, transformer basics, sampling, context and cost, embeddings, hallucination).
- AI Application Engineer path: AI Applications (prompting, structured output, RAG, evals, guardrails, caching, rate limits), then the application-focused Practice questions.
- Agent Engineer path: Foundations, the core of AI Applications, then all of Agents and the agent practice questions.
- AI Infra Engineer path: Foundations, then AI Infra (inference, training, GPUs, data, LLMOps), Fine-tuning, and the ML coding problems.
- Before Interview: the cheat sheet and the numbers page the night before.
Each article has an "Interview questions on this" section with short answers you can practise out loud.
Trade-offs in preparing
- Depth vs breadth: go deep on your role's area and keep the others at "can explain in two minutes".
- Theory vs practice: interviewers value having built and measured something; a small RAG or agent project with an eval set is worth more than memorised papers.
- Coding: don't skip standard coding practice; many loops still include it.
Common mistakes
- Preparing for the wrong role.
- Answering design questions without numbers or evals.
- Describing past projects without measurements ("it worked well").
- Ignoring failure modes and safety.
- Treating the interview as a quiz instead of a conversation about trade-offs.
Say this in the interview (about your own work)
"I built X to solve Y. The constraint was Z (latency, cost, data privacy). I tried A and measured it on an eval set of N real cases; it failed on B, so I changed to C, which improved the metric from P to Q at a cost of R. Next I'd do S."
Interview questions on this
- "Tell me about an LLM feature you shipped." Problem, constraints, design, how you measured it, what broke, what you'd change.
- "How do you evaluate an LLM system?" Test set from real data, code checks plus calibrated judges, offline before release, online after.
- "What's the hardest part of running LLMs in production?" Pick one (cost, latency, reliability, evaluation) and explain with a concrete example.
- "How do you keep up with the field?" Name a habit (reading release notes and papers, building small experiments) and a recent change you applied.
- "Why this role?" Connect your experience to the role type: application, agent, or infrastructure.
Used in
- AI Engineering: Welcome to AI Engineering; AI Engineering Cheat Sheet; Numbers Every AI Engineer Should Know; all Practice articles.
- AI System Design: How AI system design differs; the full problem list.
Sources consulted for topics (not copied): public AI engineer interview guides from Exponent, MentorCruise, Let's Data Science, Careery and techinterview.org, read in November 2026.