Making Agents Reliable: Retries, Timeouts, Checkpoints
On this page
An agent works perfectly in the demo: five steps, every one succeeds. In production, tasks take 20 steps, tools time out, the provider returns 429s at peak hours, and the model occasionally passes a bad argument. Each problem is rare, yet a third of runs fail. Reliability for agents is mostly about arithmetic and engineering, not about a smarter model.
How it works
Errors compound

A run succeeds only if every step does, so per-step success multiplies: 0.95²⁰ ≈ 0.36. Two levers follow: make each step more reliable and make the run recover from failed steps instead of failing as a whole. Shorter runs (fewer steps) help too.
Guard every step

- Timeouts on every model and tool call, so one hung call doesn't hang the run.
- Retries with exponential backoff and jitter for transient errors only (timeouts, 429, 503), with a cap.
- Validation of model output (schema, allowed values, business rules) and of tool results before using them.
- Fallbacks: a second model or provider when the primary is down or rate-limited.
- Budgets: maximum steps, wall-clock time, tokens and cost per run.
import random, time
class TransientError(Exception):
"""Timeouts, 429s, 503s: worth retrying."""
def call_with_retry(fn, max_attempts=4, base=0.5, cap=8.0, sleep=time.sleep):
for attempt in range(1, max_attempts + 1):
try:
return fn()
except TransientError:
if attempt == max_attempts:
raise
delay = min(cap, base * 2 ** (attempt - 1))
sleep(random.uniform(0, delay)) # "full jitter" spreads retries out
Classify failures

- Transient (timeouts, rate limits): retry with backoff.
- Permanent (not found, permission denied): don't retry; return a clear error to the model so it can change approach or ask the user.
- Model mistakes (invalid arguments, wrong tool, repeating itself): validate, send specific feedback, cap the number of corrections, detect loops (same call with the same arguments several times).
- Stuck or risky (no progress, or about to do something high-impact): checkpoint and escalate to a person.
Recover instead of restarting
- Checkpoint after each step so a crash resumes where it stopped (see Agent Memory and State).
- Idempotency keys on side-effecting tools, so retries and resumes never double-charge or double-send.
- Verification steps at milestones ("did the file actually change? do the tests pass?") catch silent failures early.
- Graceful degradation: return partial results with a clear note rather than failing silently.
Trade-offs and costs
- Retries add latency and cost; aggressive retries amplify outages (retry storms). Cap them and respect
Retry-After. - Validation and verification steps cost extra calls but are far cheaper than a failed run.
- Human escalation improves outcomes but needs a UI, a queue and people.
- Determinism: lower temperature for tool-calling steps reduces random mistakes, at little cost.
In production
- Measure per-step success rates by tool and by model, not just run success, to find the weakest link.
- Alert on rising retry rates, loop detections, budget hits and escalation rates.
- Chaos testing: inject tool timeouts and errors in evals to see whether the agent recovers.
- Failure modes: retry storms during a provider outage; non-idempotent tools repeated after resume; infinite correction loops; agents reporting success after a failed tool call.
When to reach for it
As soon as an agent runs more than a few steps, touches external systems, or runs unattended. The longer and more autonomous the run, the more these guards matter.
Example
A data-migration agent had 64% run success on 50 eval tasks averaging 18 steps. Traces showed most failures were transient API errors and two kinds of bad tool arguments. Fixes: retries with jitter on transient errors, schema validation with a one-shot correction message, a loop detector, checkpoints after each table, and a verification query after each migration step. Run success rose to 94%, with average cost up only 8% (example numbers).
Common mistakes
- Retrying everything, including permanent errors.
- Retries without jitter or caps, causing retry storms.
- No step or cost budget.
- Trusting the model's claim of success instead of verifying state.
- Non-idempotent side effects in a system that retries.
Say this in the interview
"Per-step errors compound, so a 95%-reliable step gives about 36% success over 20 steps. I'd raise per-step reliability with timeouts, retries with backoff and jitter for transient errors, output validation and fallback models; and make runs recoverable with checkpoints, idempotent tools, verification at milestones and escalation when stuck. Failures are classified: retry transient ones, return permanent ones to the model, correct model mistakes a bounded number of times."
Interview questions on this
- Why do long agent runs fail more? Success probability multiplies across steps.
- Which errors should you retry? Transient ones (timeouts, 429, 503), with backoff, jitter and a cap.
- How do you stop an agent from repeating a side effect after a retry? Idempotency keys checked by the tool.
- How do you detect a stuck agent? Repeated identical calls, no state change across steps, or budget nearly used without progress.
- How do you test reliability? Inject tool failures and latency in evals and measure recovery rate.
Used in
- AI Engineering: Tool Calling and the Agent Loop; Agent Memory and State; Evaluating Agents; Your Agent Is Stuck in a Loop (practical question); Model Routing and Fallbacks.
- AI System Design: Design Operator; Design Deep Research.