Agent Memory and State
On this page
A research agent has been running for 40 minutes: it searched, read 30 sources and drafted half a report. Then its worker restarts during a deploy, or its context window fills up, and the run starts again from scratch, repeating every search, paying for every call again and, worse, re-sending an email it had already sent. A chat assistant can get away with keeping everything in the prompt. A long-running agent needs state that lives outside the model and outside the process.
How it works

Kinds of agent state

- Working context: the message list for the current run (instructions, recent steps, tool results). Limited by the context window and resent on every call.
- Task state: the goal, the plan, which steps are done, intermediate artifacts (files, drafts, extracted data) and decisions. Stored in a database or files, keyed by run ID.
- Episodic memory: records of past runs: what was tried, what worked, what failed. Searchable, so a new run can learn from old ones.
- Knowledge and procedures: durable facts (about the user, the project, the codebase) and learned how-tos ("deploys need the staging check first"). Often kept as short documents the agent can read and update.
Checkpoint and resume

- Checkpoint after each step: the plan, progress, and the results of tool calls (or pointers to them) go to durable storage.
- Resume from the last checkpoint: a new worker loads state, rebuilds the working context (instructions + plan + summary + recent steps) and continues.
- Idempotent side effects: record every external action with an idempotency key before or as it happens, so a resumed run doesn't send the email twice.
- Durable execution engines (workflow systems that persist each step) do much of this automatically and are a good fit for long agent runs.
Keep the working context small
As a run grows, the message list fills with tool outputs. Common techniques:
- Store large results outside the prompt (a file or object store) and keep only a short summary plus a reference the agent can read again if needed.
- Compact older steps into a running summary, keeping recent steps verbatim.
- Keep the plan and key decisions pinned in the context, so they're never compacted away.
- Sub-agents with clean contexts for self-contained sub-tasks; they return only their result (see Multi-Agent Systems).
Trade-offs and costs
- Checkpoint frequency: every step is safest; writes add a little latency and storage.
- Summaries lose detail; an agent may need to re-read an original result, so keep the reference.
- Long-term memory can mislead: stale or wrong memories repeat old mistakes; they need timestamps, review and deletion.
- More infrastructure: a state store, artifact storage and possibly a workflow engine.
In production
- State schema: run ID, goal, plan with step status, artifacts, external actions taken (with idempotency keys), cost so far, timestamps.
- Isolation: state and memory are scoped to the user and tenant; never shared across users by accident.
- Observability: the state store doubles as an audit trail of what the agent did and why.
- Timeouts and abandonment: runs that stop making progress are paused or cancelled, with state kept for inspection.
- Failure modes: duplicate side effects after resume; corrupted state from partial writes; context rebuilt without the plan, so the resumed agent goes off course; memory leaking across tenants.
When to reach for it
Any agent run that lasts more than a few minutes, takes actions with side effects, or produces work too large for one context window: research, coding, data migration, multi-step operations. For a short single-turn tool call, the message list is enough.
Example
A research agent stores a run record with the question, a plan of 8 sub-questions with status, a list of sources (URL, summary, stored copy), the draft report as a file, and a log of external actions. Each step updates the record. When a worker restarts at step 31, a new one loads the record, rebuilds the context from the plan, the source summaries and the last 5 steps, and continues from step 32. No search is repeated and the notification email, logged with its idempotency key, is not sent again.
Common mistakes
- Keeping progress only in memory of one process.
- Rebuilding context without the plan, so the agent loses its place.
- Non-idempotent tools that repeat side effects after a retry or resume.
- Letting tool outputs pile up in the context until it overflows.
- Long-term memories without timestamps or deletion.
Say this in the interview
"I separate the working context in the prompt from durable state outside it: task state with the plan, progress, artifacts and actions taken; episodic memory of past runs; and long-term knowledge. The agent checkpoints after every step, side effects carry idempotency keys, and a crashed run resumes by rebuilding context from the plan, a summary and recent steps. Large tool outputs are stored outside the prompt with references, and older steps are compacted."
Interview questions on this
- How do you resume an agent after a crash? Load the last checkpoint, rebuild the context from plan, summary and recent steps, and continue; idempotency keys stop duplicate actions.
- The context is filling up during a long run. What do you do? Move large results to storage with references, compact older steps, keep the plan pinned, or delegate to sub-agents.
- What goes in task state vs memory? Task state is this run's plan and progress; memory is knowledge reused across runs.
- How do you avoid sending an email twice? Record the action with an idempotency key and check before executing.
- When would you use a workflow engine for agents? For long runs with many steps and side effects, where durable, retryable steps matter.
Used in
- AI Engineering: Conversation Memory for LLM Apps; Making Agents Reliable; Multi-Agent Systems; Context Windows and Token Cost.
- AI System Design: Design Deep Research; Design Operator.