Agent Memory and State

Updated Nov 16, 2026 · 4 min read
On this page

    A research agent has been running for 40 minutes: it searched, read 30 sources and drafted half a report. Then its worker restarts during a deploy, or its context window fills up, and the run starts again from scratch, repeating every search, paying for every call again and, worse, re-sending an email it had already sent. A chat assistant can get away with keeping everything in the prompt. A long-running agent needs state that lives outside the model and outside the process.

    How it works

    If progress only exists in the message list held by one process, any crash or overflow loses it.
    If progress only exists in the message list held by one process, any crash or overflow loses it.

    Kinds of agent state

    Only the first kind lives in the prompt. The others live in storage and are loaded when needed.
    Only the first kind lives in the prompt. The others live in storage and are loaded when needed.
    1. Working context: the message list for the current run (instructions, recent steps, tool results). Limited by the context window and resent on every call.
    2. Task state: the goal, the plan, which steps are done, intermediate artifacts (files, drafts, extracted data) and decisions. Stored in a database or files, keyed by run ID.
    3. Episodic memory: records of past runs: what was tried, what worked, what failed. Searchable, so a new run can learn from old ones.
    4. Knowledge and procedures: durable facts (about the user, the project, the codebase) and learned how-tos ("deploys need the staging check first"). Often kept as short documents the agent can read and update.

    Checkpoint and resume

    Write state after every step, so a crash costs one step, not the whole run. Compact old context into summaries with links.
    Write state after every step, so a crash costs one step, not the whole run. Compact old context into summaries with links.
    • Checkpoint after each step: the plan, progress, and the results of tool calls (or pointers to them) go to durable storage.
    • Resume from the last checkpoint: a new worker loads state, rebuilds the working context (instructions + plan + summary + recent steps) and continues.
    • Idempotent side effects: record every external action with an idempotency key before or as it happens, so a resumed run doesn't send the email twice.
    • Durable execution engines (workflow systems that persist each step) do much of this automatically and are a good fit for long agent runs.

    Keep the working context small

    As a run grows, the message list fills with tool outputs. Common techniques:

    • Store large results outside the prompt (a file or object store) and keep only a short summary plus a reference the agent can read again if needed.
    • Compact older steps into a running summary, keeping recent steps verbatim.
    • Keep the plan and key decisions pinned in the context, so they're never compacted away.
    • Sub-agents with clean contexts for self-contained sub-tasks; they return only their result (see Multi-Agent Systems).

    Trade-offs and costs

    • Checkpoint frequency: every step is safest; writes add a little latency and storage.
    • Summaries lose detail; an agent may need to re-read an original result, so keep the reference.
    • Long-term memory can mislead: stale or wrong memories repeat old mistakes; they need timestamps, review and deletion.
    • More infrastructure: a state store, artifact storage and possibly a workflow engine.

    In production

    • State schema: run ID, goal, plan with step status, artifacts, external actions taken (with idempotency keys), cost so far, timestamps.
    • Isolation: state and memory are scoped to the user and tenant; never shared across users by accident.
    • Observability: the state store doubles as an audit trail of what the agent did and why.
    • Timeouts and abandonment: runs that stop making progress are paused or cancelled, with state kept for inspection.
    • Failure modes: duplicate side effects after resume; corrupted state from partial writes; context rebuilt without the plan, so the resumed agent goes off course; memory leaking across tenants.

    When to reach for it

    Any agent run that lasts more than a few minutes, takes actions with side effects, or produces work too large for one context window: research, coding, data migration, multi-step operations. For a short single-turn tool call, the message list is enough.

    Example

    A research agent stores a run record with the question, a plan of 8 sub-questions with status, a list of sources (URL, summary, stored copy), the draft report as a file, and a log of external actions. Each step updates the record. When a worker restarts at step 31, a new one loads the record, rebuilds the context from the plan, the source summaries and the last 5 steps, and continues from step 32. No search is repeated and the notification email, logged with its idempotency key, is not sent again.

    Common mistakes

    • Keeping progress only in memory of one process.
    • Rebuilding context without the plan, so the agent loses its place.
    • Non-idempotent tools that repeat side effects after a retry or resume.
    • Letting tool outputs pile up in the context until it overflows.
    • Long-term memories without timestamps or deletion.

    Say this in the interview

    "I separate the working context in the prompt from durable state outside it: task state with the plan, progress, artifacts and actions taken; episodic memory of past runs; and long-term knowledge. The agent checkpoints after every step, side effects carry idempotency keys, and a crashed run resumes by rebuilding context from the plan, a summary and recent steps. Large tool outputs are stored outside the prompt with references, and older steps are compacted."

    Interview questions on this

    1. How do you resume an agent after a crash? Load the last checkpoint, rebuild the context from plan, summary and recent steps, and continue; idempotency keys stop duplicate actions.
    2. The context is filling up during a long run. What do you do? Move large results to storage with references, compact older steps, keep the plan pinned, or delegate to sub-agents.
    3. What goes in task state vs memory? Task state is this run's plan and progress; memory is knowledge reused across runs.
    4. How do you avoid sending an email twice? Record the action with an idempotency key and check before executing.
    5. When would you use a workflow engine for agents? For long runs with many steps and side effects, where durable, retryable steps matter.

    Used in

    • AI Engineering: Conversation Memory for LLM Apps; Making Agents Reliable; Multi-Agent Systems; Context Windows and Token Cost.
    • AI System Design: Design Deep Research; Design Operator.