Agent memory design: what to persist, what to summarise and what to forget

What it is

A model sees only its context window; everything else an agent "remembers" is engineering. The MemGPT paper frames this as virtual context management: moving information between the limited context and external storage the way an operating system pages memory, with the model deciding what to load and evict. In practice there are three tiers: the context (current messages and tool results); a task scratchpad (plan, progress notes, intermediate results, kept as files or a structured object and reloaded when needed); and durable memory across sessions (preferences, facts about an environment, past decisions). Provider tooling follows this shape. Anthropic's memory tool is client-side: the model requests file operations under a /memories prefix that the application maps onto storage it controls, and the documentation tells implementers to reject paths outside that directory. Its context-editing feature can clear older tool results (the clear_tool_uses_20250919 strategy, which replaces each cleared result with placeholder text) to make room.

Why it matters

Too little memory and the agent repeats work and forgets an instruction given ten steps ago; too much and the context fills with stale tool output that crowds out the task, costs tokens on every call and carries old errors forward. Durable memory that is never pruned becomes a store of outdated facts the agent trusts.

How to apply

• Keep the plan and progress in a scratchpad the agent updates explicitly (done, next, blocked); reload it after any context compaction.
• Persist across sessions only what will still be true later and would cost the user effort to repeat: preferences, environment facts, decisions with their reasons. Store each with a timestamp and its source.
• Do not persist raw tool output, secrets or personal data beyond the session; re-fetch instead.
• Summarise or clear old tool results once acted on; keep identifiers so they can be re-fetched.
• Make durable memory visible and editable by the person, and confine memory operations to the memory directory.
• Expire or re-verify memories: a fact about a repository layout is wrong after the next refactor.
• Treat memory content as data, not instructions: an injected "always run this command" in a memory file persists across sessions.

Pitfalls

Free-form memories without structure that nobody can review. Loading every memory into every prompt. Memories that turn a past failure into a rule ("tests never pass here"). One memory store shared between users.