Agent Systems
Agent Memory Is a System Architecture
A practical model for separating working context, durable knowledge, task state, and execution history.
Calling a vector database “memory” collapses several different responsibilities into one component. An agent needs immediate conversational context, persistent knowledge, task progress, and an accountable record of what it did. These have different lifetimes, consistency needs, and failure modes.
Four memory planes
Working context
Working context is the small, active set of information needed for the next decision. It should be curated for relevance and bounded for cost. Summarization can help, but summaries must retain commitments, constraints, and unresolved questions.
Durable knowledge
Durable knowledge contains facts and artifacts intended to survive across tasks. Retrieval quality depends on provenance, update policy, and access control as much as embeddings. Every retrieved item should retain its source and freshness.
Task state
Task state records goals, plans, dependencies, outputs, and completion criteria. It is structured application state, not prose. Treating it explicitly makes retries, handoffs, and interruption recovery much more reliable.
Execution history
Execution history is the append-only trace of observations, tool calls, decisions, and results. It supports debugging and evaluation. The trace should be queryable without being injected wholesale into the next prompt.
Memory needs policy
Storage alone does not decide:
- what should be remembered,
- when it can be recalled,
- which source wins when facts conflict,
- when information expires,
- who may read or change it.
These are policy decisions. A useful architecture puts policy gates between memory planes and the model rather than letting every component read and write everything.
Evaluate behavior, not retrieval alone
Memory benchmarks should test long-horizon behavior: maintaining a constraint after many steps, recovering a paused task, correcting outdated facts, and avoiding the reuse of private information. Retrieval precision is necessary, but the agent's final action is the real outcome.
The central design rule is separation. When working context, knowledge, state, and history have clear contracts, an agent becomes easier to reason about and safer to operate.