A bad AI answer can ruin one session.
A bad AI memory can quietly contaminate the next hundred.
That’s becoming a more important distinction as persistent memory moves into mainstream agent platforms. Google Cloud’s Agent Platform Memory Bank, for example, can extract information from conversations and reuse it across future sessions.
Useful? Absolutely.
But I’d ask a different product question before celebrating that your agent finally remembers things:
What has earned the right to be remembered?
Storage is cheap. Persistent bad assumptions aren’t.
Not every fact deserves to become memory
Imagine a customer-support agent encounters this:
Don’t ship replacement units to our Dallas office anymore. We moved receiving to Austin.
The model may have no trouble extracting:
Shipping address = Austin
That doesn’t mean the product should store it.
Who said it? Were they authorized to change shipping instructions? Is the CRM actually the system of record? Was this a temporary change?
The model solved the extraction problem. The product still has an authority problem.
I’d treat memory writes much like AI agent permissions.
A preference like “give me concise answers” might be safe to remember automatically.
A shipping address, account entitlement, approval rule, or production configuration probably deserves confirmation or should remain owned by another system entirely.
The useful question isn’t simply can the agent remember this?
It’s who gave it permission to make this durable?
Remember where the memory came from
Persistent memory also needs provenance.
Suppose an agent tells you:
The customer prefers weekly reports.
Great. Says who?
Maybe the customer said it yesterday. Maybe the agent inferred it from three meetings. Maybe another agent wrote it. Maybe it came from a six-month-old support ticket that stopped being true four months ago.
Once information survives the session, retrieval alone can make it look authoritative.
For consequential memories, I’d want enough metadata to reconstruct something like:
source → author → time → status → scope → expiration
That doesn’t need to clutter the user experience. It does need to exist when someone asks, “Why does the agent think this?”
This also matters for AI evaluations based on production evidence.
If an agent acts on a bad memory, “memory accuracy dropped” isn’t much of a diagnosis.
Did retrieval select the wrong fact?
Was the stored fact wrong?
Did a once-correct fact become stale?
Should the agent never have stored it?
Those are four different defects with four different fixes.
Facts have half-lives
A user may prefer concise answers for years.
An incident workaround might survive until tomorrow morning.
A temporary shipping restriction may expire Friday.
A project assumption can die during one executive meeting.
Giving all of those the same retention policy is convenient for the database and not particularly useful for the product.
I’d define a few memory classes and give each one an expiration or review rule.
The important part isn’t the taxonomy. It’s the lifecycle.
And correction has to actually mean correction.
If a customer changes a location from Dallas to Austin, keeping both memories around and hoping the model notices the timestamp is not a correction system. It’s a trivia contest.
The product should know which fact is active.
Test forgetting, not just remembering
Most memory demos ask:
Did the agent remember?
I’d add the opposite acceptance test:
Did the agent forget when it was supposed to?
For example:
Session 1: An authorized user changes preference A to B.
Session 2: The agent retrieves B and can identify its source.
Session 3: The user deletes B.
Session 4: The agent no longer uses B or reconstructs it from some forgotten summary.
Then make it nastier.
What happens when two memories conflict?
What happens when the source document gets revoked?
Can an external webpage trick the agent into storing a new “company policy”?
Can one user’s memory leak into another user’s session?
Security teams are already testing these failure modes. Palo Alto Networks has added memory-poisoning scenarios to Prisma AIRS red-team testing, and Tencent’s Zhuque Lab has demonstrated attacks targeting stored agent information.
Those are security problems, but they’re also product requirements.
A PM defining persistent memory should be specifying write authority, provenance, scope, expiration, correction, deletion, and tests for each.
Otherwise “add memory” isn’t much of a requirement.
It’s just a request to make today’s context somebody else’s future problem.
The best AI agent memory isn’t the one that remembers everything. It’s the one that knows what deserves to survive the session.
Comments
Comments are powered by GitHub Discussions through Giscus.