A new arXiv paper introduces MemTrace, a benchmark for evaluating long-term memory in LLM agents. Instead of scoring isolated questions, it measures memory at the level of a typed user fact, or knowledge point.
That distinction matters because aggregate accuracy can hide whether an agent consistently tracks the same fact as it ages, changes, or appears under different evidence conditions. Real assistants need stable memory behavior across sessions.
MemTrace targets a core problem for personalized agents: knowing whether memory systems preserve the right facts, forget outdated ones, and handle change over time.