Four kinds of memory, kept apart#
Lumping everything a character knows into one text blob is the root of most memory bugs. Keep four stores, each with its own update rule:
| Store | Holds | Updated by | Example |
|---|---|---|---|
| Facts | Structured truths about the world and the player | Game code | player.owes_lantern = true |
| Episodic | Things that happened, as short observations | After each scene | "Day 3: player promised the ferry fare by dawn" |
| Relational | Numbers describing a relationship | Validated model proposals | trust(Oda, player) = 0.62 |
| Reflections | Higher-level conclusions drawn from episodes | Periodic reflection pass | "The courier keeps promises late but keeps them" |
Facts and relational values belong to the game: the model can propose changes, code applies them. Episodic memories and reflections are text the model writes and later reads. The Generative Agents paper (Park et al., 2023) introduced this split between raw observations and synthesised reflections, and it remains the most useful reference design.
The retrieval score#
Each turn, you can only afford to put a handful of episodic memories into the prompt. Generative Agents ranks them with a weighted sum of three normalised scores:
where each hat denotes min–max scaling to across the candidate set, and:
- Recency decays exponentially with time since the memory was last accessed: . The paper uses per game hour.
- Importance is assigned once, when the memory is created. The paper asks the model to rate it on a 1–10 scale, from mundane to poignant.
- Relevance is the cosine similarity between the memory's embedding and an embedding of the current situation.
Because recency is refreshed on access, memories that keep being useful stay fresh. That's a pleasing property for a character: the things they talk about often stay vivid.
Choose the decay by half-life, not by feel#
is hard to reason about directly. Convert it to a half-life, the time for recency to fall to 0.5:
The paper's per hour has a half-life of about 138 game hours, roughly 5.8 in-game days. That suits a simulation running continuously, but your game's clock is probably different. Pick the half-life from the fiction, then derive :
| Game | Clock unit | Desired half-life | γ per unit |
|---|---|---|---|
| Day-cycle life sim | in-game hour | 3 in-game days (72 h) | 0.9904 |
| Mystery, one night per case | real minute | 20 minutes | 0.9659 |
| Strategy, turn-based | turn | 8 turns | 0.9170 |
| Companion, real-time | real day | 2 weeks | 0.9517 |
Then tune the weights . A useful starting point is equal weights, then raise for characters who should stay on topic (an interrogator) and for characters who should bring up the past unprompted (a jilted rival).
Where retrieved memories go in the prompt#
Retrieval is wasted if the model doesn't use what you retrieved. Liu et al.'s "Lost in the Middle" (2023) found that models use information best when it's at the beginning or end of a long input, and that performance drops significantly for information in the middle, even for models built for long contexts. For NPC prompts that suggests a layout:
- Top: the stable block: world rules, tone bible, character sheet. Stable content first also makes the prefix cacheable (see prompt caching for dialogue).
- Middle: the conversation summary and older raw exchanges, the material that matters least per token.
- Bottom: retrieved memories and current game facts, then the player's latest line.
This ordering serves both goals at once: cache-friendly at the top, attention-friendly at the bottom.
Summaries and reflections#
Summaries compress the current conversation; reflections compress a character's history. Both lose detail every time they're rewritten, so:
- Summarise into structure. "promised: ferry fare, due: dawn day 3, status: late" survives five rounds of re-summarisation. A paragraph doesn't.
- Never summarise facts the game owns. If code knows the player owes a lantern, don't make the model remember it too. Two sources of truth disagree eventually.
- Reflect on a schedule, not every turn. Generative Agents triggers reflection when accumulated importance crosses a threshold. In games, scene boundaries and sleep cycles are natural triggers.
How often to summarise is a cost question with a closed-form answer. See the optimal summary interval.
Testing memory#
Memory bugs are slow: they show up dozens of hours into a save. Build tests that compress time:
- Planted-fact tests. Seed a memory, simulate N hours of unrelated play, then ask about it. Measure recall at increasing N and check it matches your intended half-life.
- Contradiction tests. Plant two conflicting memories and check the character resolves them the way your fiction says (most recent wins, or most important).
- Leak tests. Make sure characters don't "remember" things they never witnessed, a common bug when memory stores are shared.
Frequently asked questions#
How do AI NPCs remember things?
Typically by storing observations outside the prompt and retrieving the most relevant ones each turn, ranked by a combination of recency, importance and relevance, plus summaries of the current conversation.
What decay rate should NPC memory use?
Pick a half-life that fits your game's fiction and convert it: γ = 2^(−1/half-life). The Generative Agents paper's 0.995 per game hour is a half-life of about 138 hours.
Where should memories go in an LLM prompt?
Near the end, close to the player's latest input. Research on long contexts shows models use information at the start or end of the input best. Keep stable content at the top for caching.
Sources#
- Park, O'Brien, Cai, Morris, Liang, Bernstein, "Generative Agents: Interactive Simulacra of Human Behavior", arXiv:2304.03442 (2023). arxiv.org/abs/2304.03442
- Liu, Lin, Hewitt, Paranjape, Bevilacqua, Petroni, Liang, "Lost in the Middle: How Language Models Use Long Contexts", TACL (2023), arXiv:2307.03172. arxiv.org/abs/2307.03172