first proposal for a memory module in mesa

[colinfrisch]

Add agent memory module for storing and retrieving information

**PR is closed - New architecture for memory in #2744 !

Summary

This PR introduces a new Memory module that enables agents to store, recall, and share information during simulations. The mesa user may need memory for a wide range of different usages. Taking this in consideration, I made the memory very flexible and able to store different types of information (cf. memory structure).

Overall, this memory system provides a simple but useful way to add memory capabilities to any Mesa agent with minimal implementation overhead.

Key features

Basic features

• Capacity-limited FIFO memory storage with automatic management
• Flexible entry types for any kind of information
• Timestamped entries tied to simulation steps
• Selective memory removal
• Entry selecting by type

Extra features - the value of this memory module is here

• Inter-agent memory sharing capabilities
• Efficient retrieval by entry type

Memory structure

After studying different data structures (plain dict, building my own structure, etc.) I chose to use an OrderedDict that have roughly the same complexity as a plain dict, but with features relative to its ordering (pop and related).

Example of a structure for my_agent.memory.memory_structure :

{
    "entry_id1" : {
        "entry_step" : 7,
        "entry_type" : "position",
        "entry_content" : [1,2]
    },
    "entry_id2" : {
        "entry_step" : 12,
        "entry_type" : "infection",
        "entry_content" : true
    },
    "entry_id3" : {
        "entry_step" : 32,
        "entry_type" : "health",
        "entry_content" : 12
    },
    "entry_id4" : {
        "external_agent_id" : "external_agent_id",
        "entry_step" : 34,
        "entry_type" : "dialog",
        "entry_content" : "Hello, I feel sick like a sheep"
    }
}

Note : the external_agent_id component in the last entry indicates that this entry was transmitted to my_agent by another agent.

Example usage

I created a whole working model (the foraging ants model) using this memory feature, feel free to explore it (and don't hesitate to give me feedbacks)

# Creating a memory for an agent
agent.memory = Memory(model=self.model, agent_id=self.unique_id, capacity=10)

# Storing information
position_entry_id = agent.memory.remember(
    entry_content=[10, 20],
    entry_type="position"
)

# Recalling information
position_data = agent.memory.recall(position_entry_id)

# Finding entries by type
all_position_entries = agent.memory.get_by_type("position")

# Sharing memory with another agent
agent.memory.tell_to(position_entry_id, other_agent)

# Removing an entry
agent.memory.forget(position_entry_id)

I would love to have your feedbacks on this proposal !

Update : Next steps

I came across a paper on LLM based agents detailing the possible configuration for memory structures in this case. I am going to experiment around this to see if we can make a general memory structure (not only around LLMs) with these elements.

[github-actions]

Performance benchmarks:

Model	Size	Init time [95% CI]	Run time [95% CI]
BoltzmannWealth	small	🔵 -0.8% [-1.3%, -0.3%]	🔵 -1.7% [-1.8%, -1.5%]
BoltzmannWealth	large	🔵 -0.2% [-0.7%, +0.4%]	🔵 -2.6% [-4.9%, -0.7%]
Schelling	small	🔵 -0.9% [-1.1%, -0.6%]	🔵 -1.5% [-1.7%, -1.3%]
Schelling	large	🔵 -0.7% [-1.5%, -0.1%]	🔵 -2.4% [-3.6%, -1.3%]
WolfSheep	small	🔵 -0.6% [-0.8%, -0.3%]	🔵 -0.5% [-0.7%, -0.2%]
WolfSheep	large	🔵 -1.3% [-2.4%, -0.0%]	🔵 -0.4% [-1.2%, +0.4%]
BoidFlockers	small	🔵 +1.4% [+0.6%, +2.0%]	🔵 -0.1% [-0.3%, +0.2%]
BoidFlockers	large	🔵 +1.3% [+0.8%, +1.7%]	🔵 +0.0% [-0.2%, +0.3%]

[EwoutH]

Thanks for your PR!

• @quaquel you’re either going to like this or have an opinion on it ;)
• @wang-boyu might be a nice (fundamental) building block for LLMs
• @Corvince this seems like the stuff you could be interested in

[quaquel]

I need some time to make up my mind about this. I do appreciate the well-thought-out and clear API.

However, I also have some questions

1. What is the overhead?
2. Why ordered dict? dicts are allways ordered these days in python. Are there specific functions you need from ordered dict? And again, what is the resulting overhead of this? If I recall correctly OrderedDict is done in python not C.
3. You placed the new module in experiments/devs, but devs is for discrete event simulation so probably it should be moved to just experimental.
4. What about finite-length memory? I often use a deque for this in my own models.

[colinfrisch]

@quaquel, thank you for the quick answer

1. What is the overhead?

I don't really get your question, are you referring to code complexity or performance overhead ? If you meant in terms of performance, can you tell my how I can verify this ?

2. Why ordered dict? dicts are allways ordered these days in python. Are there specific functions you need from ordered dict? And again, what is the resulting overhead of this? If I recall correctly OrderedDict is done in python not C.

I originally chose ordered dict over the plain dict because there were pre-implemented functionalities such as move_to_end(). But in the end, I think that I might not need them immediately

3. You placed the new module in experiments/devs, but devs is for discrete event simulation so probably it should be moved to just experimental.

Alright, I didn't really know where to put it at first, I'll move it at my next commit.

4. What about finite-length memory? I often use a deque for this in my own models.

It would indeed be quite handy to have a deque for this but I'm not sure if its compatible with the memory_storage structure that I've implemented... Do you have improvement ideas for it ?

If you have any other suggestions regarding the global idea or a specific aspect of the feature, I'd love to discuss it !

[quaquel]

1. Yes I mean performance overhead. The way to go probably is to take a few examples where one isolates the memory part and compares the runtime using timeit of a custom implementation and via the memory module. You could even do that with the ant model you created as a start.
2. What do you see as critical components for the memory module?
3. ok
4. Can you sketch the memory structure you currently have and what motivated this design? This also interacts with point 2.

[EwoutH]

Thanks again for working on this! I have two comments, one on conceptual level, and one on an implementation level.

1. Use case / value

First of all, I love the extensive example models. However, I don't immediately see how much code would be saved here, compared to implementing something like this yourself in a model. I feel the main advantage is the automatic capacity handling, but maybe I'm missing something.

Could you give a few examples of use cases in which this approach saves a significant amount of code or enables new behavior compared to just storing things agents need to remember in their attributes directly? Like Using Lists as Stacks?

I just want to prevent building a complete module if that functionality is already there in plain Python.

2. Memory efficiency

I already mentioned it in my video call, but I think the current storage way is very inefficient. I'm not an expert on this, but I did a bit of LLM brainstorming, which led to options I think are viable:

Regarding the memory_storage structure, using a nested dictionary for each entry ({"entry_step": ..., "entry_type": ...}) might not be the most memory-efficient approach. This repeats the string keys for every single entry, potentially leading to significant overhead, especially with large memory capacities or many agents.

To improve memory efficiency while maintaining clarity, could we consider using structures that store the data as attributes instead? Two good options:

1. dataclass (with slots=True): Define a MemoryEntry dataclass. Using slots=True further minimizes memory by preventing the creation of __dict__ for each instance.

   from dataclasses import dataclass, field
   
   @dataclass(slots=True)
   class MemoryEntry:
       entry_content: Any
       entry_type: Any
       entry_step: int
       external_agent_id: int | None = None
       # Add internal_id if needed, or derive unique key differently

2. collections.namedtuple: A lightweight alternative, essentially a tuple with named fields. Very memory efficient and provides readable attribute access.

   from collections import namedtuple
   MemoryEntry = namedtuple("MemoryEntry", ["entry_content", "entry_type", "entry_step", "external_agent_id"])

In both cases, the main memory_storage OrderedDict would map an ID (perhaps just the counter value) to instances of MemoryEntry. This avoids the key duplication issue and should significantly reduce the memory footprint per entry.

Slightly related, we could also consider using just the internal counter (next(self._ids)) as the key for the main OrderedDict itself, rather than the (agent_id, counter) tuple. The agent_id is already available via self.agent_id, and this might slightly reduce memory/complexity for the keys. A globally unique ID (self.agent_id, internal_id) could still be constructed when needed (e.g., in tell_to or for external logging).

Ideally you would do some benchmarks comparing the memory size and write and read performance of different memory structures. Of course that has to be weighted against code complexity and the user API.

---

Totally out of scope, but it might also be good to keep in mind what we want to do with that memory once we have it. We might consider extending the memory module's capabilities beyond simple storage and retrieval to include built-in analytical functions. This could involve features like calculating recency-weighted values for specific entry types (e.g., using exponential decay based on entry timestamps) to give more importance to recent information, or performing basic trend estimation (e.g., using linear regression over a recent window of numerical entries) to determine if values are increasing or decreasing. To allow such features, we might want to (optionally) support recording timestamps when a memory value was added.

[colinfrisch]

Thanks a lot for this detailed answer @EwoutH ! I've actually worked quite a bit on a new prototype of a memory module based on :

• the feedbacks that I've had here from you and @quaquel (usefulness and memory efficiency, mostly)
• The intent of being able to use it as a base for the LLM project (as well as a usage for all agents --> make it modulable)

After quite a lot of prototyping, I've started the code for an upgrade of the memory module corresponding to this architecture. This is starting to be a consequential code (about 300 lines) so I will double-check it and push it as a commit on the PR rather than just in a comment. Here is what the architecture could look like :

I had already upgraded the entry to a MemoryEntry class, but I also find the namedtuples method quite interesting. I feel like with this version, we could do a hybrid between namedtuples for short term memory (which might store more info than the LT memory in most cases) and upgrade to a MemoryEntry (add some code to the consolidate method). What do you think about this potential new architecture ? Do you think that a hybrid approach would be viable, or too complex ?

Meanwhile, I'll try to be back with more usecases and some statistics for the performance as well as the code.

PS : all the methods for the different classes that I have coded here aren't necessarily definitive, I'd love hear what you think about them !

[EwoutH]

In this approach, does long-term memory simply have infinite capacity compared to a fixed amount of short term memory?

[colinfrisch]

In this approach, does long-term memory simply have infinite capacity compared to a fixed amount of short term memory?

That's what I envisioned, but I'm open to discussion ! There is a forget method that could be used for this if necessary.

[EwoutH]

Maintenance and codebase wise it might be beneficial to keep them as similar as possible - or even just one class - with capacity just being np.inf in one case.

[Corvince]

• @Corvince this seems like the stuff you could be interested in

Thanks for pinging me on this one! It really helps in these times where I am less active on mesa, even though it now took me some time to respond here.

Honestly, at the moment I think this is a nice thing to have for some specific models, but I am not sure about the general usefulness. Conceptually I find it difficult to determine what agent information should be stored in a simple attribute and what should be "memory". Also the concept of "memory sharing" sounds a bit strange to me, but the topic of communication between Agents is something I think we haven't really approached in a standard way. So this is an interesting discussion pathway. Also, it seems you are developing the memory feature further, which is great to see and I am looking forward to how this evolves

[colinfrisch]

Thank you for your feedback @Corvince !

Honestly, at the moment I think this is a nice thing to have for some specific models, but I am not sure about the general usefulness.

According to me, it could be useful for very basic learning patterns, to name a few :

• prey/predator models where remembering certain areas could be useful to the agent (and allow to model patterns like migrations, etc.)
• Basic social interaction models, where remembering who is trustworthy are not could also allow to model a few interesting effects (social networks forming, basic approach strategies, etc.)

I'm also trying to think a bit about the LLM approach, where memory is necessary. Might as well build it as something that all agents can use. Or do you think it's too much ?

Also the concept of "memory sharing" sounds a bit strange to me, but the topic of communication between Agents is something I think we haven't really approached in a standard way.

I see this as "communicating in general is just a way to make a memory of something (like a sentence) and transmitting it to another being". I understand it's unusual, but if the memory module works well, it could mean opening agent communication without building other complex modules. I'd really like to hear you opinion on this !

I'm also working on an upgrade of the memory to make it a little more useful and powerful. I should push it tonight or in the next few days.