Difference between automated RL tools like Bonsai, pathmind and Alpyne Library and Anylogic cloud

[farzaneh-mahmoudi]

Hello wolfeadam,

I have an agent-based simulation model on anylogic and I want to optimize it by reinforcement learning method.

As I understand there are 3 ways to integrate Simulation and RL:

1. Anylogic Cloud & API
2. RL automated tools (Bonsai or Pathmind)
3. Alpyne library

I was wondering which method is more suitable for my project.
Actually, the question is that when do we use each one of the three mentioned methods?

Thank you for your help

[t-wolfeadam]

Hi! So first, currently the AL Cloud does not have an API for RL. Also, Pathmind is no more, so that leaves Bonsai and Alpyne currently.

Bonsai is the optimal method for getting results quickly without having to worry about implementing RL algorithms; it has a lot of ways to express what you want the policy to learn and more technical options too (e.g., # of layers and their size).
The only roadblock I can think of is that you need to pay - for the Azure resources and possibly for the licensing an exported brain. You should contact them for a quote/estimate though as depending on your usage, it could end up being relatively cheap.

Alpyne is intended for people that want to try out their own RL algorithms or want a free option (excluding costs of your own electricity usage). TBH, it's currently in an unstable state and needs reworking. I'm currently the sole developer on it and other work tasks have consumed my time, but it is on my radar.

[farzaneh-mahmoudi]

Hi,
Thank you for the intact solution you suggested.
I had one more question: What do you mean that ALpyne is unstable and needs reworking?
Thanks,
Farzaneh

[t-wolfeadam]

What do you mean that ALpyne is unstable and needs reworking?

Essentially that there are known bugs with the startup of the backend server for some peoples' systems and the handling of these internal issues is inefficient. You may encounter no issues at all, but it's something to be aware of.