Changes coming to JavaScript version

[Stevenic]

I'm thinking about re-structuring the JS version of AgentM a bit to follow the structure I've been using for a lot of my companies internal tools. Basically I want to create an AgentM CLI that consumes an agentm-core package. So all of the code that's currently in the agentm package would move to agentm-core and the current agentm package would turn into a CLI tool you could install and run globally. Applications would shift to importing agentm-core into their projects.

For the CLI tool I'm going to start with bundling in this AgentM Pulse server I've been working on but the idea is that should we ultimately need an agent runtime of some sort the CLI tool becomes that runtime. For example, if we add a bunch of evals the CLI tool becomes how you run the evals.

I don't know that it's critical for the python version to follow the same structure but thought I'd open it up for discussion. Thoughts?

[debuggerone]

Got stuck yesterday on the ci pipeline... will continue tomorrow and make that work.

Want to send errors coming up from the build to grafana - in general it works (sending simple logs works - dashboard shows them. But the error itself produces an error when it gets send to loki).

After that I will setup the core functionality like config, logs, token counts and preventing rate limit errors and parallel processing of the agents..

And then I'll use an autocoder to migrate the rest of the code add some tests and integrate the lib into the autocoder.

Both combined (logs in ci pipeline and agentm) should enhance the functionality of the autocoder which should help keeping up with the js lib.

[debuggerone]

for a cli version.. I'd say it is not neccessary.

python agentm.py run-eval

Would work just fine and pulse should be usable for both js and python backend.

[debuggerone]

Have fixed it by urlencoding the data before sending it to loki.. more like a hotfix.. don't want to get to deep in that rabbit hole now.
Maybe I can even leave it this way and decode it in a custom logprocessor in Loki or just leave it this way. GPT4o doesn't really care if it is decoded or not - it can read it anyways.

[debuggerone]

[debuggerone]

https://github.com/Stevenic/agentm-py/blob/main/INSTALL.md#4-directory-structure-before-running-installpy

proposal for structure - core functionality will just be added to the core folder - let's see how much we need to make a sense out of creating a core / baselib for it

I'd suggest to keep track on token usage and also api rate limits inside a sqlite database.

To create the settings.py and the database there should be a script called install.py which then runs you through the installation e.g. asks for an openai API key.

[Stevenic]

I'd suggest to keep track on token usage and also api rate limits inside a sqlite database.

One of the things I did in the JS version is to return usage details in a standardized way that should support tracking token usage and errors across a broad range of model providers. I stopped short of forming any opinions abut where that information is stored at. Ideally you want to be as unopinionated as possible. Some projects may be fine using sqlite but others will have strong preferences from some other technology so the best approach from an SDK perspective is to just not form an opinion about where things are stored.

You can and should totally have a plugin mechanism of some sort that makes it easy to use a database like sqlite but i wouldn't ship one in the box. That's based on me spending the last decade building and then having to document and maintain SDK's that were probably too opinionated.

The JS version of the library provides a logger that outputs usage & errors to the console but that's all I plan to support out of the box. It's super easy to add something more opinionated though.

[debuggerone]

I'll make it configurable in the install process then - same goes for logging - where the logs are saved to a path now but can also be pushed to a monitoring tool like grafana, logstash or graylog...