[FEATURE]: Support running Mostly without FastAPI/Gunicorn in local mode

[spreeni]

Feature Summary

Mostly currently starts a FastAPI server that the client communicates with, even in local mode. It would be useful to bypass this server when running locally to avoid the overhead and simplify integration.

Problem and Solution

I noticed that mostly seems to run a fastapi server with which the client then communicates in local mode. I assume this is to be consistent with the cloud offering, which I understand. But it would be nice to have the option to directly run functions without the overhead of needing fastapi+gunicorn dependencies and running a server locally.

As I want to utilise mostly in a webservice of my own, this complicates that and leads to errors around nested event loops.

It's also a bit unintuitive that the Mostly client starts a webserver in the background - if that's correct - as it seems to prevent splitting the FastAPI service and client across separate Docker containers.

What’s the current recommended approach for using Mostly in a Dockerized web service? And could local-only usage work without the FastAPI intermediary?

Potential Alternatives

No response

Additional Context

No response

[shuangwu5]

as it seems to prevent splitting the FastAPI service and client across separate Docker containers.

Would our docker image help in this case?
With this, one would be able to launch the server in a container (let's call it A).
Then from your host machine or another container of your fastapi app (let's call it B), you could connect to it using the client mode of SDK with mostly = MostlyAI(base_url="http://localhost:8080")
If you're connecting from B, you might have to configure some docker network policies to ensure that port 8080 of A is reachable byB and set the right base_url accordingly.

[spreeni]

@shuangwu5 Thanks for the quick answer! Yeah true, that would help to separate the ressources and responsibilities.

I personally still went towards direct calling of the python functionalities to avoid the overhead of an extra web server and constant serialization/deserialization of passed data, so I think it could still be a useful feature.