Agent auto-detect

[zorbaTheRainy]

Describe the feature you would like to see

Right now you add agents to the host or principal device via the DOZZLE_REMOTE_AGENT ENV. For multiple agents, you comma separate them.

However, if you have a lot of agents, this can get out of hand.

It would be nice if the agents/principal could automatically find each other.****

Describe how you would like to see this feature implemented

I can think of a few ways to do this, but the easier way is to have the agent contact the principal.

1. Create an ENV DOZZLE_REMOTE_PRINCIPAL that is used by the agent. It includes the URL of the principal (e.g, http://192.168.1.2:8080).
2. The agent then sends a message to the principle saying "I am here and I want to connect. My URL is 192.168.1.3:7007."
3. The principle than adds that agent to its internal DOZZLE_REMOTE_AGENT array/list and connects per normal.

That way when I add a new Dozzle agent to the system, I don't have to edit the principal's compose.yaml and restart it (adding the agent's IP to a long list of IPs), instead I just have to point the agent to principal.

This is similar to traefik's labels but on different non-swarm nodes.
The connection is non-persistent. When you restart the principal you lose all the agents (not imported via the principal's DOZZLE_REMOTE_AGENT ENV), and it is up to the agent to repeat step #2.

Describe any alternatives you've considered

Alternatively, the DOZZLE_REMOTE_AGENT ENV could accept an IP range (e.g., 192.168.1.0/24:7007) and the principal could poll the range to see if an agent exists. Again, adding any found to the internal representation of the DOZZLE_REMOTE_AGENT ENV.

[amir20]

I like the idea. When I first introduced agents, I spent a lot of time examining broadcast protocols for automating discovery, but I was not very successful.

I hold two principles for Dozzle:

1. There shouldn't be multiple ways to accomplish the same task, as maintenance will be difficult.
2. The solution should work for everyone.

Regarding your solution 1, there are two problems:

1. I have seen many instances where agents are deployed in an internal network, but the UI is hosted elsewhere. The UI can access the agent, but the agent cannot access the UI. This setup wouldn't work.
2. Dozzle lacks data storage. The only data storage available is /data when authentication is set up. This limitation makes it harder for users to restart the Dozzle UI, as everything would disappear until the agent pings again. I don't think this provides a good user experience. Dozzle containers are meant to be ephemeral, which is why many configurations are handled through ENV variables.

Alternatively, the DOZZLE_REMOTE_AGENT ENV could accept an IP range

However, this would lead to performance issues because Dozzle would need to ping every IP in that range, and each ping has associated latency.

If there is a better way to implement agent discovery, I would love to explore it. I just can't think of any. Agents can run on different hosts and networks, making it difficult for them to discover each other.

[zorbaTheRainy]

All valid points.
I can't solve issue 1. So, I think that kills the idea.
As for issue 2, if you wanted instant availability, you'd put those IPs in the compose's ENV. But, as I said, issue 1 is the killer problem.

Another way I can think of is, since the agent assignment already has 2 ways of doing things (command line args and ENV), how about a third, an /data/agents.yml file?
or even agents.txt as you only need 1 IP:port per line.
Like with users.yml this would only be something a user would setup if they wanted something "extra".

That really isn't "automatic" but it would be more manageable for a user with have a lot of agents.

[amir20]

What's your use case? How many agents do you have? User config is complex so it makes sense to a have file. Agents are just comma separated.

I was playing around with https://github.com/schollz/croc for a while. It uses a public relay unfortunately but it works. Dozzle could do something similar where agents are registered.

Do you know what podman does? I don't use it but that is the only other similar tool that has a agent.

Personally, I am pretty tight on time so if comma separated works, then I'd say that's good enough until agent config gets more complicated.