Proposal: split into a service and client

[evilhamsterman]

As I've been digging into the code I see a lot of what looks like workarounds and band aids to deal with the ephemerality of a clab. That plus the suid requirement for the binary gives me the heeby jeebies. Anyone can execute it and if they manage to find a vulnerability they have full root access. The fact it also messes around with critical files like /etc/hosts is also a little scary

I propose splitting the project into two parts, a service with elevated privileges that listens on a socket, and a client that communicates with the service over the socket. Similar to how Docker itself runs. This would provide several advantages

• A separation of concerns, leaving the administration logic to the service and the presentation to the client. Possibly even open up alternative clients like GUIs or TUIs
• Improve the security significantly by controlling access to the socket, again similar to Docker. The only thing users can run is the client which runs with suid.
• The service can maintain a consistent datastore, say in a KV like NutsDB or even just a JSON/YAML, of labs, obviating the need for the mix of labels, .bak files, and work arounds, and helping make sure things like --cleanup can always work
• Remove the need to modify /etc/hosts by providing a simple DNS service either embedded in the server or through something like CoreDNS with gRPC or an autoreloaded hosts file. Then use split DNS on different OS to forward to the server. See Tailscale for an example
• Remove the need to modify ssh_configs by making use of SSH Match and KnownHostsFile/KnownHostsCommand
• The client could run on any platform, say Windows while the service runs in WSL, or MacOS while the service runs in a Docker container or the new Containers in Tahoe.

Downside is it would be a major redesign

[hellt]

there are a lot of good ideas here, thanks @evilhamsterman

On the security concerns

Before diving into this, I wanted to highlight that primarily containerlab is intended for controlled lab environments where a bad actor should not have easy access to; At the same time the popularity of the tool made the installation targets to range from personal machines to shared labs with public access.

For the separation of security concerns there was a clab-api created that would allow the clab run on the host where the lab are scheduled, while the clients are using REST API to control the lab creation https://github.com/srl-labs/clab-api-server

With upcoming improvement to the clab code base which would decouple the CLI from the package API we would improve the clab-api-server to not use subshelling and use the go package directly.

The REST API should be then used to create clients, UI/CLI or otherwise.

Some notes:

• The clab "daemon" part will require suid/root, as we call netlink API for once.
• we still run containers in privileged mode, which means that having access to the container node and jailbreaking from it gives you root access. This can and should be changed with each node requesting its own capability sets.

Datastore

The ephemerality was a low-hanging fruit for us. Having a simple persistent datastore is a good change and can unlock more interesting capabilities.

But it needs to be invisible for a user, something that does not bring any installation requirements.

SSH configs

We do not modify the original ssh configs, we add to the list of ssh hosts via a new file in the ssh_config.d. If there is a user-scoped path where this could be added without requiring elevated privileges that would be nice.

[evilhamsterman]

Good to know you're already working on an api server project I'll take a look at that.

Security

The security concerns about not being in a controlled environment is my point. It does end up running on a lot of user devices that are a lot more exposed. Laptops of employees and students are a good example, they get taken around and connected to lots of uncontrolled networks. I'd place containerlab in the same field as projects like GNS3 or EVE-NG, it is a very effective tool for sharing and teaching lab designs. That's why I've been contributing a bunch recently I'm using it in my company to simulate our network and test/share ideas for changes.

It's not the worst thing but anything that could be improved is always a win I think. I believe you can drop the root requirement for accessing the netlink by instead of setting SUID or running as root, you can just setcap cap_net_admin+ep containerlab. Like I said for hosts you can run a DNS and setup split DNS for a local domain like .clab. Not an easy thing to do but possible read the Tailscale blog on doing it for a good example https://tailscale.com/blog/sisyphean-dns-client-linux.

Datastore

Agreed on not requiring extra installs, but the needs aren't high an embedded datastore BadgerDB, SQLite, or even just a folder of json files would probably be enough.

SSH

For SSH there's many options

For a start user ssh configs support Include as well so you could just put the same file in ~/.ssh/clab.d/lab.conf though it does require a one time addition to the users ~/.ssh/config
I do something like that in my own tool I've been working on https://github.com/evilhamsterman/tailshale. Though I had to eventually drop the include part because for some reason it doesn't handle the knownhostscommand in an included file. Though even if you did it at a system level you could add a file that uses Match exec instead of Hosts, then when ssh runs it will call your app and if it exits successfully it will use the following settings

example

Match exec `/usr/bin/containerlab check-host %h`
  User admin
  StrictHostKeyChecking=no
  UserKnownHostsFile=/dev/null

I've been hitting bugs and issues that I've personally run into while adopting containerlab at work and in my homelab. I hope they are appreciated and I will continue to do so. I may offer up some patches for options like I mentioned above for you to evaluate if that's alright.