Tracee on k8s

[josedonizetti]

I have been testing tracee on K8S, and wondering about the current strategy of having two binaries into one container connected through a pipe. Is this the strategy tracee will be using or are others been considered? Like a single binary or for k8s deploying two different containers on the same pod, perhaps communicating over unix socket?

This questions are more related to some issues I have been looking into, for example:

• tracee-ebpf, and tracee-rules have different security requirements, so one needs less "capabilities" than the other, perhaps someone deploying it would prefer to limit each binary differently (Drop unnecessary capabilities on start #286)
• having observability for different binaries will require exposing two prometheus endpoint for monitoring on the same container for different purposes (Improve observability #887)
• we can't have different liveness/readiness probes to monitor each individual binary using only one container (Wait for tracee-ebpf to init before signaling readiness #930)
• supporting different configurations also require configuring different files for the same container, and if live reload is supported it implies that changing something on the rules side will also trigger a reload of the ebpf side (configuration from file #467, Tracee Configuration API #636)

[yanivagman]

I'm actually in favor of combining the two, and from some other reasons #820 (comment)
The reason we chose to split the binaries in first place was that we thought it might be useful to run tracee-rules standalone, as it may have other inputs which are not tracee-ebpf, and also be able to aggregate inputs from different nodes.
My current thoughts are that there are more reasons to combine the two rather than keeping them separate, as described above.