Kafka DevOps by Confluent uses shell-operator

[shurup]

Confluent has released its Kafka DevOps project which is a great example of using shell-operator.

This new project is a "simulated production environment running a streaming application targeting Apache Kafka on Confluent Cloud." Applications and resources there are managed via operators and GitOps approach. More details are available in the Confluent's blog.

Shell-operator is used to implement two operators residing inside Docker images.

Confluent Cloud Operator

The ccloud-operator utilizes the Confluent Cloud CLI and shell scripts built into a Docker image to implement a Kubernetes Operator style (Declare → Apply) solution from the command line. Using the shell-operator project enables us to build basic event-driven scripts, called hooks, which run inside the Kubernetes cluster, reacting to changes in Kubernetes resources. This allows us to declare basic Kubernetes ConfigMap resources, which defines the Confluent Cloud services we want to manage, and respond to changes to the resources automatically.

• More details.
• Code on GitHub.

Kafka Connect Operator

Connector configurations are defined in JSON and managed via the Connect worker HTTP API, which makes them well suited to the Declare → Apply model of Kubernetes. This is accomplished in kafka-devops using another shell-operator tool called connect-operator.

• More details.
• Code on GitHub.

[shurup]

From Confluent Developer Podcast (Dec 22, 2020) "Mastering DevOps with Apache Kafka, Kubernetes, and Confluent Cloud ft. Rick Spurgeon and Allison Walther":

To make writing operators much easier, Rick and Allison also share about Crossplane, KUDO, Shell-operator, and how to use these tools.