[Spring Boot + Kafka + Avro] Model Application

Prerequisites

Java

The project is based on Java Development Kit (JDK) version 17. You will need to install JDK 17 (or later) on your development machine.

Maven

mvn clean is a command used in Maven build automation tool to remove any files and directories generated by previous builds. When you execute the mvn clean command, it will delete the target directory of the project. This directory is where the compiled classes, generated resources, and other build artifacts are stored.

The project is using Apache Avro data serialization format. The code can be generated automatically with mvn generate-sources.

To run Maven tests, you can use the mvn test command in the project's root directory. This will compile the project's source code and execute all unit tests found in the src/test/java directory.

Kafka

The project uses the Apache Kafka distributed streaming platform. You can run Kafka locally or use managed Kafka provide by Confluent Cloud. Currently, there are 2 popular ways to run Kafka locally: docker & confluent cli.

Option 1: Docker

You can run Apache Kafka in Docker. There is a docker-compose.yml provided for you. It will spin up a single Zookeeper instance, single Kafka instance & single Schema Registry instance. Run it with docker-compose up or docker-compose up -d (for detached mode: run containers in the background).

Option 2: Confluent CLI

Please, see https://docs.confluent.io/confluent-cli/current/overview.html for details.

When installed, make sure that provided right JAVA_HOME and CONFLUENT_HOME environment configurations. Adding confluent binaries to the PATH might be helpful too.

export JAVA_HOME="/usr/local/opt/openjdk@11"
export CONFLUENT_HOME="~/confluent/confluent-7.3.1"
export PATH="$PATH:~/confluent/confluent-7.3.1/bin"

Clean up your local setup with

confluent local destroy

Start required services with

confluent local services kafka start
confluent local services schema-registry start
confluent local services connect start

Run Spring Boot Application

You can pass a directory to --spring.config.location. Spring will load only the file application.properties and profile specific file like application-{profile name}.properties.

Then you can start your applications as

# Producer
mvn spring-boot:run -Dmaven.test.skip=true -Dstart-class=io.confluent.producer.BasicProducerApplication \
-Dspring-boot.run.profiles=local-encrypted-payload

# Consumer
mvn spring-boot:run -Dmaven.test.skip=true -Dstart-class=io.confluent.consumer.BasicConsumerApplication \
-Dspring-boot.run.profiles=local-encrypted-payload

# Streams
mvn spring-boot:run -Dmaven.test.skip=true -Dstart-class=io.confluent.streams.CountApplication \
-Dspring-boot.run.profiles=local-encrypted-payload

# Web
mvn spring-boot:run -Dmaven.test.skip=true -Dstart-class=io.confluent.web.BasicWebApplication \
-Dspring-boot.run.profiles=local-encrypted-payload -Dspring.main.web-application-type=servlet

Contributing

Contributions to the project are welcome. To contribute, fork the repository, make your changes, and submit a pull request. Please ensure that your changes are well-documented and thoroughly tested.