PLC4X Kafka Connect 0.12.0 – NoClassDefFoundError for PlcDriverManager in Docker

[i7pacheco]

Hello everyone,

I’m integrating PLC4X with Kafka Connect (version 0.12.0, I've tried 0.8.0 and 0.10.0 too) inside a Docker container to collect data from an S7 PLC. I’m running into persistent issues with plugin dependencies and classpath resolution, and would appreciate guidance from the community.

Environment

• Kafka Connect: 3.6.x (Confluent Platform, Dockerized)
• PLC4X Kafka Connector: 0.12.0
• Docker: Custom image, plugins in /usr/share/java/plc4x/
• OS: Linux (containerized)
• Goal: Connect to S7 PLC and publish data to a Kafka topic

What I’ve Done

• Placed all PLC4X JARs (including plc4j-apache-kafka-0.12.0.jar, plc4j-api-0.12.0.jar, plc4j-spi-0.12.0.jar, plc4j-driver-s7-0.12.0.jar, and other dependencies) in /usr/share/java/plc4x/.
• Set plugin.path to include /usr/share/java/.
• Verified that org.apache.plc4x.kafka.Plc4xSourceConnector appears in the /connector-plugins/ REST API output.
• Used the advanced connector configuration format.

Problem

When I POST the connector config, I get a 500 error and the logs show:

Caused by: java.lang.NoClassDefFoundError: org/apache/plc4x/java/PlcDriverManager
	at org.apache.plc4x.kafka.config.Source.validate(Source.java:76)
	...
Caused by: java.lang.ClassNotFoundException: org.apache.plc4x.java.PlcDriverManager
	at java.base/java.net.URLClassLoader.findClass(URLClassLoader.java:476)
	...

I also tried running jdeps -summary plc4j-apache-kafka-0.12.0.jar in the container and saw:

plc4j-apache-kafka-0.12.0.jar -> java.base
plc4j-apache-kafka-0.12.0.jar -> not found

What I’ve Checked

• All JARs are present in /usr/share/java/plc4x/ (listing below).
• No duplicate or mismatched versions.
• Restarted Kafka Connect after every change.
• The connector config uses the advanced format and references the correct classes.

Questions

1. What could cause NoClassDefFoundError for PlcDriverManager even when plc4j-api-0.12.0.jar is present?
2. Are there any known issues with PLC4X classloading in Dockerized Kafka Connect?
3. Is there a definitive list of all dependencies (including transitive ones) for the PLC4X Kafka connector 0.12.0?
4. Are there best practices for structuring the plugin path or for troubleshooting classpath issues in this scenario?
5. Any tips for debugging further or confirming exactly which JAR is missing or not being loaded?

Directory Listing

Here’s what’s in /usr/share/java/plc4x/:

plc4j-apache-kafka-0.12.0.jar
plc4j-api-0.12.0.jar
plc4j-spi-0.12.0.jar
plc4j-driver-s7-0.12.0.jar
commons-codec-*.jar
commons-lang3-*.jar
jackson-*.jar
bcprov-*.jar
bcpkix-*.jar
bcutil-*.jar
...

Connector Config (Snippet)

{
  "name": "plc4x-source",
  "config": {
    "connector.class": "org.apache.plc4x.kafka.Plc4xSourceConnector",
    "tasks.max": "1",
    "default-topic": "sensor_data",
    "sources": "plc1",
    "sources.plc1.connectionString": "s7://192.168.1.100/0/1",
    "sources.plc1.jobReferences": "jobA",
    "sources.plc1.jobReferences.jobA": "sensor_data",
    "jobs": "jobA",
    "jobs.jobA.interval": "1000",
    "jobs.jobA.fields": "tag1,tag2",
    "jobs.jobA.fields.tag1": "%DB501.DBD1:REAL",
    "jobs.jobA.fields.tag2": "%DB501.DBD5:REAL"
  }
}

The docker compose file is:

services:
  zookeeper:
    image: confluentinc/cp-zookeeper:7.6.0
    container_name: zookeeper
    environment:
      ZOOKEEPER_CLIENT_PORT: 2181
      ZOOKEEPER_TICK_TIME: 2000
    ports:
      - "2181:2181"

  kafka:
    image: confluentinc/cp-kafka:7.6.0
    container_name: kafka
    depends_on:
      - zookeeper
    ports:
      - "9092:9092"
      - "9093:9093"
    environment:
      KAFKA_BROKER_ID: 1
      KAFKA_ZOOKEEPER_CONNECT: zookeeper:2181
      KAFKA_LISTENERS: PLAINTEXT://0.0.0.0:9092,PLAINTEXT_HOST://0.0.0.0:9093
      KAFKA_ADVERTISED_LISTENERS: PLAINTEXT://kafka:9092,PLAINTEXT_HOST://localhost:9093
      KAFKA_LISTENER_SECURITY_PROTOCOL_MAP: PLAINTEXT:PLAINTEXT,PLAINTEXT_HOST:PLAINTEXT
      KAFKA_INTER_BROKER_LISTENER_NAME: PLAINTEXT
      KAFKA_AUTO_CREATE_TOPICS_ENABLE: "true"
      KAFKA_OFFSETS_TOPIC_REPLICATION_FACTOR: 1
    volumes:
      - ./kafka_data:/var/lib/kafka/data

  kafka-connect:
    image: confluentinc/cp-kafka-connect:7.6.0
    container_name: kafka-connect
    depends_on:
      - kafka
    environment:
      CONNECT_BOOTSTRAP_SERVERS: kafka:9092
      CONNECT_GROUP_ID: connect-cluster
      CONNECT_CONFIG_STORAGE_TOPIC: connect-configs
      CONNECT_OFFSET_STORAGE_TOPIC: connect-offsets
      CONNECT_STATUS_STORAGE_TOPIC: connect-status
      CONNECT_KEY_CONVERTER: org.apache.kafka.connect.json.JsonConverter
      CONNECT_VALUE_CONVERTER: org.apache.kafka.connect.json.JsonConverter
      CONNECT_PLUGIN_PATH: "/usr/share/java,/usr/share/confluent-hub-components,/usr/share/java/plc4x"
      CONNECT_REST_ADVERTISED_HOST_NAME: kafka-connect
      # Only working with a single node
      CONNECT_OFFSET_STORAGE_REPLICATION_FACTOR: 1
      CONNECT_CONFIG_STORAGE_REPLICATION_FACTOR: 1
      CONNECT_STATUS_STORAGE_REPLICATION_FACTOR: 1

    ports:
      - "8083:8083"
    volumes:
      - ./plugins:/plugins
      - ./timescale-sink.json:/data/timescale-sink.json

I prepared some series of bat files that help me set up the connectors:

Installing the PLC4X plugin:

@echo off
REM === CONFIGURATION ===
set "PLC4X_VERSION=0.12.0"
set "CONNECT_CONTAINER=kafka-connect"

REM === Step 1: Download the PLC4X Kafka Connect standalone ZIP ===
echo Downloading PLC4X Kafka Connect standalone ZIP...
curl -L -o plc4x-kafka.zip https://repo1.maven.org/maven2/org/apache/plc4x/plc4j-apache-kafka/%PLC4X_VERSION%/plc4j-apache-kafka-%PLC4X_VERSION%-standalone.zip

REM === Step 2: Extract the ZIP using PowerShell ===
echo Extracting PLC4X connector files...
powershell -Command "Expand-Archive -Path 'plc4x-kafka.zip' -DestinationPath 'plc4x-kafka' -Force"

REM === Step 3: Copy the connector JAR(s) into the Kafka Connect container ===
echo Copying connector JARs into the container...
docker exec %CONNECT_CONTAINER% mkdir -p /usr/share/java/plc4x
@REM for %%f in (plc4x-kafka\*.jar) do docker cp %%f %CONNECT_CONTAINER%:/usr/share/java/plc4x/
@REM for /r plc4x-kafka %%f in (*.jar) do docker cp "%%f" %CONNECT_CONTAINER%:/usr/share/java/
for /r plc4x-kafka %%f in (*.jar) do (
    echo Copying "%%f" to %CONNECT_CONTAINER%:/usr/share/java/plc4x/
    docker cp "%%f" %CONNECT_CONTAINER%:/usr/share/java/plc4x/
    if errorlevel 1 echo ERROR copying "%%f"
)

REM === Step 4: Restart the Kafka Connect container ===
echo Restarting Kafka Connect container...
docker restart %CONNECT_CONTAINER%

REM === Step 5: Wait for Kafka Connect to restart (adjust time if needed) ===
echo Waiting for Kafka Connect to restart...
timeout /t 30

REM === Step 6: Verify the connector is available ===
echo Checking available connector plugins:
docker exec %CONNECT_CONTAINER% curl http://localhost:8083/connector-plugins

REM === Step 7: Clean up temporary files ===
echo Cleaning up...
rmdir /s /q plc4x-kafka
del plc4x-kafka.zip

echo.
echo Done! If you see 'org.apache.plc4x.kafka.Plc4xSourceConnector' above, installation succeeded.
pause

Registering the plugin:

@echo off
REM === CONFIGURATION ===
set "PLC4X_VERSION=0.12.0"
set "CONNECT_CONTAINER=kafka-connect"

echo Registering PLC4X connector...
@REM docker cp plc4x-s7-source.json kafka-connect:/data/
@REM docker exec kafka-connect curl -X POST -H "Content-Type: application/json" --data @/data/plc4x-s7-source.json http://localhost:8083/connectors
docker cp plc4x-s7-source.json %CONNECT_CONTAINER%:/tmp/plc4x-s7-source.json
docker exec %CONNECT_CONTAINER% curl -X POST -H "Content-Type: application/json" --data @/tmp/plc4x-s7-source.json http://localhost:8083/connectors

echo Registered connectors:
docker exec %CONNECT_CONTAINER% curl -s http://localhost:8083/connectors

Updating the plugin:

@echo off
REM === CONFIGURATION ===
set "CONNECT_CONTAINER=kafka-connect"
set "CONNECTOR_NAME=plc4x-source"
set "CONFIGURATION_FILENAME=plc4x-s7-source"
set "FULL_JSON=%CONFIGURATION_FILENAME%.json"
set "CONFIG_ONLY_JSON=%CONFIGURATION_FILENAME%-config-only.json"

REM === Extract only the config object using PowerShell ===
echo Extracting config section from %FULL_JSON%...
powershell -Command "Get-Content %FULL_JSON% | ConvertFrom-Json | Select-Object -ExpandProperty config | ConvertTo-Json | Set-Content %CONFIG_ONLY_JSON%"

REM === Copy the config-only JSON to the container ===
echo Copying config-only JSON to container...
docker cp %CONFIG_ONLY_JSON% %CONNECT_CONTAINER%:/tmp/%CONFIG_ONLY_JSON%

REM === Update the connector using PUT ===
echo Updating PLC4X connector...
docker exec %CONNECT_CONTAINER% curl -X PUT -H "Content-Type: application/json" --data @/tmp/%CONFIG_ONLY_JSON% http://localhost:8083/connectors/%CONNECTOR_NAME%/config

REM === Show registered connectors ===
echo Registered connectors:
docker exec %CONNECT_CONTAINER% curl -s http://localhost:8083/connectors

REM === Show connector configuration ===
echo Current configuration of %CONNECTOR_NAME%:
docker exec %CONNECT_CONTAINER% curl -s http://localhost:8083/connectors/%CONNECTOR_NAME%/config

REM === Optional: Clean up temp file ===
del %CONFIG_ONLY_JSON%

Any help or troubleshooting suggestions would be greatly appreciated!

Thanks in advance.

[chrisdutz]

Just to double check ... does Kafka traverse into sub-directories? So does it find the jars inside the "plc4x" folder inside the shared directory?

However I think even if I wrote the initial Kafka plugin, that's like 7-8 years ago and as Confluent kept on wanting us to keep it updated for free, I sort of dropped it from my schedule some time ago. I am 100% certain that this part of PLC4X could use quite a bit of updating. Are you interested in doing that?