-
Notifications
You must be signed in to change notification settings - Fork 377
Description
Hi guys! I'm a newer using Kafka. It's a awesome project about Kafka Connector.
I use the MqttSourceConnector
to collect data from mqtt to kafka, just like the example.
name=mqtt-source
connector.class=io.lenses.streamreactor.connect.mqtt.source.MqttSourceConnector
tasks.max=1
connect.mqtt.kcql=INSERT INTO mqtt SELECT * FROM /mjson WITHCONVERTER=`io.lenses.streamreactor.connect.converters.source.JsonSimpleConverter`
[connect.mqtt.client.id](http://connect.mqtt.client.id/)=dm_source_id
connect.mqtt.hosts=tcp://mqtt:1883
connect.mqtt.service.quality=1
in property connect.mqtt.kcql
,the WITHCONVERTER is io.lenses.streamreactor.connect.converters.source.JsonSimpleConverter
.
When I test for it, I got message with some fields that i don't want to get, like schema
, and I also need to get some other information from mqtt. So, It's great for me to custom Converter
. Unfortunately, I can not find docs about How to custome Converter.
But get some information from the issue #1015
To implement your converter, you can check out the project and create a new module depending on
kafka-connect-common
and implementio.lenses.streamreactor.connect.converters.source.Converter
. Ensure that the resulting JAR is placed in your plugin.path location.
so I did. with some steps like:
git clone ....
- install scala and install sbt (because I'm a javaer)
- download many dependencies with bad network which make me crazy.
......
Finally, I complie and build the kafka-connect-common
successfully and i can't find the interface io.lenses.streamreactor.connect.converters.source.Converter
, but find it in kafka-connect-sql-common
, and i build and using it to custom my own Converter com.inside.ByteArraySourceConverter
just extends BytesConverter
. But unfortunately it not work correctly.
left is using io.lenses.streamreactor.connect.converters.source.JsonSimpleConverter
and it run correctly, right is bad, it just like stoped before Creating task.
So anybody can tell me what wrong with it and How can i cutom a Converter using with WITHCONVERTER
? Thank for you very much, Please forgive my poor English.