Skip to content

Commit 8c8ab8f

Browse files
committed
Add instructions to run in distributed mode
- Add sample json file under /config
1 parent f20706d commit 8c8ab8f

File tree

2 files changed

+30
-1
lines changed

2 files changed

+30
-1
lines changed

README.md

Lines changed: 14 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -59,6 +59,8 @@ To run the connector, you must have:
5959

6060
The connector can be run in a Kafka Connect worker in either standalone (single process) or distributed mode. It's a good idea to start in standalone mode.
6161

62+
### Running in standalone mode
63+
6264
You need two configuration files, one for the configuration that applies to all of the connectors such as the Kafka bootstrap servers, and another for the configuration specific to the MQ source connector such as the connection information for your queue manager. For the former, the Kafka distribution includes a file called `connect-standalone.properties` that you can use as a starting point. For the latter, you can use `config/mq-source.properties` in this repository.
6365

6466
The connector connects to MQ using either a client or a bindings connection. For a client connection, you must provide the name of the queue manager, the connection name (one or more host/port pairs) and the channel name. In addition, you can provide a user name and password if the queue manager is configured to require them for client connections. If you look at the supplied `config/mq-source.properties`, you'll see how to specify the configuration required. For a bindings connection, you must provide provide the name of the queue manager and also run the Kafka Connect worker on the same system as the queue manager.
@@ -69,9 +71,20 @@ To run the connector in standalone mode from the directory into which you instal
6971
bin/connect-standalone.sh connect-standalone.properties mq-source.properties
7072
```
7173

74+
### Running in distributed mode
75+
76+
You need an instance of Kafka Connect running in distributed mode. The Kafka distribution includes a file called `connect-distributed.properties` that you can use as a starting point, or follow [Running with Docker](#running-with-docker) or [Deploying to Kubernetes](#deploying-to-kubernetes)
77+
78+
To start the MQ connector, you can use `config/mq-source.json` in this repository after replacing all placeholders and use a command like this:
79+
80+
```shell
81+
curl -X POST -H "Content-Type: application/json" http://localhost:8083/connectors \
82+
--data "@./config/mq-source.json"
83+
```
84+
7285
## Running with Docker
7386

74-
This repository includes a Dockerfile to run Kafka Connect in distributed mode. It also adds in the MQ Source Connector as an available connector plugin. It uses the default connect-distributed.properties and connect-log4j.properties files.
87+
This repository includes a Dockerfile to run Kafka Connect in distributed mode. It also adds in the MQ Source Connector as an available connector plugin. It uses the default `connect-distributed.properties` and `connect-log4j.properties` files.
7588

7689
1. `mvn clean package`
7790
1. `docker build -t kafkaconnect-with-mq-source:0.0.1 .`

config/mq-source.json

Lines changed: 16 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,16 @@
1+
{
2+
"name": "mq-source",
3+
"config":{
4+
"connector.class": "com.ibm.eventstreams.connect.mqsource.MQSourceConnector",
5+
"tasks.max": "1",
6+
"topic": "<TOPIC>",
7+
8+
"value.converter": "org.apache.kafka.connect.converters.ByteArrayConverter",
9+
10+
"mq.queue.manager": "<QUEUE_MANAGER>",
11+
"mq.connection.name.list": "<CONNECTION_NAME_LIST>",
12+
"mq.channel.name": "<CHANNEL_NAME>",
13+
"mq.queue": "<QUEUE>",
14+
"mq.record.builder": "com.ibm.eventstreams.connect.mqsource.builders.DefaultRecordBuilder"
15+
}
16+
}

0 commit comments

Comments
 (0)