feat: Fix ccdt configuration and update docs

Contributes to mhub/qp-planning#1748

Signed-off-by: Katherine Stanley <katheris@uk.ibm.com>

Author: katheris

README.md

@@ -155,13 +155,30 @@ The configuration options for the Kafka Connect source connector for IBM MQ are
 | mq.queue                     | The name of the source MQ queue                             | string  |               | MQ queue name                                           |
 | mq.user.name                 | The user name for authenticating with the queue manager     | string  |               | User name                                               |
 | mq.password                  | The password for authenticating with the queue manager      | string  |               | Password                                                |
+| mq.ccdt.url                  | The URL for the CCDT file containing MQ connection details  | string  |               | URL for obtaining a CCDT file                           |
 | mq.record.builder            | The class used to build the Kafka Connect record            | string  |               | Class implementing RecordBuilder                        |
 | mq.message.body.jms          | Whether to interpret the message body as a JMS message type | boolean | false         |                                                         |
 | mq.record.builder.key.header | The JMS message header to use as the Kafka record key       | string  |               | JMSMessageID, JMSCorrelationID, JMSCorrelationIDAsBytes |
 | mq.ssl.cipher.suite          | The name of the cipher suite for TLS (SSL) connection       | string  |               | Blank or valid cipher suite                             |
 | mq.ssl.peer.name             | The distinguished name pattern of the TLS (SSL) peer        | string  |               | Blank or DN pattern                                     |
 | topic                        | The name of the target Kafka topic                          | string  |               | Topic name                                              |
 
+### Using a CCDT file
+Some of the connection details for MQ can be provided in a [CCDT file](https://www.ibm.com/support/knowledgecenter/en/SSFKSJ_9.0.0/com.ibm.mq.con.doc/q016730_.htm) by setting `mq.ccdt.url` in the Kafka Connect source connector configuration file. If this is set the following configuration options are no longer required: `mq.connection.name.list`, `mq.channel.name`.
+
+### Externalizing secrets
+[KIP 297](https://cwiki.apache.org/confluence/display/KAFKA/KIP-297%3A+Externalizing+Secrets+for+Connect+Configurations) introduced a mechanism to externalize secrets to be used as configuration for Kafka connectors. If using this mechanism the desired config provider must be configured in the Worker configuration file:
+
+```
+# Additional properties for the worker configuration to enable use of ConfigProviders
+# multiple comma-separated provider types can be specified here
+# config.providers=file
+# config.providers=file,other-provider
+# config.providers.file.class=org.apache.kafka.common.config.provider.FileConfigProvider
+# Other ConfigProvider implementations might require parameters passed in to configure() as follows:
+# config.providers.other-provider.param.foo=value1
+# config.providers.other-provider.param.bar=value2
+```
 
 ## Support
 A commercially supported version of this connector is available for customers with a support entitlement for [IBM Event Streams](https://developer.ibm.com/messaging/event-streams/).

config/mq-source.properties

@@ -24,7 +24,7 @@ mq.queue.manager=
 # A list of one or more host(port) entries for connecting to the queue manager. Entries are separated with a comma - required (unless using CCDT)
 mq.connection.name.list=
 
-# The name of the server-connection channel - equired (unless using CCDT)
+# The name of the server-connection channel - required (unless using CCDT)
 mq.channel.name=
 
 # The name of the source MQ queue - required
@@ -36,8 +36,13 @@ mq.queue=
 # The password for authenticating with the queue manager - optional
 # mq.password=
 
+# Alternatively can use a ConfigProvider for externalising secrets (see README.md for more details)
+# Variable references are of the form ${provider:[path:]key} where the path is optional,
+# depending on the ConfigProvider implementation.
+# mq.password=${file:/var/run/secret.properties:secret-key}
+
 # The CCDT URL to use to establish a connection to the queue manager - optional
-# mq.ccd.url=
+# mq.ccdt.url=
 
 # The record builders control conversion of data between the messages in MQ and the internal Kafka Connect representation - required
 mq.record.builder=com.ibm.eventstreams.connect.mqsource.builders.DefaultRecordBuilder

src/main/java/com/ibm/eventstreams/connect/mqsource/MQSourceConnector.java

@@ -166,11 +166,11 @@ public class MQSourceConnector extends SourceConnector {
                       CONFIG_DOCUMENTATION_MQ_QUEUE_MANAGER, CONFIG_GROUP_MQ, 1, Width.MEDIUM,
                       CONFIG_DISPLAY_MQ_QUEUE_MANAGER);
 
-        config.define(CONFIG_NAME_MQ_CONNECTION_NAME_LIST, Type.STRING, ConfigDef.NO_DEFAULT_VALUE, Importance.HIGH,
+        config.define(CONFIG_NAME_MQ_CONNECTION_NAME_LIST, Type.STRING, null, Importance.MEDIUM,
                       CONFIG_DOCUMENTATION_MQ_CONNNECTION_NAME_LIST, CONFIG_GROUP_MQ, 2, Width.LONG,
                       CONFIG_DISPLAY_MQ_CONNECTION_NAME_LIST);
 
-        config.define(CONFIG_NAME_MQ_CHANNEL_NAME, Type.STRING, ConfigDef.NO_DEFAULT_VALUE, Importance.HIGH,
+        config.define(CONFIG_NAME_MQ_CHANNEL_NAME, Type.STRING, null, Importance.MEDIUM,
                       CONFIG_DOCUMENTATION_MQ_CHANNEL_NAME, CONFIG_GROUP_MQ, 3, Width.MEDIUM,
                       CONFIG_DISPLAY_MQ_CHANNEL_NAME);