Expose kafka connect in diferent ports

[ohuarte-matrix]

Hi all,

I have interest to use apache camel syslog connector, so I need to expose an UDP port
Is posible to configure port container for KafkaConnector or KafkaConnect ?
If ok, I suposse i have only to create a service to expose ports, but are there more steps?

Many thanks

[scholzj]

I don't think we support this. I don't think creating a service would be enough => the port would not be exposed on the Pod, so the service would not be able to access it.

[cwtrex]

@scholzj I was just about to ask about this. I identified that it works if you manually set the pods to expose the port.
My test method was external kubernetes tcp connection (streaming data) -> nginx ingest -> svc -> pod with exposed port (temporarily cause the operator would reset the pod)
I'm using the netty connector configured to 0.0.0.0 with port 8182.
While the port was active on the pod, traffic was flowing into the kafka topic without issue.

I can't direct traffic using a producer outside of the kubernetes setup due to not having any where to put external kafka brokers and the internal brokers are only able to use internal kubernetes dns. I attempted to use the ingress dns (single FQDN) and that ended very poorly.

So the solution (although potentially not actually in existence) would be exposing a port on a kafka connect cluster with service and then tying it to an ingress as described above as that got events into kafka without issue (and easily might I add).

So while the operator may not support it, @scholzj is there by chance a manual method to get this done? Obviously having the operator handle as much as possible would be great, but honestly any method to get around the dns and connectivity limitations using a method like the one above would be great. Thank you for any assistance on this.

[cwtrex]

@scholzj Just saw your responses on Slack. Much appreciated. Still looking around for the standalone Kafka connect documentation. Something is always better than nothing as far as solutions go and seems like the answer will be having two kafka connect clusters: 1 stand alone and one managed by the operator.

[ohuarte-matrix]

many thanks for your support

[cwtrex]

@ohuarte-matrix I created a feature request for the operator to handle this in the future if you want to add any additional information there for needs. Issue #5989
I'm also going to attempt to do a standalone by borrowing YAML from an auto generated cluster. I'll let you know how it goes.