Strimzi
Replies: 6 comments 19 replies
-
|
From the incomplete log it is quite hard to tell which part failed the TLS. Whether it is the OAuth or the Connect part. In general:
|
Beta Was this translation helpful? Give feedback.
1 reply
-
|
Thanks for the answers. Here my replies:
If you can please clarify to me point 2 I think I am good to try again :) |
Beta Was this translation helpful? Give feedback.
-
This section of the Kafka CR: configuration:
brokerCertChainAndKey:
secretName: tls-stage-data-property-xyz-p
certificate: tls.crt
key: tls.keytells the operator to configure the broker to use this certificate as the served certificate in the ingress listener. So when you connect to it, you need to trust this certificate. So this section in the KafkaConnect CR: tls:
trustedCertificates:
- secretName: kakfa-stage-cluster-ca-cert
certificate: ca.crtNeeds to refer to a public key of a certificate which signed the one in |
Beta Was this translation helpful? Give feedback.
2 replies
-
|
ok, got it all wrong then :) Thanks a lot for the explanation. Let me try to set it up correctly. |
Beta Was this translation helpful? Give feedback.
-
|
No worries. Give it a try and let me know if it helped or not. |
Beta Was this translation helpful? Give feedback.
-
|
@scholzj I am bit confused here. The secret kakfa-stage-cluster-ca has the following keys:
Therefore, my question is: what should I put in the certificate key section? Apologies for the dumb question. |
Beta Was this translation helpful? Give feedback.
5 replies
-
|
The If you want to use the the certificates issues by the Strimzi Cluster CA, you do not need to configure anything in |
Beta Was this translation helpful? Give feedback.
-
|
perfect! so in Kafka connect I have this now: and here the Kafka part: Would it be ok? |
Beta Was this translation helpful? Give feedback.
-
|
Guess that this should be on port 9093? |
Beta Was this translation helpful? Give feedback.
-
|
Well, in your configuration you shared on the beginning, you had the port 9092 completely without TLS (and without authentication) - so if you wanna use that, you can do something like this: apiVersion: kafka.strimzi.io/v1beta1
kind: KafkaConnect
metadata:
# ...
spec:
version: 2.6.0
replicas: 1
bootstrapServers: kakfa-stage-kafka-tls-bootstrap:9092
config:
group.id: connect-cluster
offset.storage.topic: connect-cluster-offsets
config.storage.topic: connect-cluster-configs
status.storage.topic: connect-cluster-statusYour listener on 9093 is the ingress one. So for that, you should use the |
Beta Was this translation helpful? Give feedback.
-
|
You also seem to have typo in the OAuth config for Connect ... |
Beta Was this translation helpful? Give feedback.
-
|
Same error with above configuration: |
Beta Was this translation helpful? Give feedback.
8 replies
-
|
changed to internal svc uri: |
Beta Was this translation helpful? Give feedback.
-
|
|
Beta Was this translation helpful? Give feedback.
-
|
that is a very good question! I have no idea. How can I find it out? |
Beta Was this translation helpful? Give feedback.
-
|
You can start by sharing the full log and not just part of a single error message ;-). |
Beta Was this translation helpful? Give feedback.
-
|
:) Since that was a staging enviroment, I reverted back to plain communication. I am now creating the prod cluster. Soon as I hit the error, I will post the logs. |
Beta Was this translation helpful? Give feedback.
-
|
I have a similar problem The --enable-ssl-passthrough flag enables the SSL Passthrough feature, args:
- /nginx-ingress-controller
- --election-id=ingress-controller-leader
- --ingress-class=nginx
- --configmap=$(POD_NAMESPACE)/ingress-nginx-controller
- --validating-webhook=:8443
- --validating-webhook-certificate=/usr/local/certificates/cert
- --validating-webhook-key=/usr/local/certificates/key
- --enable-ssl-passthroughHere is my configuration for Kafka: apiVersion: kafka.strimzi.io/v1beta1
kind: Kafka
metadata:
name: cpaas-system
namespace: kafka
spec:
entityOperator:
topicOperator: {}
userOperator: {}
kafka:
authorization:
superUsers:
- broker-user
- ANONYMOUS
type: simple
config:
log.message.format.version: 2.5
offsets.topic.replication.factor: 1
transaction.state.log.min.isr: 1
transaction.state.log.replication.factor: 1
jmxOptions:
"-Xms": 8192m
"-Xmx": 8192m
listeners:
external:
type: ingress
tls: true
configuration:
bootstrap:
host: bootstrap.myingress.com
brokers:
- broker: 0
host: broker-0.myingress.com
- broker: 1
host: broker-1.myingress.com
- broker: 2
host: broker-2.myingress.com
plain:
authentication:
type: scram-sha-512
replicas: 3
resources:
limits:
cpu: "1"
memory: 1Gi
requests:
cpu: 500m
memory: 500Mi
storage:
type: ephemeral
template:
pod:
securityContext:
fsGroup: 0
runAsUser: 0
version: 2.5.0
zookeeper:
jmxOptions:
"-Xms": 4096m
"-Xmx": 4096m
replicas: 3
resources:
limits:
cpu: "1"
memory: 1000Mi
requests:
cpu: 500m
memory: 500Mi
storage:
type: ephemeral
template:
pod:
securityContext:
fsGroup: 0
runAsUser: 0
$ kubectl -n kafka get secret cpaas-system-cluster-ca-cert -o jsonpath='{.data.ca\.crt}' | base64 -d > ca.crt
$ kubectl -n kafka get secret cpaas-system-cluster-ca-cert -o jsonpath='{.data.ca\.password}' | base64 -d
$ keytool -keystore user-truststore.jks -alias CARoot -import -file ca.crt
cat << EOF > ingress.properties
security.protocol=SSL
ssl.truststore.location=user-truststore.jks
ssl.truststore.password=password
ssl.endpoint.identification.algorithm=
EOF
$ kubectl -n ingress-nginx get services
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
ingress-nginx-controller NodePort 10.104.18.239 <none> 80:30491/TCP,443:30214/TCP 7h4m
ingress-nginx-controller-admission ClusterIP 10.106.60.149 <none> 443/TCP 7h4m
$ kafka-topics.sh --bootstrap-server bootstrap.myingress.com:30214 --command-config ./ingress.properties --list
[2021-02-20 18:11:14,133] WARN The configuration 'ssl.truststore.location' was supplied but isn't a known config. (org.apache.kafka.clients.admin.AdminClientConfig)
[2021-02-20 18:11:14,133] WARN The configuration 'ssl.truststore.password' was supplied but isn't a known config. (org.apache.kafka.clients.admin.AdminClientConfig)
[2021-02-20 18:11:14,133] WARN The configuration 'ssl.endpoint.identification.algorithm' was supplied but isn't a known config. (org.apache.kafka.clients.admin.AdminClientConfig)
[2021-02-20 18:11:14,617] ERROR [AdminClient clientId=adminclient-1] Connection to node 1 (broker-1.myingress.com/10.0.129.171:443) failed authentication due to: SSL handshake failed (org.apache.kafka.clients.NetworkClient)
[2021-02-20 18:11:14,619] WARN [AdminClient clientId=adminclient-1] Metadata update failed due to authentication error (org.apache.kafka.clients.admin.internals.AdminMetadataManager)
org.apache.kafka.common.errors.SslAuthenticationException: SSL handshake failed
Caused by: javax.net.ssl.SSLHandshakeException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target
at sun.security.ssl.Alert.createSSLException(Alert.java:131)
at sun.security.ssl.TransportContext.fatal(TransportContext.java:324)
at sun.security.ssl.TransportContext.fatal(TransportContext.java:267)
at sun.security.ssl.TransportContext.fatal(TransportContext.java:262)
at sun.security.ssl.CertificateMessage$T12CertificateConsumer.checkServerCerts(CertificateMessage.java:654)
at sun.security.ssl.CertificateMessage$T12CertificateConsumer.onCertificate(CertificateMessage.java:473)
at sun.security.ssl.CertificateMessage$T12CertificateConsumer.consume(CertificateMessage.java:369)
at sun.security.ssl.SSLHandshake.consume(SSLHandshake.java:377)
at sun.security.ssl.HandshakeContext.dispatch(HandshakeContext.java:444)
at sun.security.ssl.SSLEngineImpl$DelegatedTask$DelegatedAction.run(SSLEngineImpl.java:968)
at sun.security.ssl.SSLEngineImpl$DelegatedTask$DelegatedAction.run(SSLEngineImpl.java:955)
at java.security.AccessController.doPrivileged(Native Method)
at sun.security.ssl.SSLEngineImpl$DelegatedTask.run(SSLEngineImpl.java:902)
at org.apache.kafka.common.network.SslTransportLayer.runDelegatedTasks(SslTransportLayer.java:425)
at org.apache.kafka.common.network.SslTransportLayer.handshakeUnwrap(SslTransportLayer.java:509)
at org.apache.kafka.common.network.SslTransportLayer.doHandshake(SslTransportLayer.java:363)
at org.apache.kafka.common.network.SslTransportLayer.handshake(SslTransportLayer.java:286)
at org.apache.kafka.common.network.KafkaChannel.prepare(KafkaChannel.java:174)
at org.apache.kafka.common.network.Selector.pollSelectionKeys(Selector.java:547)
at org.apache.kafka.common.network.Selector.poll(Selector.java:485)
at org.apache.kafka.clients.NetworkClient.poll(NetworkClient.java:549)
at org.apache.kafka.clients.admin.KafkaAdminClient$AdminClientRunnable.processRequests(KafkaAdminClient.java:1272)
at org.apache.kafka.clients.admin.KafkaAdminClient$AdminClientRunnable.run(KafkaAdminClient.java:1203)
at java.lang.Thread.run(Thread.java:748)
Caused by: sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target
at sun.security.validator.PKIXValidator.doBuild(PKIXValidator.java:456)
at sun.security.validator.PKIXValidator.engineValidate(PKIXValidator.java:323)
at sun.security.validator.Validator.validate(Validator.java:271)
at sun.security.ssl.X509TrustManagerImpl.validate(X509TrustManagerImpl.java:315)
at sun.security.ssl.X509TrustManagerImpl.checkTrusted(X509TrustManagerImpl.java:278)
at sun.security.ssl.X509TrustManagerImpl.checkServerTrusted(X509TrustManagerImpl.java:141)
at sun.security.ssl.CertificateMessage$T12CertificateConsumer.checkServerCerts(CertificateMessage.java:632)
... 19 more
Caused by: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target
at sun.security.provider.certpath.SunCertPathBuilder.build(SunCertPathBuilder.java:141)
at sun.security.provider.certpath.SunCertPathBuilder.engineBuild(SunCertPathBuilder.java:126)
at java.security.cert.CertPathBuilder.build(CertPathBuilder.java:280)
at sun.security.validator.PKIXValidator.doBuild(PKIXValidator.java:451)
... 25 more
Error while executing topic command : org.apache.kafka.common.errors.SslAuthenticationException: SSL handshake failed
[2021-02-20 18:11:14,624] ERROR java.util.concurrent.ExecutionException: org.apache.kafka.common.errors.SslAuthenticationException: SSL handshake failed
at org.apache.kafka.common.internals.KafkaFutureImpl.wrapAndThrow(KafkaFutureImpl.java:45)
at org.apache.kafka.common.internals.KafkaFutureImpl.access$000(KafkaFutureImpl.java:32)
at org.apache.kafka.common.internals.KafkaFutureImpl$SingleWaiter.await(KafkaFutureImpl.java:89)
at org.apache.kafka.common.internals.KafkaFutureImpl.get(KafkaFutureImpl.java:260)
at kafka.admin.TopicCommand$AdminClientTopicService.getTopics(TopicCommand.scala:333)
at kafka.admin.TopicCommand$AdminClientTopicService.listTopics(TopicCommand.scala:252)
at kafka.admin.TopicCommand$.main(TopicCommand.scala:66)
at kafka.admin.TopicCommand.main(TopicCommand.scala)
Caused by: org.apache.kafka.common.errors.SslAuthenticationException: SSL handshake failed
Caused by: javax.net.ssl.SSLHandshakeException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target
at sun.security.ssl.Alert.createSSLException(Alert.java:131)
at sun.security.ssl.TransportContext.fatal(TransportContext.java:324)
at sun.security.ssl.TransportContext.fatal(TransportContext.java:267)
at sun.security.ssl.TransportContext.fatal(TransportContext.java:262)
at sun.security.ssl.CertificateMessage$T12CertificateConsumer.checkServerCerts(CertificateMessage.java:654)
at sun.security.ssl.CertificateMessage$T12CertificateConsumer.onCertificate(CertificateMessage.java:473)
at sun.security.ssl.CertificateMessage$T12CertificateConsumer.consume(CertificateMessage.java:369)
at sun.security.ssl.SSLHandshake.consume(SSLHandshake.java:377)
at sun.security.ssl.HandshakeContext.dispatch(HandshakeContext.java:444)
at sun.security.ssl.SSLEngineImpl$DelegatedTask$DelegatedAction.run(SSLEngineImpl.java:968)
at sun.security.ssl.SSLEngineImpl$DelegatedTask$DelegatedAction.run(SSLEngineImpl.java:955)
at java.security.AccessController.doPrivileged(Native Method)
at sun.security.ssl.SSLEngineImpl$DelegatedTask.run(SSLEngineImpl.java:902)
at org.apache.kafka.common.network.SslTransportLayer.runDelegatedTasks(SslTransportLayer.java:425)
at org.apache.kafka.common.network.SslTransportLayer.handshakeUnwrap(SslTransportLayer.java:509)
at org.apache.kafka.common.network.SslTransportLayer.doHandshake(SslTransportLayer.java:363)
at org.apache.kafka.common.network.SslTransportLayer.handshake(SslTransportLayer.java:286)
at org.apache.kafka.common.network.KafkaChannel.prepare(KafkaChannel.java:174)
at org.apache.kafka.common.network.Selector.pollSelectionKeys(Selector.java:547)
at org.apache.kafka.common.network.Selector.poll(Selector.java:485)
at org.apache.kafka.clients.NetworkClient.poll(NetworkClient.java:549)
at org.apache.kafka.clients.admin.KafkaAdminClient$AdminClientRunnable.processRequests(KafkaAdminClient.java:1272)
at org.apache.kafka.clients.admin.KafkaAdminClient$AdminClientRunnable.run(KafkaAdminClient.java:1203)
at java.lang.Thread.run(Thread.java:748)
Caused by: sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target
at sun.security.validator.PKIXValidator.doBuild(PKIXValidator.java:456)
at sun.security.validator.PKIXValidator.engineValidate(PKIXValidator.java:323)
at sun.security.validator.Validator.validate(Validator.java:271)
at sun.security.ssl.X509TrustManagerImpl.validate(X509TrustManagerImpl.java:315)
at sun.security.ssl.X509TrustManagerImpl.checkTrusted(X509TrustManagerImpl.java:278)
at sun.security.ssl.X509TrustManagerImpl.checkServerTrusted(X509TrustManagerImpl.java:141)
at sun.security.ssl.CertificateMessage$T12CertificateConsumer.checkServerCerts(CertificateMessage.java:632)
... 19 more
Caused by: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target
at sun.security.provider.certpath.SunCertPathBuilder.build(SunCertPathBuilder.java:141)
at sun.security.provider.certpath.SunCertPathBuilder.engineBuild(SunCertPathBuilder.java:126)
at java.security.cert.CertPathBuilder.build(CertPathBuilder.java:280)
at sun.security.validator.PKIXValidator.doBuild(PKIXValidator.java:451)
... 25 more
(kafka.admin.TopicCommand$)
[
|
Beta Was this translation helpful? Give feedback.
2 replies
-
|
The ingress always connects through port 443. So connecting to |
Beta Was this translation helpful? Give feedback.
-
|
I found the problem, thanks |
Beta Was this translation helpful? Give feedback.
-
|
Adding the ca certificate to cacerts still doesn't work keytool -keystore /usr/lib/jvm/java-1.8.0-openjdk-1.8.0.272.b10-1.el7_9.x86_64/jre/lib/security/cacerts -alias kafka-zookeeper-ca-cert -import -file ca.crt
|
Beta Was this translation helpful? Give feedback.
1 reply
-
|
doesn't work. |
Beta Was this translation helpful? Give feedback.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Uh oh!
There was an error while loading. Please reload this page.
-
I am struggling to get Kafka connect to communicate with Kafka over TLS with Oauth2.
Here is my configuration for Kafka:
and here the configuration for Kafka-Connect:
Here the error from Kafka Connect:
Beta Was this translation helpful? Give feedback.
All reactions