MirrorMaker 2 - Timed out waiting for a node assignment.

[john-mcpeek]

Hi, I'm trying to setup MM2. Pretty vanilla config. Just trying to get it running. Topics from the source cluster are not being created, but it does "seem" that it is talking to the source and target clusters. I can nc to the source cluster from the mm2 pod. There are multiple timeouts that keep happening.

2021-04-14 13:14:57,805 INFO [AdminClient clientId=adminclient-16] Metadata update failed (org.apache.kafka.clients.admin.internals.AdminMetadataManager) [kafka-admin-client-thread | adminclient-16] org.apache.kafka.common.errors.TimeoutException: Call(callName=fetchMetadata, deadlineMs=1618406097803, tries=1, nextAllowedTryMs=1618406097905) timed out at 1618406097805 after 1 attempt(s) Caused by: org.apache.kafka.common.errors.TimeoutException: Timed out waiting to send the call. 2021-04-14 13:14:57,806 INFO [AdminClient clientId=adminclient-16] Metadata update failed (org.apache.kafka.clients.admin.internals.AdminMetadataManager) [kafka-admin-client-thread | adminclient-16] org.apache.kafka.common.errors.TimeoutException: Call(callName=fetchMetadata, deadlineMs=1618406127805, tries=1, nextAllowedTryMs=-9223372036854775709) timed out at 9223372036854775807 after 1 attempt(s) Caused by: org.apache.kafka.common.errors.TimeoutException: Timed out waiting to send the call. 2021-04-14 13:14:57,809 ERROR Scheduler for MirrorSourceConnector caught exception in scheduled task: creating upstream offset-syncs topic (org.apache.kafka.connect.mirror.Scheduler) [Scheduler for MirrorSourceConnector-creating upstream offset-syncs topic] org.apache.kafka.connect.errors.ConnectException: Timed out while checking for or creating topic(s) 'mm2-offset-syncs.dev02.internal'. This could indicate a connectivity issue, unavailable topic partitions, or if this is your first use of the topic it may have taken too long to create. at org.apache.kafka.connect.util.TopicAdmin.createTopics(TopicAdmin.java:317) at org.apache.kafka.connect.mirror.MirrorUtils.createCompactedTopic(MirrorUtils.java:109) at org.apache.kafka.connect.mirror.MirrorUtils.createSinglePartitionCompactedTopic(MirrorUtils.java:114) at org.apache.kafka.connect.mirror.MirrorSourceConnector.createOffsetSyncsTopic(MirrorSourceConnector.java:283) at org.apache.kafka.connect.mirror.Scheduler.run(Scheduler.java:93) at org.apache.kafka.connect.mirror.Scheduler.executeThread(Scheduler.java:112) at org.apache.kafka.connect.mirror.Scheduler.lambda$execute$2(Scheduler.java:63) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515) at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:304) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) at java.base/java.lang.Thread.run(Thread.java:834) Caused by: org.apache.kafka.common.errors.TimeoutException: Call(callName=createTopics, deadlineMs=1618406097802, tries=1, nextAllowedTryMs=1618406097903) timed out at 1618406097803 after 1 attempt(s) Caused by: org.apache.kafka.common.errors.TimeoutException: Timed out waiting for a node assignment.
This is in an Openshift 4.5 and 4.6 cluster.

This happens no matter what I do:
error.txt

MirrorMaker2 yaml
mm2.txt

Kafka yaml
kafka.txt

[scholzj]

Well, from the exception it sounds like it is trying to create a topic mm2-offset-syncs.dev02.internal and fails. So I think you need to search the Kafka clusters for details why does it fail.

[tooptoop4]

@john-mcpeek did u solve?

[asatsi]

Having same issue here. Any luck anyone?