同步数据库,

[dlvkin]

我现在正在做一个mysql binlog 同步到hdfs上

StreamExecutionEnvironment env = StreamExecutionEnvironment.getExecutionEnvironment();
env.setParallelism(1);
env.getCheckpointConfig().setCheckpointTimeout(60000L);
env.getCheckpointConfig().setMinPauseBetweenCheckpoints(500);
env.getCheckpointConfig().setTolerableCheckpointFailureNumber(10);
env.getCheckpointConfig().setCheckpointingMode(CheckpointingMode.EXACTLY_ONCE);
env.getCheckpointConfig().setExternalizedCheckpointCleanup(CheckpointConfig.ExternalizedCheckpointCleanup.RETAIN_ON_CANCELLATION);
env.setRestartStrategy(RestartStrategies.fixedDelayRestart(3, org.apache.flink.api.common.time.Time.of(330, TimeUnit.SECONDS)));
env.setStateBackend(new RocksDBStateBackend("hdfs://192.168.1.200:9000/out/hive/warehouse/flink/ck/", true));

//sinnk
SelfDateTimeBucketWithPartitionAssigner bucketAssigner = new SelfDateTimeBucketWithPartitionAssigner<>("yyyy-MM-dd", ZoneId.of("Asia/Shanghai"));
final StreamingFileSink streamingFileSink = StreamingFileSink
.forBulkFormat(new Path(writeFs), factory)
.withRollingPolicy(OnCheckpointRollingPolicy.build())
.withBucketAssigner(bucketAssigner)
.withBucketCheckInterval(1000 * 60*5)
//.withOutputFileConfig(config)
.build();

2022-03-14 11:37:40,867 WARN [com.ververica.cdc.debezium.DebeziumSourceFunction] - Consumer subtask 0 received confirmation for unknown checkpoint id 18
2022-03-14 11:37:43,869 WARN [com.ververica.cdc.debezium.DebeziumSourceFunction] - Consumer subtask 0 received confirmation for unknown checkpoint id 19
2022-03-14 11:37:46,870 WARN [com.ververica.cdc.debezium.DebeziumSourceFunction] - Consumer subtask 0 received confirmation for unknown checkpoint id 20
2022-03-14 11:37:49,867 WARN [com.ververica.cdc.debezium.DebeziumSourceFunction] - Consumer subtask 0 received confirmation for unknown checkpoint id 21
2022-03-14 11:37:52,868 WARN [com.ververica.cdc.debezium.DebeziumSourceFunction] - Consumer subtask 0 received confirmation for unknown checkpoint id 22
2022-03-14 11:37:55,885 WARN [com.ververica.cdc.debezium.DebeziumSourceFunction] - Consumer subtask 0 received confirmation for unknown checkpoint id 23
2022-03-14 11:37:59,519 WARN [com.ververica.cdc.debezium.DebeziumSourceFunction] - Consumer subtask 0 received confirmation for unknown checkpoint id 24
2022-03-14 11:38:01,867 WARN [com.ververica.cdc.debezium.DebeziumSourceFunction] - Consumer subtask 0 received confirmation for unknown checkpoint id 25
2022-03-14 11:38:04,917 WARN [com.ververica.cdc.debezium.DebeziumSourceFunction] - Consumer subtask 0 received confirmation for unknown checkpoint id 26
2022-03-14 11:38:07,872 WARN [com.ververica.cdc.debezium.DebeziumSourceFunction] - Consumer subtask 0 received confirmation for unknown checkpoint id 27
2022-03-14 11:38:10,857 WARN [com.ververica.cdc.debezium.DebeziumSourceFunction] - Consumer subtask 0 received confirmation for unknown checkpoint id 28
2022-03-14 11:38:13,877 WARN [com.ververica.cdc.debezium.DebeziumSourceFunction] - Consumer subtask 0 received confirmation for unknown checkpoint id 29
2022-03-14 11:38:16,858 WARN [com.ververica.cdc.debezium.DebeziumSourceFunction] - Consumer subtask 0 received confirmation for unknown checkpoint id 30
2022-03-14 11:38:19,873 WARN [com.ververica.cdc.debezium.DebeziumSourceFunction] - Consumer subtask 0 received confirmation for unknown checkpoint id 31
2022-03-14 11:38:22,859 WARN [com.ververica.cdc.debezium.DebeziumSourceFunction] - Consumer subtask 0 received confirmation for unknown checkpoint id 32
2022-03-14 11:38:25,864 WARN [com.ververica.cdc.debezium.DebeziumSourceFunction] - Consumer subtask 0 received confirmation for unknown checkpoint id 33
2022-03-14 11:38:28,861 WARN [com.ververica.cdc.debezium.DebeziumSourceFunction] - Consumer subtask 0 received confirmation for unknown checkpoint id 34
2022-03-14 11:38:31,876 WARN [com.ververica.cdc.debezium.DebeziumSourceFunction] - Consumer subtask 0 received confirmation for unknown checkpoint id 35
2022-03-14 11:38:34,860 WARN [com.ververica.cdc.debezium.DebeziumSourceFunction] - Consumer subtask 0 received confirmation for unknown checkpoint id 36
2022-03-14 11:38:37,879 WARN [com.ververica.cdc.debezium.DebeziumSourceFunction] - Consumer subtask 0 received confirmation for unknown checkpoint id 37
2022-03-14 11:38:40,917 WARN [com.ververica.cdc.debezium.DebeziumSourceFunction] - Consumer subtask 0 received confirmation for unknown checkpoint id 38
2022-03-14 11:38:43,866 WARN [com.ververica.cdc.debezium.DebeziumSourceFunction] - Consumer subtask 0 received confirmation for unknown checkpoint id 39
2022-03-14 11:38:46,879 WARN [com.ververica.cdc.debezium.DebeziumSourceFunction] - Consumer subtask 0 received confirmation for unknown checkpoint id 40
2022-03-14 11:38:49,863 WARN [com.ververica.cdc.debezium.DebeziumSourceFunction] - Consumer subtask 0 received confirmation for unknown checkpoint id 41
2022-03-14 11:38:52,861 WARN [com.ververica.cdc.debezium.DebeziumSourceFunction] - Consumer subtask 0 received confirmation for unknown checkpoint id 42
2022-03-14 11:38:55,865 WARN [com.ververica.cdc.debezium.DebeziumSourceFunction] - Consumer subtask 0 received confirmation for unknown checkpoint id 43
2022-03-14 11:38:58,873 WARN [com.ververica.cdc.debezium.DebeziumSourceFunction] - Consumer subtask 0 received confirmation for unknown checkpoint id 44
2022-03-14 11:39:01,863 WARN [com.ververica.cdc.debezium.DebeziumSourceFunction] - Consumer subtask 0 received confirmation for unknown checkpoint id 45
2022-03-14 11:39:04,864 WARN [com.ververica.cdc.debezium.DebeziumSourceFunction] - Consumer subtask 0 received confirmation for unknown checkpoint id 46

重新启动flink 任务之后,同步任务文件难以完成合并
.part-0-0.inprogress.8f399cdb-eb54-487a-bc5e-75cdb2a5375e
.part-0-0.inprogress.b7a4d10e-2930-4e8c-9700-283b43d2586b

请问,这个状态同步,怎么不保存? 任务重启之后,合并为什么出现这样状态