我正在尝试使用milvus2.2.6独立版本进行导入向量，但是目前出现了一些问题

[zhenzi0322]

[2023/04/19 02:01:18.810 +00:00] [DEBUG] [querynode/flow_graph_insert_node.go:412] ["Do insert done"] [collectionID=440891468920129788] [segmentID=440891468920339016] [len=498]
[2023/04/19 02:01:18.810 +00:00] [DEBUG] [querynode/flow_graph_service_time_node.go:76] ["update tSafe:"] [collectionID=440891468920129788] [tSafe=440892036356767745] [tSafe_p=2023/04/19 02:00:45.526 +00:00] [tsLag=33.284817902s] [channel=by-dev-rootcoord-dml_1_440891468920129788v1]
[2023/04/19 02:01:19.180 +00:00] [WARN] [retry/retry.go:44] ["retry func failed"] ["retry time"=0] [error="role rootcoord[nodeID: 0] is not serving, reason: sate code: Abnormal"]
[2023/04/19 02:01:19.247 +00:00] [WARN] [retry/retry.go:44] ["retry func failed"] ["retry time"=0] [error="role rootcoord[nodeID: 0] is not serving, reason: sate code: Abnormal"]
[2023/04/19 02:01:19.977 +00:00] [WARN] [datacoord/services.go:896] ["DataCoord.GetMetrics failed"] [traceID=76774bf99f1002f0] [nodeID=3] [req="{\"metric_type\":\"system_info\"}"] [error="DataCoord 3 is not ready"]
[2023/04/19 02:01:19.977 +00:00] [WARN] [retry/retry.go:44] ["retry func failed"] ["retry time"=0] [error="role datacoord[nodeID: 3] is not serving, reason: sate code: Abnormal"]
[2023/04/19 02:01:19.980 +00:00] [DEBUG] [proxy/impl.go:3956] [Proxy.GetProxyMetrics] [traceID=691da5c24bb8874a] [nodeID=8] [req="{\"metric_type\":\"system_info\"}"] [metricType=system_info]
[2023/04/19 02:01:20.028 +00:00] [WARN] [datacoord/services.go:896] ["DataCoord.GetMetrics failed"] [traceID=2eee42e476c7b16e] [nodeID=3] [req="{\"metric_type\":\"system_info\"}"] [error="DataCoord 3 is not ready"]
[2023/04/19 02:01:20.129 +00:00] [WARN] [datacoord/services.go:896] ["DataCoord.GetMetrics failed"] [traceID=33f7f3d246725d19] [nodeID=3] [req="{\"metric_type\":\"system_info\"}"] [error="DataCoord 3 is not ready"]
[2023/04/19 02:01:20.330 +00:00] [WARN] [datacoord/services.go:896] ["DataCoord.GetMetrics failed"] [traceID=24487b852eab2fde] [nodeID=3] [req="{\"metric_type\":\"system_info\"}"] [error="DataCoord 3 is not ready"]
[2023/04/19 02:01:20.731 +00:00] [WARN] [datacoord/services.go:896] ["DataCoord.GetMetrics failed"] [traceID=788439b0b6734d40] [nodeID=3] [req="{\"metric_type\":\"system_info\"}"] [error="DataCoord 3 is not ready"]
[2023/04/19 02:01:21.533 +00:00] [WARN] [datacoord/services.go:896] ["DataCoord.GetMetrics failed"] [traceID=55752e85bf38555c] [nodeID=3] [req="{\"metric_type\":\"system_info\"}"] [error="DataCoord 3 is not ready"]
[2023/04/19 02:01:22.534 +00:00] [WARN] [datacoord/services.go:896] ["DataCoord.GetMetrics failed"] [traceID=4d06aafdc4cc2bcc] [nodeID=3] [req="{\"metric_type\":\"system_info\"}"] [error="DataCoord 3 is not ready"]
[2023/04/19 02:01:23.536 +00:00] [WARN] [datacoord/services.go:896] ["DataCoord.GetMetrics failed"] [traceID=7cf45abe46d2be43] [nodeID=3] [req="{\"metric_type\":\"system_info\"}"] [error="DataCoord 3 is not ready"]
[2023/04/19 02:01:23.778 +00:00] [WARN] [datacoord/compaction.go:142] ["unable to alloc timestamp"] [error="role rootcoord[nodeID: 0] is not serving, reason: sate code: Abnormal"]
[2023/04/19 02:01:23.778 +00:00] [WARN] [retry/retry.go:44] ["retry func failed"] ["retry time"=0] [error="role rootcoord[nodeID: 0] is not serving, reason: sate code: Abnormal"]
[2023/04/19 02:01:24.537 +00:00] [WARN] [datacoord/services.go:896] ["DataCoord.GetMetrics failed"] [traceID=5fddb3a681007f3a] [nodeID=3] [req="{\"metric_type\":\"system_info\"}"] [error="DataCoord 3 is not ready"]
[2023/04/19 02:01:25.345 +00:00] [INFO] [proxy/proxy.go:411] ["close scheduler"] [role=proxy]
[2023/04/19 02:01:25.345 +00:00] [INFO] [proxy/proxy.go:419] ["close channels time ticker"] [role=proxy]
[2023/04/19 02:01:25.345 +00:00] [WARN] [proxy/impl.go:641] ["DescribeCollection failed to enqueue"] [error="syncTimeStamp Failed:role rootcoord[nodeID: 0] is not serving, reason: sate code: Abnormal"] [traceID=99f2874bb62159d] [role=proxy] [db=] [collection=sucai999]
[2023/04/19 02:01:25.345 +00:00] [INFO] [proxy/impl.go:689] ["DescribeCollection done"] [traceID=591db0a7ff1dbf81] [role=proxy] [MsgID=440892037266931726] [BeginTS=440892037266931726] [EndTS=440892037266931726] [db=] [collection=sucai999]
[2023/04/19 02:01:25.346 +00:00] [WARN] [retry/retry.go:44] ["retry func failed"] ["retry time"=0] [error="role rootcoord[nodeID: 0] is not serving, reason: sate code: Abnormal"]
[2023/04/19 02:01:25.346 +00:00] [INFO] [msgstream/mq_msgstream.go:182] ["start to close mq msg stream"] ["producer num"=2] ["consumer num"=0]
[2023/04/19 02:01:25.346 +00:00] [INFO] [proxy/channels_mgr.go:303] ["all dml stream removed"]
[2023/04/19 02:01:25.539 +00:00] [WARN] [datacoord/services.go:896] ["DataCoord.GetMetrics failed"] [traceID=1349086299aad8ec] [nodeID=3] [req="{\"metric_type\":\"system_info\"}"] [error="DataCoord 3 is not ready"]
[2023/04/19 02:01:25.738 +00:00] [WARN] [retry/retry.go:44] ["retry func failed"] ["retry time"=0] [error="role rootcoord[nodeID: 0] is not serving, reason: sate code: Abnormal"]
[2023/04/19 02:01:25.738 +00:00] [ERROR] [datanode/flow_graph_insert_buffer_node.go:503] ["insertBufferNode flushBufferData failed, err = All attempts results:\nattempt #1:role rootcoord[nodeID: 0] is not serving, reason: sate code: Abnormal\nattempt #2:context canceled\n"] [stack="github.com/milvus-io/milvus/internal/datanode.(*insertBufferNode).Sync\n\t/go/src/github.com/milvus-io/milvus/internal/datanode/flow_graph_insert_buffer_node.go:503\ngithub.com/milvus-io/milvus/internal/datanode.(*insertBufferNode).Operate\n\t/go/src/github.com/milvus-io/milvus/internal/datanode/flow_graph_insert_buffer_node.go:139\ngithub.com/milvus-io/milvus/internal/util/flowgraph.(*nodeCtx).work\n\t/go/src/github.com/milvus-io/milvus/internal/util/flowgraph/node.go:127"]
panic: insertBufferNode flushBufferData failed, err = All attempts results:
attempt #1:role rootcoord[nodeID: 0] is not serving, reason: sate code: Abnormal
attempt #2:context canceled


goroutine 22726 [running]:
github.com/milvus-io/milvus/internal/datanode.(*insertBufferNode).Sync(0xc00693c680, 0xc00a7c8d20?, {0xc00548eb58?, 0x0?, 0x0?}, 0xc00b7c10e0)
        /go/src/github.com/milvus-io/milvus/internal/datanode/flow_graph_insert_buffer_node.go:504 +0x1285
github.com/milvus-io/milvus/internal/datanode.(*insertBufferNode).Operate(0xc00693c680, {0xc002e8f4c0, 0x1, 0x1})
        /go/src/github.com/milvus-io/milvus/internal/datanode/flow_graph_insert_buffer_node.go:139 +0x485
github.com/milvus-io/milvus/internal/util/flowgraph.(*nodeCtx).work(0xc00690eaf0)
        /go/src/github.com/milvus-io/milvus/internal/util/flowgraph/node.go:127 +0x2ac
created by github.com/milvus-io/milvus/internal/util/flowgraph.(*nodeCtx).Start
        /go/src/github.com/milvus-io/milvus/internal/util/flowgraph/node.go:71 +0x79

[zhenzi0322]

[2023/04/19 02:01:18.810 +00:00] [DEBUG] [querynode/flow_graph_insert_node.go:412] ["Do insert done"] [collectionID=440891468920129788] [segmentID=440891468920339016] [len=498]
[2023/04/19 02:01:18.810 +00:00] [DEBUG] [querynode/flow_graph_service_time_node.go:76] ["update tSafe:"] [collectionID=440891468920129788] [tSafe=440892036356767745] [tSafe_p=2023/04/19 02:00:45.526 +00:00] [tsLag=33.284817902s] [channel=by-dev-rootcoord-dml_1_440891468920129788v1]
[2023/04/19 02:01:19.180 +00:00] [WARN] [retry/retry.go:44] ["retry func failed"] ["retry time"=0] [error="role rootcoord[nodeID: 0] is not serving, reason: sate code: Abnormal"]
[2023/04/19 02:01:19.247 +00:00] [WARN] [retry/retry.go:44] ["retry func failed"] ["retry time"=0] [error="role rootcoord[nodeID: 0] is not serving, reason: sate code: Abnormal"]
[2023/04/19 02:01:19.977 +00:00] [WARN] [datacoord/services.go:896] ["DataCoord.GetMetrics failed"] [traceID=76774bf99f1002f0] [nodeID=3] [req="{\"metric_type\":\"system_info\"}"] [error="DataCoord 3 is not ready"]
[2023/04/19 02:01:19.977 +00:00] [WARN] [retry/retry.go:44] ["retry func failed"] ["retry time"=0] [error="role datacoord[nodeID: 3] is not serving, reason: sate code: Abnormal"]
[2023/04/19 02:01:19.980 +00:00] [DEBUG] [proxy/impl.go:3956] [Proxy.GetProxyMetrics] [traceID=691da5c24bb8874a] [nodeID=8] [req="{\"metric_type\":\"system_info\"}"] [metricType=system_info]
[2023/04/19 02:01:20.028 +00:00] [WARN] [datacoord/services.go:896] ["DataCoord.GetMetrics failed"] [traceID=2eee42e476c7b16e] [nodeID=3] [req="{\"metric_type\":\"system_info\"}"] [error="DataCoord 3 is not ready"]
[2023/04/19 02:01:20.129 +00:00] [WARN] [datacoord/services.go:896] ["DataCoord.GetMetrics failed"] [traceID=33f7f3d246725d19] [nodeID=3] [req="{\"metric_type\":\"system_info\"}"] [error="DataCoord 3 is not ready"]
[2023/04/19 02:01:20.330 +00:00] [WARN] [datacoord/services.go:896] ["DataCoord.GetMetrics failed"] [traceID=24487b852eab2fde] [nodeID=3] [req="{\"metric_type\":\"system_info\"}"] [error="DataCoord 3 is not ready"]
[2023/04/19 02:01:20.731 +00:00] [WARN] [datacoord/services.go:896] ["DataCoord.GetMetrics failed"] [traceID=788439b0b6734d40] [nodeID=3] [req="{\"metric_type\":\"system_info\"}"] [error="DataCoord 3 is not ready"]
[2023/04/19 02:01:21.533 +00:00] [WARN] [datacoord/services.go:896] ["DataCoord.GetMetrics failed"] [traceID=55752e85bf38555c] [nodeID=3] [req="{\"metric_type\":\"system_info\"}"] [error="DataCoord 3 is not ready"]
[2023/04/19 02:01:22.534 +00:00] [WARN] [datacoord/services.go:896] ["DataCoord.GetMetrics failed"] [traceID=4d06aafdc4cc2bcc] [nodeID=3] [req="{\"metric_type\":\"system_info\"}"] [error="DataCoord 3 is not ready"]
[2023/04/19 02:01:23.536 +00:00] [WARN] [datacoord/services.go:896] ["DataCoord.GetMetrics failed"] [traceID=7cf45abe46d2be43] [nodeID=3] [req="{\"metric_type\":\"system_info\"}"] [error="DataCoord 3 is not ready"]
[2023/04/19 02:01:23.778 +00:00] [WARN] [datacoord/compaction.go:142] ["unable to alloc timestamp"] [error="role rootcoord[nodeID: 0] is not serving, reason: sate code: Abnormal"]
[2023/04/19 02:01:23.778 +00:00] [WARN] [retry/retry.go:44] ["retry func failed"] ["retry time"=0] [error="role rootcoord[nodeID: 0] is not serving, reason: sate code: Abnormal"]
[2023/04/19 02:01:24.537 +00:00] [WARN] [datacoord/services.go:896] ["DataCoord.GetMetrics failed"] [traceID=5fddb3a681007f3a] [nodeID=3] [req="{\"metric_type\":\"system_info\"}"] [error="DataCoord 3 is not ready"]
[2023/04/19 02:01:25.345 +00:00] [INFO] [proxy/proxy.go:411] ["close scheduler"] [role=proxy]
[2023/04/19 02:01:25.345 +00:00] [INFO] [proxy/proxy.go:419] ["close channels time ticker"] [role=proxy]
[2023/04/19 02:01:25.345 +00:00] [WARN] [proxy/impl.go:641] ["DescribeCollection failed to enqueue"] [error="syncTimeStamp Failed:role rootcoord[nodeID: 0] is not serving, reason: sate code: Abnormal"] [traceID=99f2874bb62159d] [role=proxy] [db=] [collection=sucai999]
[2023/04/19 02:01:25.345 +00:00] [INFO] [proxy/impl.go:689] ["DescribeCollection done"] [traceID=591db0a7ff1dbf81] [role=proxy] [MsgID=440892037266931726] [BeginTS=440892037266931726] [EndTS=440892037266931726] [db=] [collection=sucai999]
[2023/04/19 02:01:25.346 +00:00] [WARN] [retry/retry.go:44] ["retry func failed"] ["retry time"=0] [error="role rootcoord[nodeID: 0] is not serving, reason: sate code: Abnormal"]
[2023/04/19 02:01:25.346 +00:00] [INFO] [msgstream/mq_msgstream.go:182] ["start to close mq msg stream"] ["producer num"=2] ["consumer num"=0]
[2023/04/19 02:01:25.346 +00:00] [INFO] [proxy/channels_mgr.go:303] ["all dml stream removed"]
[2023/04/19 02:01:25.539 +00:00] [WARN] [datacoord/services.go:896] ["DataCoord.GetMetrics failed"] [traceID=1349086299aad8ec] [nodeID=3] [req="{\"metric_type\":\"system_info\"}"] [error="DataCoord 3 is not ready"]
[2023/04/19 02:01:25.738 +00:00] [WARN] [retry/retry.go:44] ["retry func failed"] ["retry time"=0] [error="role rootcoord[nodeID: 0] is not serving, reason: sate code: Abnormal"]
[2023/04/19 02:01:25.738 +00:00] [ERROR] [datanode/flow_graph_insert_buffer_node.go:503] ["insertBufferNode flushBufferData failed, err = All attempts results:\nattempt #1:role rootcoord[nodeID: 0] is not serving, reason: sate code: Abnormal\nattempt #2:context canceled\n"] [stack="github.com/milvus-io/milvus/internal/datanode.(*insertBufferNode).Sync\n\t/go/src/github.com/milvus-io/milvus/internal/datanode/flow_graph_insert_buffer_node.go:503\ngithub.com/milvus-io/milvus/internal/datanode.(*insertBufferNode).Operate\n\t/go/src/github.com/milvus-io/milvus/internal/datanode/flow_graph_insert_buffer_node.go:139\ngithub.com/milvus-io/milvus/internal/util/flowgraph.(*nodeCtx).work\n\t/go/src/github.com/milvus-io/milvus/internal/util/flowgraph/node.go:127"]
panic: insertBufferNode flushBufferData failed, err = All attempts results:
attempt #1:role rootcoord[nodeID: 0] is not serving, reason: sate code: Abnormal
attempt #2:context canceled


goroutine 22726 [running]:
github.com/milvus-io/milvus/internal/datanode.(*insertBufferNode).Sync(0xc00693c680, 0xc00a7c8d20?, {0xc00548eb58?, 0x0?, 0x0?}, 0xc00b7c10e0)
        /go/src/github.com/milvus-io/milvus/internal/datanode/flow_graph_insert_buffer_node.go:504 +0x1285
github.com/milvus-io/milvus/internal/datanode.(*insertBufferNode).Operate(0xc00693c680, {0xc002e8f4c0, 0x1, 0x1})
        /go/src/github.com/milvus-io/milvus/internal/datanode/flow_graph_insert_buffer_node.go:139 +0x485
github.com/milvus-io/milvus/internal/util/flowgraph.(*nodeCtx).work(0xc00690eaf0)
        /go/src/github.com/milvus-io/milvus/internal/util/flowgraph/node.go:127 +0x2ac
created by github.com/milvus-io/milvus/internal/util/flowgraph.(*nodeCtx).Start
        /go/src/github.com/milvus-io/milvus/internal/util/flowgraph/node.go:71 +0x79

导入向量中途就会出现错误，不是一下子就退出程序，而是在导入了一些数据才会出现。

[zhenzi0322]

[xiaofan-luan]

rootcoord is not serving.
did you check your etcd works correctly? it's all the same issues bro.

[zhenzi0322]

你另外问的那个问题“syncTimestamp Failed:context deadline exceeded” 可能也跟你调用 insert/load的方式有关系

好的

[zhenzi0322]

另外说一下，我改用成了2.2.0的版本。

[zhenzi0322]

collection.load() 调用一次就好，无需多次调用。你可以 insert -> insert -> insert() ... -> load() 或者 load() -> insert() -> insert() -> insert() ...

collection.load(_async=True)的问题我会尝试重现一下，你之前的做法是这样么？ insert() -> load(_async=True) -> insert() -> load(_async=True)

这种做法在milvus版本2.2.0和2.2.6也会出现。

[zhenzi0322]

还有一个问题的就是：我之前采用milvus2.2.0版本旧的数据有没有办法可以快速导入到新的版本2.2.6中。而不用我重新导入。这样可以加快我的导入到milvus的速度。

[yhmo]

还有一个问题的就是：我之前采用milvus2.2.0版本旧的数据有没有办法可以快速导入到新的版本2.2.6中。而不用我重新导入。这样可以加快我的导入到milvus的速度。

目前只能用导入导出工具来做 https://github.com/zilliztech/milvus-backup

[zhenzi0322]

我重启一次docker milvus就会加载一次，有没有什么办法可以快速地加载完毕。

目前是一直在加载中，我也不知道什么时候才能加载完毕。目前就是一直在等待它的完成才能开始下一步的操作。

[yhmo]

如果一个collection在重启之前是loaded状态，那么重启时，就会自动去加载这个collection。从存储系统读数据的带宽就直接决定了加载时的时长。

[yhmo]

还有一个影响因素：如果collection里有许多partition，比如几百个partition，那么就会耗时很久