-
Notifications
You must be signed in to change notification settings - Fork 159
Open
Description
i`m facing an issue with seed command, orchestrator reach to 99% and then stops the process.
On the 99% data copy it returns
"10 iterations have passed without progress. Bailing out."
I looked over the seed logic.
And seems like this counter going to 10.
if targetAgentPoll.MySQLDiskUsage == bytesCopied {
numStaleIterations++
}
Which means the node that sending the data is even to the node that receiving the data.
But, this condition returns false for some reason:
if _, commandCompleted, _ := seedCommandCompleted(targetHostname, seedId);
What exactly seedCommandCompleted(targetHostname, seedId);
checks?
The whole piece of code
for !copyComplete {
targetAgentPoll, err := GetAgent(targetHostname)
if err != nil {
return log.Errore(err)
}
if targetAgentPoll.MySQLDiskUsage == bytesCopied {
numStaleIterations++
}
bytesCopied = targetAgentPoll.MySQLDiskUsage
copyFailed := false
if _, commandCompleted, _ := seedCommandCompleted(targetHostname, seedId); commandCompleted {
copyComplete = true
if _, commandSucceeded, _ := seedCommandSucceeded(targetHostname, seedId); !commandSucceeded {
// failed.
copyFailed = true
}
}
if numStaleIterations > 10 {
copyFailed = true
}
if copyFailed {
AbortSeedCommand(sourceHostname, seedId)
AbortSeedCommand(targetHostname, seedId)
Unmount(sourceHostname)
return updateSeedStateEntry(seedStateId, errors.New("10 iterations have passed without progress. Bailing out."))
}
var copyPct int64 = 0
if sourceAgent.MountPoint.MySQLDiskUsage > 0 {
copyPct = 100 * bytesCopied / sourceAgent.MountPoint.MySQLDiskUsage
}
seedStateId, _ = submitSeedStateEntry(seedId, fmt.Sprintf("Copied %d/%d bytes (%d%%)", bytesCopied, sourceAgent.MountPoint.MySQLDiskUsage, copyPct), "")
if !copyComplete {
time.Sleep(30 * time.Second)
}
}
Metadata
Metadata
Assignees
Labels
No labels