Add a function to remove duplicates.

[korenevskiy]

Add a function to remove duplicates at the time of folder comparison.
I'm comparing 2 duplicate folders, with different archive times.
In order not to lose the unique files, I have to copy the unique files left and right. To then be sure that the folders are the same and then I will delete one. But such copying is a long process.
.
Please add the function delete duplicates in one of the parties.
I compared 2 folders with a capacity of 200 TB and 500 TB.
I had to copy the remaining 300 TB to the first folder to delete it.
Please add the removal of duplicates, so that after the comparison, one folder has only unique files.