Duplicate data detection

[d4rken]

Should a feature similar to SD Maid 1's "Duplicates" feature should be re-added?

According to stats I had before I removed analytics, Duplicates was not a very popular feature.

It's still a good way to gain additional storage space, but has some issues:

• There is no one "right" way to select duplicates which makes it more difficult to use.
• Storage is super convoluted in Android and one of the toughest points was making sure that SD Maid didn't scan the same storage twice, just via different paths. This has NOT gotten easier with newer Android versions.
• A lot of users expect a feature like this to detect "similar" images, e.g. a high res photo in your DCIM folder and a low res variant generated by a messenger. This is vastly more complex than comparing checksums to find exact matches.

[BROKEN81]

It would be nice, however, if it's not a popular feature, then no use to create more work for yourself.
I think it comes down to most people like familiarity.

[kanishk-upadhyay]

Should a feature similar to SD Maid 1's "Duplicates" feature should be re-added?

According to stats I had before I removed analytics, Duplicates was not a very popular feature.

It's still a good way to gain additional storage space, but has some issues:

• There is no one "right" way to select duplicates which makes it more difficult to use.
• Storage is super convoluted in Android and one of the toughest points was making sure that SD Maid didn't scan the same storage twice, just via different paths. This has NOT gotten easier with newer Android versions.
• A lot of users expect a feature like this to detect "similar" images, e.g. a high res photo in your DCIM folder and a low res variant generated by a messenger. This is vastly more complex than comparing checksums to find exact matches.

yessir i regularly use the duplicates feature available on sd maid 1 as i find it better than other apps
even giving it the same functionality as the previous version would be fine though

[tathastu871]

I use termux app with fdupes -Nrd option to remove duplicates.
Maybe fdupes can be compiled into sdmaid

[adRn-s]

Thanks! I will try this.

[Wjxfi]

I really need this!

[tathastu871]

@d4rken If possible try adding fdupes utility for removing duplicates. Simple fdupes -Nrd will work without user intervrmyion.
Howvert it must be users responsivility to use duplicte removal as there is no guarantee it will not remove some important fille

[adRn-s]

barely related to the topic, termux also has a package, go-findimagedupes, to find similar images. The process/ algorithm for it is described here: https://github.com/jhnc/findimagedupes#description Again, it's prone to false positives. So, it might not be a good fit for SD Maid.

[telrod11]

As I upgraded this morning, this is the first thing that I noticed missing in the new build.

Yes, please bring it back

[d4rken]

I've started working on this