Skip to content

A high-performance CLI tool written in Rust to scan, detect, preview, and clean up duplicate files using content-based hashing (BLAKE3). Works across macOS, Linux, and Windows.

Notifications You must be signed in to change notification settings

c0d3cr4ft3r/duplicate-finder

Folders and files

NameName
Last commit message
Last commit date

Latest commit

Β 

History

1 Commit
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 

Repository files navigation

πŸ” Duplicate Finder

A high-performance CLI tool written in Rust to scan, detect, preview, and clean up duplicate files using content-based hashing (BLAKE3). Works seamlessly across macOS, Linux, and Windows.


βš™οΈ Features

  • βœ… Multi-threaded scanning using Rayon
  • βœ… File grouping by size and content hash (BLAKE3)
  • βœ… Smart deletion policies:
    • Keep newest / oldest by modification time
    • Keep file with the shortest path
    • Keep file matching a regex pattern
  • βœ… Interactive file deletion mode
  • βœ… Preview files using system viewer

πŸš€ Usage

cargo run --release -- <path> [OPTIONS]

Required Argument

  • <path> β€” Directory to scan for duplicates

Options

Flag Description
--open Preview one file per duplicate set
--interactive Interactive mode to manually keep/delete
--keep-newest Auto-keep the newest file in each group
--keep-oldest Auto-keep the oldest file in each group
--keep-shortest Auto-keep the shortest path file
--keep-regex <PAT> Auto-keep file matching a regex pattern

Only one --keep-* policy can be used at a time.


πŸ–₯️ Examples

# Basic scan
duplicate_finder ~/Downloads

# Smart deletion by newest modified file
duplicate_finder ~/Downloads --keep-newest

# Interactive cleanup
duplicate_finder ~/Downloads --interactive

# Keep files with 'important' in the name
duplicate_finder ~/Downloads --keep-regex "important"

πŸ§ͺ Installation

git clone https://github.com/c0d3cr4ft3r/duplicate-finder.git
cd duplicate-finder
cargo build --release
./target/release/duplicate_finder --help

🧠 How it works

  1. Scans all files and groups them by size
  2. Computes BLAKE3 hashes in parallel
  3. Groups files with identical hashes
  4. Runs the chosen cleanup strategy

🧰 Dependencies

  • walkdir
  • rayon
  • blake3
  • tabled
  • clap
  • dialoguer
  • regex

πŸ“ License

MIT β€” do whatever you want, just don't blame me.

About

A high-performance CLI tool written in Rust to scan, detect, preview, and clean up duplicate files using content-based hashing (BLAKE3). Works across macOS, Linux, and Windows.

Topics

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages