Duplicate Clean

A simple tool to delete duplicate files.

Table of Content

• Compatible Platform
• Usage
• Example

Compatible Platform

• macOS
• Linux

Usage

Example

Manually creating duplicate files

1. Creating a directory named 'test' under '/tmp'

mkdir /tmp/test

2. Coping dupclean source files to directory /tmp/test

cp -r /path/to/dupclean/ /tmp/test/

3. Creating duplicate files

for f in `ls -1 /tmp/test/modules/`; do cp /tmp/test/modules/{"$f","$f".bak}; done

Downloading dupclean bin file and coping to /tmp

cp /path/to/dupclean /tmp

Listing and deleting duplicate files with dupclean

1. Listing files

$ /tmp/dupclean -r -f /tmp/test
---------------------------------------- Duplicate Results ----------------------------------------
checksum:  6e5e4d9714608714ce952f5c42555b3d
         hold  1692413845 '/tmp/test/modules/__init__.py'
         hold  1692414154 '/tmp/test/modules/__init__.py.bak'
checksum:  d55ecb1291a11d396eccfaadfb9133f5
         hold  1692413845 '/tmp/test/modules/filter.py'
         hold  1692414154 '/tmp/test/modules/filter.py.bak'
checksum:  ed0178bb65e6747a77017ff3fccd26af
         hold  1692413845 '/tmp/test/modules/datatype.py'
         hold  1692414154 '/tmp/test/modules/datatype.py.bak'
checksum:  9a20e19c2f93f5e689408d0994ad4b2e
         hold  1692413845 '/tmp/test/modules/listduplicate.py'
         hold  1692414154 '/tmp/test/modules/listduplicate.py.bak'
---------------------------------------- Duplicate Results ----------------------------------------
Need taking action, followed with "--run" parameter.
You can also move the files to directory followed by "-b" or "--backup" instead of deleting directly.

2. Adding keyword and suffix to mark files as burn or hold

$ /tmp/dupclean -r -f /tmp/test --suffix-to-burn '.bak' --key-to-burn 'init' 
---------------------------------------- Duplicate Results ----------------------------------------
checksum:  6e5e4d9714608714ce952f5c42555b3d
         burn  1692413845 '/tmp/test/modules/__init__.py'
         burn  1692414154 '/tmp/test/modules/__init__.py.bak'
checksum:  d55ecb1291a11d396eccfaadfb9133f5
         hold  1692413845 '/tmp/test/modules/filter.py'
         burn  1692414154 '/tmp/test/modules/filter.py.bak'
checksum:  ed0178bb65e6747a77017ff3fccd26af
         hold  1692413845 '/tmp/test/modules/datatype.py'
         burn  1692414154 '/tmp/test/modules/datatype.py.bak'
checksum:  9a20e19c2f93f5e689408d0994ad4b2e
         hold  1692413845 '/tmp/test/modules/listduplicate.py'
         burn  1692414154 '/tmp/test/modules/listduplicate.py.bak'
---------------------------------------- Duplicate Results ----------------------------------------
Need taking action, followed with "--run" parameter.
You can also move the files to directory followed by "-b" or "--backup" instead of deleting directly.

3. Setting -c 1 option to hold at least on copy of each file

Both of origin and copy of __init__ are marked as burn from last output.

After adding -c 1 option:

$ /tmp/dupclean -r -f /tmp/test --suffix-to-burn '.bak' --key-to-burn 'init' -c 1
---------------------------------------- Duplicate Results ----------------------------------------
checksum:  6e5e4d9714608714ce952f5c42555b3d
         burn  1692413845 '/tmp/test/modules/__init__.py'
         hold  1692414154 '/tmp/test/modules/__init__.py.bak'
checksum:  d55ecb1291a11d396eccfaadfb9133f5
         hold  1692413845 '/tmp/test/modules/filter.py'
         burn  1692414154 '/tmp/test/modules/filter.py.bak'
checksum:  ed0178bb65e6747a77017ff3fccd26af
         hold  1692413845 '/tmp/test/modules/datatype.py'
         burn  1692414154 '/tmp/test/modules/datatype.py.bak'
checksum:  9a20e19c2f93f5e689408d0994ad4b2e
         hold  1692413845 '/tmp/test/modules/listduplicate.py'
         burn  1692414154 '/tmp/test/modules/listduplicate.py.bak'
---------------------------------------- Duplicate Results ----------------------------------------
Need taking action, followed with "--run" parameter.
You can also move the files to directory followed by "-b" or "--backup" instead of deleting directly.

4. Deleting files with --run action or just move to backup directory with -b /path/to/backup --run

$ mkdir /tmp/backup
$ /tmp/dupclean -r -f /tmp/test --suffix-to-burn '.bak' --key-to-burn 'init' -c 1 -b /tmp/backup --run

Code URL

GitHub