Removing Duplicate Files

As a follow on to my previous post about the Link Shell Extension software and hard links, I thought I’d say something about finding and removing duplicate files especially now that I’ve found a free program that can turn duplicates into hard links.

I had been using Easy Duplicate Finder version 2.2.4 and was fairly pleased with it.  It finds duplicates based on parameters such as name and file size as well as content (based on a byte to byte comparison or a hash of the whole file) and had a good selection assistance feature.

However, it seems any version of Easy Duplicate Finder above 2.4 is no longer free, so when I started having a few crashes with it I went looking for an alternative free duplicate finder.

My favourite resource for finding new freeware is Gizmo's Freeware.  They have volunteer editors who review freeware in different categories, explain the differences and give their recommendations.  Their Best Free Duplicate File Detector page led me to try Duplicate Cleaner.

I’m quite pleased with Duplicate Cleaner.  In addition to having all the features of Easy Duplicate Finder, it has a better Selection Assistant and a variety of File Removal options that include Delete, Delete to Recycle Bin, Move, Copy, Rename and of course the aforementioned Create Hardlinks.

I use hard links when I’ve been sorting things like pictures to upload somewhere and really just want to keep the selection for reference and don’t need a spare copy of the photos.  I use Link Shell Extension now for making hard linked copies of my pictures without taking up any more disk space but for those selections I’ve made before I can clean them up with Duplicate Cleaner’s Create Hardlinks options.

The Create Hardlinks is a very safe option in that you can’t accidentally delete something with it.  It will only make a hard link to identical file contents and the only thing removed is the second copy of that identical data.  The duplicate file is still there but under the hood it only points to the original copy of the file.  However, if the original is ever deleted then the duplicate becomes the sole and complete copy of the file.

Most people are just going to use Duplicate Cleaner for actually deleting duplicates and I’d use it for that even if it didn’t have the extra hard links feature.  I really like the options to select from a list of duplicates using “Select by Location” where it keeps the files from one specific folder path and “Select by text pattern” where I can Mark or Unmark based on any string of text in the file name.

The text pattern has a Column Name drop down box where you can switch to searching for path, creation date, image size, bit rate and a variety of other fields that you would see if you right clicked the file, clicked Properties and chose the Details tab.

The beautiful thing is that you can mark or unmark to augment your original selection criteria.  It doesn’t wipe your existing selections and go only by the text pattern.

Be aware when running the Duplicate Cleaner installer that there is optional sponsor software (currently Bing Toolbar) that you can uncheck if you don’t want it included.