Download it here:
DupeCRCDetect (23-Dic-2005), 296kb


DupeCRCDetect is a little utility which scans folders and shows the duplicate files it finds. I wrote it for myself, to help me organize and clean large collections of files, like images, lyrics, etc.

DupeCRCDetect works by calculating the CRC32 of the possible duplicates. A CRC32 is some kind of a checksum which is not designed to, but can be used to identify file contents. So if two files have the same CRC32, then there's a great chance that they have the same contents.

DupeCRCDetect has many functionalities designed to help managing large amounts of duplicates in different folders. Some of its functions have little use and others are for every day. Little tricks are hiding around the corner, so you only have to start using the program to learn a few ways of making your duplicated-life easy.

Use DupeCRCDetect with care, anyway. It allows to delete files from your hard disk, and any tool which allows you to do these things is inherently dangerous.

Just a final note: when working with large amounts of files, DupeCRCDetect can be quite slow. Your mileage will vary depending on your computer. Also note that DupeCRCDetect doesn't calculate the CRC of all the files; just files which can have duplicates. It's magic.

Explanation & Usage

DupeCRCDetect keeps an internal list of files. You can decide which files are in it, by adding or removing entire folders. Every button and list in the GUI has most of these hints.

Button 'Add...' adds a folder, and all of its files, to the internal list. Manually adding folders is kind of ugly (the control is awful, but does it the way I want), you're encouraged to use drag & drop instead. If you're adding a folder which is the parent of any subfolder already added, the child subfolders are deleted from the list (but the files are kept).

Button 'Remove' deletes an already added folder from the list. You have to select a folder to be able to delete it. All the files from it are removed from the internal list.

Button 'Exclude...' excludes from the internal list any files which are in the folder which you select. It doesn't remove the folder from the list, just the files. This allows good flexibility with the 'Reload' button.

Button 'Reload' cleans the internal file list and reloads all folders in the list, again. So any excluded files are reintroduced. Or after a run of 'Find Dupes', you can start from zero again.

Button 'Find Dupes' removes from the internal file list all the unique files, and presents the list of duplicated files, along with their sizes and CRC's.

Button 'Delete!' actually deletes the selected files from your hard disk. Use with care. It prompts for confirmation. After file deletion, the duplicates are refreshed again. If a file selected to be deleted is still not being deleted, it's probably because the file is somehow locked or it's read-only. For security reasons, I decided to not force the deletion of read-only files. Change the files attributes if you want to delete them from DupeCRCDetect.

Button 'Remove' removes selected filenames from the list, and duplicates are refreshed. It doesn't delete the files. Useful if you want to discard some of the duplicates, without actually deleting the files.

Button 'Select All' just selects every file.

Button 'Select Odd' selects one from each two files. Note that given any list of duplicates sorted by CRC, there are always at least two files for each given CRC. Thus, selecting one from each two, and deleting them, leaves at least one file of each given CRC. Note: Using 'Select Odd' changes the sorting to CRC.

Button 'Safe Sel' allows safe selection. It ensures that for each CRC, there's at least one file not selected. So if you select everything, and then use this button, you can safely delete the selected files because you'll still keep at least one file of each CRC. Note: Using 'Select Odd' changes the sorting to CRC.

Button 'Invert' inverts the current selection. Easy and not really useful, except in very specific circumstances. If you ever get to use it, then you'll know what specific circumstances they are ^_^

Note that the file list shown can be sorted by Name, Size or CRC just by clicking the correspondent column header. You can select/deselect more than one file pressing Control or Shift, just like in Windows Explorer. You can also double-click the column header separators to adjust the width of the column to that of the longest item in that column.

If you happen to have a shortcut to DupeCRCDetect in the desktop, like I have, then you can drag any folder(s) over the icon and DupeCRCDetect will add them to the internal list on load. Just send as many folder names in the command line as you want.

Finally, DupeCRCDetect supports drag & drop of folders from Windows Explorer or the desktop during run-time. Just drop the folders over the folders list box and they will be added. These two extra methods of adding folders are the preferred ones, and they follow the rules of the manual 'Add...' button above.

Current Issues

When you close the application while it's calculating the CRC of a file, you have to wait until the file finishes. This can be a bummer for large files.

Since excluding folders is just a temporal change of the internal file list, there's no management of excluded folders planned. Excluded folders aren't kept anywhere and when you reload folders, all files are readded again, without exclusions. This is the intended behaviour.

I don't plan to add any 'Cancel' button, really.

Sometimes the application crashes when closing it. See if you can reproduce the crash in a consistent way and report it to me so I can fix it.


DISCLAIMER: This program is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. In no event the author shall be liable for damages, including any direct, indirect, special, incidental, or consequential damages of any character arising as a result of the use or inability to use this program (including but not limited to damages for loss of goodwill, loss of data, computer failure or malfunction, depression, or any and all other commercial damages or losses).

Simple license: DupeCRCDetect is free for personal use, free to distribute, but it can't be sold or used in commercial environments. Feel free to consider donating a small amount of money to me via PayPal if you find DupeCRCDetect useful, specially if you're a company or a rich individual ;) However, it's not a requirement, and it will never be.

DupeCRCDetect is written in Object Pascal and built with Delphi 7. It's available at www.borland.com, either in Trial mode, or in Personal Edition which is completely free to keep and use forever. Check out also projects like Free Pascal and Lazarus if you're interested in Object Pascal free development.

DupeCRCDetect also uses a few functions and components from the Jedi Code Library and the Jedi Visual Component Library, best known as JCL and JVCL. These libraries are really great if you want to code cool apps in any Delphi version. They're available at www.delphi-jedi.org. Thanks to the JEDI community for these wonderful libraries.

I'm not planning to release the source, but you can ask for it politely. I don't plan to add anything else, and although the app is quite unoptimized, it does its job well, and I don't tend to aaronize the code after a second rewrite. However if anyone can come up with a good bug or idea, I'll think about implementing it, or fixing. You can contact me at pi 'at' logiqx 'dot' com.

Thanks to Bitxo & Attradis for betatesting this little utility so I can release it to the public.


23-12-2005: DupeCRCDetect 0.5.2
   * Cleaned greatly the source and removed some unneeded operations.
   * Solved a rare bug where reloading folders under certain conditions would
      still re-add already calculated files and show more files than wanted.

20-12-2005: DupeCRCDetect 0.5.1
   + First public release.
   * Changed the way internal buffers for CRC's are handled, and further
      cleaned the CRC functions.
   * Removed the ability to change the name of files in the list.

19-12-2005: DupeCRCDetect 0.5
   + Added a warning for unsafe operations related to showing the internal
      list of files in unappropiate occasions.
   * Solved a serious slowdown which ocurred when adding/reloading folders
      after having calculated CRC's.
   * Solved showing the internal list of files without finding duplicates
   * Solved various odd crashes when closing the application in certain states.
      This forced me to add internal support for cancelling operations,
      although I don't want to have a 'Cancel' button.

18-12-2005: DupeCRCDetect 0.4
   + Added multiple folder selection for removal.
   + Added 'Select All' button.
   + Added progress report for file list management and most operations.
   + Beeps when finished, if the operation takes more than 5 seconds.
   * Solved application stall during most long operations.
   * Changed the dynamic buffer for a static buffer, so CRC calculation should
      be slightly faster.
   * Solved a bug where certain buttons were still disabled when selecting
   * Solved a few more minimal bugs and problems.

17-12-2005: DupeCRCDetect 0.3
   + Added a nice icon.
   + Added selfish, egocentric, narcisist, flashy and HUGE author notice.
   + Added 'Safe Selection' functionality.
   * Minor cosmetic changes, small fixes, minimal speedups.
   * Solved a bug where 'Select Even' wouldn't sort the list by CRC.
   * Solved bugs for certain functions when they weren't sorted by CRC.
   * Solved a few minor bugs about when are buttons enabled and disabled.
16-12-2005: DupeCRCDetect 0.2
   + Added Select Odd and Invert selection facilities.
   + Added support for command-line folder addition.
   + Added support for drag & drop folders from Windows Explorer and alikes.
   + Added hints for all the controls.
   * Changed name from DupeDetect to DupeCRCDetect.
   * Changed internal file class to use less memory (about 50% less).
15-12-2005: DupeDetect 0.1
   * Completely rewritten from scratch, with more stuff like progress report.
   + Added Remove, Exclude and Reload folders functionality
   + Added Remove files (opposed to Delete).
   * Solved two bugs where a file is still shown when there aren't any

November 2001: DupeDetect 0.0
   + Basic functionality, adds folders, detects duplicates, allows to delete
      files. And nothing else. Been using it for 4 years ^_^