Overclock.net › Forums › Software, Programming and Coding › Other Software › Best method for finding duplicate files?
New Posts  All Forums:Forum Nav:

Best method for finding duplicate files?

post #1 of 5
5/22/12 at 8:14pm
Thread Starter 
Turns out that my wife's pictures/videos folder is chock full of duplicates. Instead of painstakingly verifying each and every picture (it never hurts to take a trip down memory lane), would it be easier to find duplicates based off CRC or MD5 hashes? I will be manually going through after the software does it's thing, just to make sure.




I've found this freeware; http://www.funduc.com/dupfiles.htm and want a second opinion.
had a fatal error.
(14 items)
  		  
CPUMotherboardGraphicsRAM
E8400 DFI LP DK P45-T2RS Sapphire 4850 512MB 2x2GB G.Skill 1066 (5-5-5-15-2T 1.9V) 
Hard DriveOptical DriveOSMonitor
820GB Total LG Super DVD Burner SATA VII Enterprise x64 Gateway FHD2401 
PowerCaseMouseMouse Pad
Corsair VX550W NZXT Source 210 White Logitech G400 Razer Kabuto 
Audio
Creative Audigy 
View all
  hide details  
Reply
had a fatal error.
(14 items)
  		  
CPUMotherboardGraphicsRAM
E8400 DFI LP DK P45-T2RS Sapphire 4850 512MB 2x2GB G.Skill 1066 (5-5-5-15-2T 1.9V) 
Hard DriveOptical DriveOSMonitor
820GB Total LG Super DVD Burner SATA VII Enterprise x64 Gateway FHD2401 
PowerCaseMouseMouse Pad
Corsair VX550W NZXT Source 210 White Logitech G400 Razer Kabuto 
Audio
Creative Audigy 
View all
  hide details  
Reply
post #2 of 5
5/23/12 at 11:01am
I've used Dupdetector for pictures, but it can choke on giant directories.

The best I've seen for finding all types of duplicates is Tree Size Pro, and finding duplicates is only part of what it does. Dealing with lots of hard drives in docking stations I can't image living without this program.
Nehalem
(13 items)
  		  
CPUMotherboardGraphicsRAM
i7 950 @ 4.3GHz Evga X58 3SLI 2X SLI Evga GTX 275 FTWs 756/1620/1296 6GB Mushkin Blackline 1600MHz 78720591T 
Hard DriveMonitorPower
OCZ Vertex2 60GB Sony FW900 Corsair 850TX 
View all
  hide details  
Reply
Nehalem
(13 items)
  		  
CPUMotherboardGraphicsRAM
i7 950 @ 4.3GHz Evga X58 3SLI 2X SLI Evga GTX 275 FTWs 756/1620/1296 6GB Mushkin Blackline 1600MHz 78720591T 
Hard DriveMonitorPower
OCZ Vertex2 60GB Sony FW900 Corsair 850TX 
View all
  hide details  
Reply
post #3 of 5
5/23/12 at 11:11am
Thread Starter 
Thanks.

The program I linked above worked magic.

It did choke on scanning across multiple drives (I was just messing around at this point), but it did find dupes at a level I was very pleased with.


I deleted close to 2GB worth of dupes from both my wife's pictures and pictures that I've had of my family since 2004.

I rescanned both directories with MD5 and CRC32, and both scans were consistent. I manually verified, and yeah, they were all dupes.

Largely, I'm pleased with this freeware.
had a fatal error.
(14 items)
  		  
CPUMotherboardGraphicsRAM
E8400 DFI LP DK P45-T2RS Sapphire 4850 512MB 2x2GB G.Skill 1066 (5-5-5-15-2T 1.9V) 
Hard DriveOptical DriveOSMonitor
820GB Total LG Super DVD Burner SATA VII Enterprise x64 Gateway FHD2401 
PowerCaseMouseMouse Pad
Corsair VX550W NZXT Source 210 White Logitech G400 Razer Kabuto 
Audio
Creative Audigy 
View all
  hide details  
Reply
had a fatal error.
(14 items)
  		  
CPUMotherboardGraphicsRAM
E8400 DFI LP DK P45-T2RS Sapphire 4850 512MB 2x2GB G.Skill 1066 (5-5-5-15-2T 1.9V) 
Hard DriveOptical DriveOSMonitor
820GB Total LG Super DVD Burner SATA VII Enterprise x64 Gateway FHD2401 
PowerCaseMouseMouse Pad
Corsair VX550W NZXT Source 210 White Logitech G400 Razer Kabuto 
Audio
Creative Audigy 
View all
  hide details  
Reply
post #4 of 5
3/22/13 at 1:55pm
Duplicate Files Deleter is a powerful tool to find and resolve duplicate photos, documents, spreadsheets, MP3's, and more!
Removing duplicates will also help to speed up indexing and reduces back up size and time.
Your computer isn’t fully optimized until you’ve removed all unnecessary duplicate files. Let Duplicate Files Deleter remove the duplicates!
http://krojamsoft.com/Products.php
Reply
Reply
post #5 of 5
4/17/13 at 2:43pm
Directory Report is very fast and should not choke on network drives
http://www.file-utilities.com
Check the Task Manager anyway
Reply
Reply
New Posts  All Forums:Forum Nav:
  Return Home
  Back to Forum: Other Software
Overclock.net › Forums › Software, Programming and Coding › Other Software › Best method for finding duplicate files?