Luminous Landscape Forum
Equipment & Techniques => Digital Asset Management => Topic started by: louoates on April 03, 2014, 06:08:15 pm
-
Help! Anyone have experience with Mac software to help find and delete duplicate photo files across multiple hard drives? My 4TB external backup drives are spilling over and I know I've been paranoid in keeping way too many copies of the same photographs but on different drives. Price is not a factor but not trashing my Mac is.
-
I'm confused, if you have duplicate backups, why not use clone copy this will duplicate the drive but is you delete one file you just do a "backup clone" and it will erase the same file on all backup drives as well... Or am I misunderstanding what you Are doing?
-
Do you want to search for specific files, or are you looking to let the software scan all your files to look for duplicates? If the former, then use Spotlight. If the latter, then perhaps a program like Printwindow5 (http://www.printwindowapp.com/default.html) might help.
-
There’s PhotoSweeper, on the App Store for $10. Has good reviews; I haven’t used it, so can’t recommend one way or the other. There is also Photo Duplicate Cleaner, free on the App Store, also has good reviews. There are probably more too, those are just the ones I saw on the first page of the Photo category.
-
Thanks, all.
I'm trying to delete enough photo files so that Time Machine can put my entire system backup onto a 4TB external drive. I have almost 1 TB too many files. The best solution would be to have a program scan my 4 internal drives and auto-identify identical files that I can manually delete. I know I have duplicate psd files across 2 or 3 drives. I've noticed that some software checks just for identical folders and file names, not for exact file data.
I will probably delete most of my old raw files also.
I am looking at Araxis's FDF (Find Duplicate Files) software and may download a trial version to see if it works. All comments are welcome, especially if you've purchased and used FDF or other apps.
-
Why not make it easier on you and get a couple more large drives, one for the files, the other a clone.
Compared to file management and finding dupe files, big drives are cheap. However, the time for go out to shoot and to enjoy photography are priceless!
-
I agree with langier. Life's too short. If you delete a file and it is a referenced file for something else it will just cause problems. Get a big Drobo or something and stop making duplicates.
I just have one copy on a hard drive and all the programs like Aperture, Capture One, Lightroom reference to the same files. It is actually on a server so all the computers can use it. That drive is TimeMachine backed up each night.
-
I'm taking your advice and not worrying about the dupes. Thanks for all your comments.
-
Pretty sure Adobe Lightroom has a find duplicates feature... I can't be certain but I think it does.
-
No it doesn't. It just has a way to avoid importing duplicates.
-
No it doesn't. It just has a way to avoid importing duplicates.
OH, gotcha, I knew I remembered something about that, thanks for the correct info
-
For the record I have found duplicates using Bridge set to sort by file size. This works well for rendered files like JPG or TIFF but I imagine won't be at all useful for RAW files.
-
You might check out Gemini:
http://macpaw.com/gemini
Available at the Mac App store
-
There’s PhotoSweeper, on the App Store for $10. Has good reviews; I haven’t used it, so can’t recommend one way or the other. There is also Photo Duplicate Cleaner, free on the App Store, also has good reviews. There are probably more too, those are just the ones I saw on the first page of the Photo category.
I bought it last month and used it, and was very pleased. It was flexible, worked well, and gives nice feedback so you can see what you're doing.
-
Help! Anyone have experience with Mac software to help find and delete duplicate photo files across multiple hard drives? My 4TB external backup drives are spilling over and I know I've been paranoid in keeping way too many copies of the same photographs but on different drives. Price is not a factor but not trashing my Mac is.
Hi, I was looking for something to manage all my clicked pictures on my Mac (which I had too much). I came across an amazing duplicate photos cleaner which even brings up the similar photos. You can try it from here:
www.duplicatephotofixer.com/Photofixer.aspx?utm_source=frm
-
The easiest way is to sort your files by size and then name. If you see two files with the same size and name, then they are almost certainly dups. HOWEVER: if both copies have ever been added to a cataloging program such as Lightroom, or if you have made different edits which are stored next to each copy in a sidecar file, you will most likely cause problems by deleting one.
What I did several years ago when face with this issue was to do the 80/20 fix. I had no catalogs or sidecar files.
0. Make a copy of everything onto an external drive, then disconnect the drive from my system.
1. Using the above sort, I identified the most egregious duplicate folders and deleted them.
2. Grouped the remaining images into some sort of calendar-based directory structure. In my case, I established a Photo Master folder, then a YYYY sub folder, then a YYYY-MM subfolder within.
3. Installed LR and imported everything into the catalog using the "do not move" option to leave the images in the Photo Master folder.
4. Over time, I regrouped, eliminated dups that were obvious and keyworded images I came across.
5. All new imports get copied from the SD card into the appropriate folder within this structure, with a Shoot Name added to the yyyy-mm- folder name.
-
Duplicate Detective is pretty good and inexpensive. http://www.macworld.com/article/2086400/duplicate-detective-review-make-short-work-of-finding-duplicate-files.html
-
Hi,
try Photo Duplicate. Its simple and works fast.
-
First of all, Time Machine sucks for backup. It's okay for version tracking, though. In general I don't trust any Apple-made solutions, I use their OS and that's it. I worked as Applecare support, so maybe I know something.
I use Carbon Copy Cloner --and there is also Super Duper-- for backups. These both make bootable exact clones, no games with symbolic links and not being able to find the real file like with Time Machine.
I have two backup presets. One is called Identical, which I use on drives the same size in case I would need to swap them in to replace my internal drive. it makes an exact copy of what is on my drive. So if I back up using Identical and then reboot from that drive there is no difference in content. I keep a few of these around, it would take maybe 20 minutes to swap that drive into my MacBookPro if the internal drive died. In this mode, if I delete it on my internal drive, it won't be on the backup.
The other preset I call Attic Mode, and it just keeps piling stuff up. So if I have a folder of photos on my desktop and back up, then move it to my Photos folder and back up again, that folder is in both places on that backup drive. This is done on purpose, in case bit rot makes a certain file unopenable. It also allows me to do roughly the same thing as Time Machine, and find something I deleted off my internal drive but which I had backed up at some point. For this I use a drive that is several times larger than the internal drive. It will also be bootable. When it fills up I get another larger one.
Another point of these bootable drives is that after backing up I can boot from one of them and run disk utilities on my internal drive to keep the directories healthy.
-
Thanks for posting your software tips. I tried most of the apps that were mentioned here and they helped me find some dupes already.
However, do you know of an app that actually scans the content of an image (reads and matches the pixels) instead of relying on metadata such as creation date, resolution or file size?
Over the course of years of backing up multiple iPhones through multiple (bad) ways, I have a huge pile of unsorted phone pictures that are both a pain to sift through (since there are so many dupes still) and at the same time quite dear to me.
Any hints?
-
You could check out SnapSelect from Macphun.
-
I use Disk Genius from OWC. It has a dupe finder that sorts by content on each drive. I haven't explored it fully yet, but it sounds like what you want.
Neil
-
You might check out Gemini:
http://macpaw.com/gemini
Available at the Mac App store
I had installed the MacPaw Cleaner on my laptop but somehow I dont trust this company.
I think they are from Ukraine... found a trail in my firewall. Now I use Duplicate Photo Finder
-
Try to instal this app https://itunes.apple.com/us/app/duplicate-file-finder-remover/id1152611077. It's a good freeware alternative to some paid programs.
-
Another vote for photosweeper after I got multiple dupes polluting my Lightroom database. Happened when I changed mobiles over the years and all the cloud sync services cross copied older images eg. OneDrive, Dropbox, iCloud via Lr mobile. Some jpegs have multiple dupes! Photosweeper has powerful analytical criteria to search for dupes. Its selection panel lets you confirm which dupes to tag for deletion having cross checked any metadata or you can autodelete, but that's asking for trouble. I also received a note from the developer with updates for high sierra so it is actively developed.
It looks a bit simple minded but has powerful features underneath.