Luminous Landscape Forum
Equipment & Techniques => Computers & Peripherals => Topic started by: Eric Brody on March 23, 2019, 11:51:53 am
-
Do I need to make the significant investment in ever larger hard drives? Currently, I have about 80,000 images going back many years, stored on an 8TB drive. It lives in an OWC 4 drive JBOD box and is automatically backed up nightly with Carbon Copy Cloner to two other 8TB drives. I also use Backblaze and keep an 8TB backup drive off site. I believe my backups are reasonably secure. I like the idea of having access to all my digital files quickly. I have done enough keywording to be able to locate older files quite easily.
The problem is that the 8TB drive is just about full.
I am wondering what other folks do in this situation. The larger drives have a larger cost per TB but are faster. One question is how large-10TB, 12TB, even 14TB (which might last me the rest of my life-I'm in my early 70's). One friend's suggestion was to use softRAID 5 and put four 8TB drives in the box. How does one back up a RAID volume? I realize I'm getting into complex territory here.
Suggestions would be welcome.
Thanks.
Eric
-
One solution could be to split the library into two - by whatever criteria you want - and then keep one as the "recent" or "active" and the other as "archive".
-
Get a bigger drive! Much better than splitting catalogs. And no, making a RAID isn't difficult, at least not on a Mac if that's what you're using.
-
If that’s a JBOD box with 4 drive capacity why not just add another drive? You have the backup part sorted.
-
Whatever you do, splitting your catalog is a terrible option - just ditch that thought!
Lightroom catalogs can easily handle an image collection spread across many logical or actual drives - no issue there... I get that just adding another physical drive will complicate your backing up solution, but it is a cheap option...
Much better though would be to go with the RAID option already suggested...
-
For a single drive, something like this works very well -
https://www.bhphotovideo.com/c/product/1360172-REG/g_technology_0g05383_g_drive_pro_12tb_7200_2x.html
For a RAID array I recommend this (but it is not cheap) -
https://www.bhphotovideo.com/c/product/1339806-REG/promise_technology_p3r4hd16us_16tb_pegasus3_r3.html
-
May i ask, what kind of computer do you have. To get a more complete image of how you work?
-
I just realized you probably are on Mac.
On Windows you can use Storage spaces: you add several drives and Windows sees it as one so your back up routine would be the same even if you change the drives. It equally distributes the information between them, behaves kind like JBOD but you can replace any if you have enough room and give enough time to the system to redistribute the data.
If you have something similar in Mac it should be the fastest way.
-
Hi Kers, Thanks for asking. I am currently using a 2013 Mac Pro (trashcan) with 64GB RAM, dual D300 video cards, and a 1 TB SSD.
FWIW, I also want to speed up saving PS files. I tried saving files (each of which is 1-2GB, from a Sony A7RIII) back to the SSD, just to see how fast it would be. It's really not much faster than saving to one of the 8TB hard drives in the Thunderbolt 2 JBOD.
I even downloaded a trial of SoftRAID XT and tried it with a set up of three 6TB drives (12TB RAID4) in the Thunderbay and it was not significantly faster either.
Thanks.
Eric
-
Hey Eric, for a point of reference, approximately how large are the photo directories for the last few years - aka how much are you adding each year?
I'm reading your disk layout as:
TB2 enclosure: [8tb Active] [8tb Backup1] [8tb Backup2] [empty]
I think you're close, but there seems to be an issue or two. The platter hard drives (8tb) should be noticeably slower than the internal SSD, or a SSD in the OWC JBOD if it's connected by TB2. RAID4/5 should be a bit faster than a single drive, just in that there are 3x the drive writes for 1.5x the data (slice A, slice B & Parity). Just to confirm when you save to the internal drive of the Mac Pro it's crazy fast (relative to saving to the external drive).
The more I'm thinking about it there is a few options to consider. You can upgrade your 4 disks to 12tb's for around $1,600 and gain some 3tb of free space, and maintain your current workflow and methods.
If you were to add disks, I'd actually think about snagging a "OWC Mercury Elite Pro Quad RAID" as an external usb3 enclosure that we'd move or add disks to. TB2 is great, but it has a price premium, and for backup disks to be in a separate enclosure adds another level of protection (enclosure going bad). I'm thinking of adding 4x 8tb drives, so you'd be gaining around 7tb of usable space. I would also add in a 1-2tb SATA SSD to the shopping cart and configure it like:
TB2 enclosure
[New8tb Active] [New8tb Backup1] [Old8tb Active] [NewSSD]
USB3 enclosure
[New8tb Backup2] [Old8tb Backup1] [Old8tb Backup2] [empty]
4th new 8tb is the second offsite disk.
I'd most likely mix the new and old disks - as in have all data on new disks & old disks. This would give you fast access to the new & old data, the initial backup of new data would be within the TB2 enclosure, and all data would live on both enclosures.
The SSD would be on the faster bus & would allow for better saving performance. I'd check which SSD you tested with as some of the older ones that need to 'wipe' a cell before writing to it (that whole TRIM thing) may have impacted it. OWC sells the enclosure with and without SoftRAID XT, it adds $70 to the price.
-
OWC sells the enclosure with and without SoftRAID XT, it adds $70 to the price.
[/quote]
Last I knew, the package with the enclosure and Softraid does not include the standalone version of Softraid. The package Softraid only allows use with the specific enclosure and lacks some features that the standalone version offers and which I think are worthwhile.
-
Yea, the 'XT' branded one requires the OWC enclosures, so it should work fine with the one he has, plus the new one. XT gives RAID0/1/1+0/4/5 while the 'XT Lite' is limited RAID0/1.