Current Storage options ? (Suggestions)

[Christopher]

I'm quite happy with my current setup. One workstation Dual Xeon, 48GB ram, 1 System SSD, 1 SSD for Lightroom, 4 HDDs in RAID 0 as scratch. All my images are on a RAID5 with 4x2TB.

Nor the Problem is that 5,4TB are not enough. I'm running out of space really soon. I think there are two options to go:

1. Get a larger case which houses all stuff. Something like a Cube where I can put the mainboard on top and the hard drives on the bottom. There I could extend the storage from 4x2TB to 8x2TB or even 3TB.
Here I was also thinking on moving from RAID 5 to RAID 6, to get some more security. (I always have 3 backups of all important files) Does it make sense ?

2. Option remove "storage" drives from the main computer case. So just leave in the SSDs and Scratch disks. Now there is a question which way to turn.

- Light peak isn't an option, while I could wait till fall, nothing will change. As far as I know there isn't an option to add Light peak without replacing the motherboard. In addition I don't think Light peak will come to current systems.

- Fibre channel ? I have no experiences with it. When i take Lacie for example I would need a case with drives and interface like "http://www.lacie.com/de/products/product.htm?id=10507" and a card in my computer like "http://www.lacie.com/de/products/product.htm?id=10547"
If I do that, how is the whole system working ? I mean are these drives displayed as one normal hard-drives or more like network drives ? Thanks for any explanation.

- Last option would be to use a JBOD case and attach SAS cables. Something like "http://www.lacie.com/de/products/product.htm?id=10536" In addition, I would need to replace my current RAID card with one, that has external SAS/Sata connections.
Here the question is what type of JBOD cases are there ? Which support the type of connection I want ? I haven't found many options so far.



AT the ENd one more question. I was considering moving from RAID 5 to RAID 10. Any thoughts on that ?


Here some information. I mainly work with larger files. 400MB to 10-20GBs.

Thanks for the help.

Chris

[kaelaria]

Drobo Pro?

[Josh-H]

Drobo Pro?

Took the words out of my mouth. I've been running a Drobo Pro for the last 18 months as a back up to my mac pro. Great product in my experience.

[fike]

I think you need to split your archive into active and inactive archive--I use current year and prior year archives.  Your 4TB RAID system is big, and I would bet that a substantial amount of what you store on that is older and not actively used--your performance enhancing RAID5 is probably not as essential for those older files. 

Because I am sure you have some sort of backup plan, your older files are probably on at least 5 drives (four in the RAID and at least one more backup).  With enough backups, you can safely move those files onto a single local drive or NAS system where the performance isn't as good.

Here is what I do:

• Two 2TB drives in a RAID1 for current year archive files and all working and final product images
• One 1.5TB hard drive (no RAID) for prior year archives
• Two separate bare drive packages in rotation for offsite backup (each package contains one 1.5TB drive and one 2TB drive)
• Yearly hard drive snapshots of prior year archive and current year archive placed in fire safe box and never overwritten

With this system everything is always on at least three different drives, and frequently four drives:
a) Freshly downloaded from camera is on RAID1 plus local backup=3 drives
b) 6 months since download from camera is on RAID1, local backup and offsite backup drives=4 drives
c) Old downloads that are more than a year old are on a single hard drive inside the computer case and are on the local rotation of the backup disk, and the offsite rotation of the backup disk, and a yearly snapshot=4 drives

[Christopher]

Have you ever tried to open a 5Gb file from a Drobo Pro ;-) I have one and it is nice for storage, but useless for live editing.

I am considering splitting it up completely, but I have to admit that I really enjoy going back through some older files when I have some time and see if I can get something new from them. It wouldn't be so much fun if i needed to hook up something just to search through them. Especially, now that storage is so cheap.

[alain]

Hi Christopher,

I would think about replacing you're 4HDD raid 0 scratch with 2  sata 3 (600MB/s) SSD's, maybe even 1 fast SSD should equal the 4 HDD's in sequential read.

see:
http://www.anandtech.com/show/4341/ocz-vertex-3-max-iops-patriot-wildfire-ssds-reviewed/7

Given that you already have some 2.5" SSD's (I think) you could probably gain 3 3.5" slots and certainly 2.

[Steve Weldon]

+1 for the recommendations to split up your work into active and inactive archives.  I'd go a step further and use your LR SSD for your most current files you're processing, and as a place to move files from the archives which you want to work on.  

Of course it's possible to have the fastest storage with all your files, and if your budget allows then great.  But most choose an economy vs. performance based solution as it appears you also use.

I'm no longer a fan of RAID drives.  I've had the best RAID cards and cycled through all the different options, but ultimately I don't find them worth the expense or hassle.  I backup my files in at least three different locations.  I'd much rather use a straight 2-3tb internal drive by itself, and backup to another 2-3tb drive using an automatic backup program like Norton Ghost.  I use WD Black drives for my main archive drives, and WD Green for their backup drives.  In this manner I have 24tb in my case, all managed in the background via Norton Ghost during hours I normally sleep.

And then I backup on my network to NAS drives.  Annually I backup my years work which almost always fits on a separate 2.5 inch notebook drive, and I store it off-site in a fireproof safe.  Each year I use my docking station to run up each drive.

The NAS drives are my workhorses. I prefer them with the fastest USB/FW/Esata interfaces possible, but I only use these interfaces for initial moving of files, or restoring other drives.  These are used to provide a third backup of my internal drives, system drive backups for all my PC's, video/audio files that I can stream to my HDTV via my WD Live Hub, and whatever else.. I find NAS storage ultra convenient.

The lesser expensive NAS solutions are slow, in the 10-12mbps range.  Lately, some manufacturers have came out with some more speedy solutions promising speeds as fast as a standalone internal hard drive.  I'm currently researching this model, the DS2411+ Synology which advertises transfer speeds over a gigabyte network of 165mbps.. which is fast.  It's got a lot of other features I'm attracted to as well such as FTP features, multitasking, and it can interface with home surveillance systems.  The tech specs of the box are impressive as well.  The only reason I don't order this now is it's still using a USB 2.0 interface and I can't help but think a USB 3.0 interface is coming very soon.   Drobo's are nice, but I like the features/speed on this better as well as the price.

[fike]

I am also becoming a bit less enamored with RAID systems. they are just like hard drives...it's not a question of whether they will fail, but when.  With that said, I stick with RAID because it offers a bit more certainty than depending on my scheduled or manual backups.

I despise the constant rigamarole of keeping backups synchronized and all the backup jobs running...partly because I like to turn my machine off at night and partly because I rotate the backup destination drives in and out of off-site service.  This makes it so that I need to restart the backup each time I bring a new drive into rotation. furthermore, I dislike using imaging or other compressed backup formats because they are proprietary, and I may someday want to get to that data after the compression format has become obsolete.   It is a real pain.  Lately, I have started manually using Tera copy for backups. Every time I return from a large photo job or trip, after downloading, I kick-off the copy of the new files.  I probably should just reconfigure downloader pro to do that for me.

It really is too bad that cloud technology is so far from practical for this problem, because it really would be ideal.  Unfortunately with the size of raw files and the relatively slow internet connections we have, its impractical for anything besides a small collection of final copies.

[Steve Weldon]

It's true, any solution has its advantages and disadvantages.

RAID systems do fail.  And when they fail, say a RAID card/case hardware failure, most often the way to recover your data is to buy a new whatever failed in the first place.  Talk about proprietary, I find this more than irritating.  If the system eats itself through a software error, then that's proprietary as well.  Other than a straight hard disk failure recovering from a RAID failure is worse than hit and miss.  RAID manufacturers like you to think hard drives fail the most often, but that's old school.  Despite their mechanical assemblies, in my experience, I've found them 'as' reliable or more than any RAID card I've owned.  This is mostly because they don't cool the processors on RAID cards/cases like they should.  Heat kills.

And what do you back up a large RAID to other than another large RAID?  Most people don't bother.  The expense is extreme.


If you're going to use backup programs..  as you point out, you really shouldn't archive a backup file.  This goes back to your backup strategy.  I have my current work and current archives that I backup three times.  But my inactive archives are stored by year on separate hard drives in their original file formats, a copy each on/off site.  A lot of copies I know, but you only get burned once.  Or in my case I'm rather thick, so twice..

I leave my machine on so I schedule backups for the wee hours.  But I don't have the workload for a backup to take more than a few minutes anyway.. so I just as easily get them done during a lunch break.  And if not, systems are powerful enough these days to not bother you if they run in the background. 

Mostly though it's a flaw in strategy.  I should only be using Norton Ghost for my system backups.  For my drive to drive copies I should be using a cloning software instead.  I need to fix that.  That way it will happen in real time with a minimum performance hit.

[andyptak]

This is an issue I've struggled with numerous times over the years and I'm in the middle of re-tooling everything after yet another failure.

Firstly, I have many Terrabytes and I'm at a loss when I read about folks who are professional or prodigious shooters and have less than 2TBs - how come, I can do that in a year!

I had a big, Lacie box, RAID 5, which failed and I took it to a hardrive recovery specialist who charged me an additional $1200 on top of an already hefty fee because it was a RAID box - propriatery format and all of that.

Next came a Drobo, slow as molasses and poor customer service - into the garbage it went.

Than a Sans Digital box with four 2TB enterprise drives and set to RAID 10. It was toast in about a year and I couldn't recover the data without a clean room hardrive specialist, again.

All of the time I was making additional copies on multiple external Seagates, as big as I could find. It was time consuming and clunky, but these things saved my bacon more than once - I've only given half the story here.

Now, I'm ignoring the experts and forsaking RAID altogether. I bought 8 3TB USB 3.0 Seagate externals, four as primary drives and four as synched backups, using Memeo. I also backup to an HP server with 8 TB of Toshiba enterprise drives and using MS Server.

The Seagates are only consumer drives with a 2 year warranty, so I'm not sure, but I figure I have flexibility in a non-propriatery format and I'll see how it goes.

[Wseaton]

Having been involved in corporate data center support for more than a decade and worked with every level and type of storage hardware I can give you my $.02.

First, avoid RAID 5/6 like smallpox. It's an outdated technology that only mattered when 9gig SCSI drives cost $700 and using three drives -vs- four drives made financial sense. I've seen RAID 5 parity failures wipe out terrabytes of data on very expensive SAN clusters, and 99% of the time it's because a controller card went bad over a period of time and corrupted the parity stipe across multiple volumes. I've seen smart people lose jobs because of this very issue, and while I've had sympathy for them, I'm sick of arguing about it because I'm right. RAID 5/6 sucks and needs to be outlawed. If you want redundancy from HD failure stick to RAID 1. RAID 10 maybe.....in theory RAID 10 (multiple 1:1 mirrors with a stripe written across them while RAID 0/1 is reverse) should be just as hardy as RAID 1, but a faulty controller card can still kill data faster than a table saw goes through balsa wood.

I don't trust appliances - period. I kinda trust my $250,000 Hitachi SANs.....I don't trust anything else because I've seen eveything from desktops to million dollar iSeries arrays take a crap over the weekend. Me knows better. So, what's a digital photographer supposed to do other than pay a fortune for online back-ups and a T3 to your house? Let's break it down - first is the added expense of RAID. The reason we use RAID is for HD redundancy, and we all know HD's fail. If you use some of the inexpensive JBOD / Linux solutions you get a lot of cheap storage, and often good performance ratios, *BUT* you lose the peace of mind with RAID. So, how do we get rid of the need for RAID and lessen the chance of HD failure...I'm getting to that :-)

All HD's are subject to MTF (mean time to failure), and controller cards will fail as well. However, as many rodeos as I've been to and as many enterprise data centers I've been in one thing I've *never seen* is a HD fail when it's just sitting not powered up. In theory a non powered drive will last almost forever, or at least until gravity slowly bends the platters and actuators until they are out of alignment. Don't laugh - HD orientation was a problem up until the mid 90's. The short form is - if you want to maximize HD reliability, then don't plug them in. No worry about summer lighting strikes grounding through MB interfaces or power supplies burning up and sending 110VAC down through the 12volts rails.

So, my suggestion to maximize speed, durability, and price per terrabyte is to first split your data up between archive and non-working. Build a big, generic tower, stick gig ethernet in it, and use your favorite OS to create a non-proprietary storage pool you can add to at will. When a drive fills up, archive it, and unplug it. Or, use an external eSATA connector on your workstation and as you archive data unplug the drive and stick it in a cheap foam case for storage. As new HD's come out with more storage feel free to use them.

If you just have to have 5 terrabytes of online storage at all times, and have it be reliable and speedy to access, you are talking about a dedicated fiber channel or iSCSI SANs, and preferable Enterprise level - period. Otherwise, putting your drives in an offline state for archiving negates the need for RAID.

[andyptak]

@Wseaton

I wouldn't promote my expertize in this area, because I have none. I'm just a frustrated photographer who feels that they are single handedly propping up the share price of Seagate and Western Digital with my continuous purchases of their products.

That being said, I have heard stories of dormant drives suffering from "data decay", or some such term, or just failing to fire up after sitting idle for a long period. The best piece I read about this (can't remember where) stated that about once a year a drive should be started up and that each block should be addressed to "wake up" those pixels. Copying the data to another drive was one suggestion of keeping the original data in archival condition.

Thoughts?

[alain]

@Wseaton

I wouldn't promote my expertize in this area, because I have none. I'm just a frustrated photographer who feels that they are single handedly propping up the share price of Seagate and Western Digital with my continuous purchases of their products.

That being said, I have heard stories of dormant drives suffering from "data decay", or some such term, or just failing to fire up after sitting idle for a long period. The best piece I read about this (can't remember where) stated that about once a year a drive should be started up and that each block should be addressed to "wake up" those pixels. Copying the data to another drive was one suggestion of keeping the original data in archival condition.

Thoughts?

Hi

I use md5 checksums to verify if my backuped files (disks) have not changed and an the same time read the complete disk. 
I use either md5deep or more recent fileverifier++.  (A scan may take hours...)