Pages: 1 [2]   Go Down

Author Topic: RAID 5 recommendations please  (Read 20720 times)

RicAgu

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 267
RAID 5 recommendations please
« Reply #20 on: April 07, 2007, 07:12:47 pm »

I always like multiple options for connecting.  I had a bad firewire experience years ago and now always want at leaset three connection options.  Firewire 400, 800 (x2) and USB so kind of like a quad connect.  They do make a real quad connect that includes SATA II.

Best of luck
Logged

Dennishh

  • Full Member
  • ***
  • Offline Offline
  • Posts: 160
RAID 5 recommendations please
« Reply #21 on: April 07, 2007, 08:28:01 pm »

I’ve been using eSata for about three months now and find it to be the best thing going. I’m on a PC with windows XP, so you guys with Macs should have no problems with this set up. I use two Sonnet Fusion 500p five bay enclosures http://www.sonnettech.com/product/fusion500p.html One eSata cable connects five drives in each box. I use two wd raptors at raid 0 and three wd 500 gig drives for a raid 5 array. This box costs $400 the drives are about $150.00. These are all running from a $50 two port card. http://www.usb-ware.com/pci-express-2-port-esata-ii-card.htm The key to this whole set up is the silicon image 3132 chipset. You will need an e-sata card for Mac, I think pc express makes the same card for Mac. I had a problems with the array always having to rebuild and found out the culprit was the type of hard drive. The drives for desktops are not good with these enclosures. The western Digital WD5000ys drives are made for raid external enclosures. I have had no rebuilds at all after starting to use these drives. http://www.newegg.com/Product/Product.asp?...C-AdwordsFeeder
This set up is so fast!!!
Dennis
Boston
« Last Edit: April 07, 2007, 08:29:08 pm by Dennishh »
Logged

nma

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 312
RAID 5 recommendations please
« Reply #22 on: April 07, 2007, 09:28:47 pm »

Quote
[span style=\'font-size:8pt;line-height:100%\'](if this is the wrong forum to ask this question, please forgive me. I checked the forum descriptions but couldn't find one that directly related to storage equipment. The description for the "Managing Megabytes" forum states "Discussion regarding The Great Luminous Landscape 2006 State-of-The-Art Shootout"!)[/span]

hello all.

I need to back up my photo library (and other computer files). I've had too many incidents with DVDs and they take too long to back up to anyway so I've decided to go with RAID 5.

Preferably, I'd like:
  • A minimum of 3 active drives + 1 spare drive
  • FireWire 800 AND 400. USB not necessary. ie; stand-alone RAID without the need for a RAID card (I'm using a Core Duo 2 iMac and a MacBook Pro).
  • Hotswap (albeit optional because I'm not running a business. I can shut the RAID down if I need to.)
  • Compatible with commonly available 3.5" drives (eg, Barracudas from Seagate because I can get 5-year warranties when I buy these locally here in Hong Kong).
I don't want OEM drives because they only offer 1 year warranty.
I don't want Lacie because I've had too many problems with their equipment.

Any recommendations?





kind regards,
Gregory
[a href=\"index.php?act=findpost&pid=110970\"][{POST_SNAPBACK}][/a]

Dear All,

We all face serious difficulties in safegaurding our photo data, but Raid 5 is not a backup solution or archival. Professionally, we find them to be convenient, but unreliable. They only provide recovery if a single drive fails. Other failures modes causing data loss cannot be recovered. And this statement is based on experience, not theory. Examples include problems with the raid controller that corrupt the array as well as problems with the power supplies.  

Better solutions involve multiple disk copies,  tape backup, mirrored raid 5, etc.  No backup strategy should be relied on unless it is tested.  Simple is best.  Things are changing very quickly. No one actually knows how long these elctromagnetic devices will work. Can we be sure that a drive left on the shelf for two years will function reliably? I don't think so.  

Soultions involving backup on AIT-tape are rated at ~ 20 years in controlled environements.  There is much more expereince with tape. It is not very convenient nor cheap, but it works.  Still, there are pitfalls with tape, as well: Proprietary recovery sofware may  cease to be supported. Tapes have to be rewritten as they age.  As you change computers or operating systems you have to ensure that software and drivers exist.  


All this means that one must think carefully and not be impulsive or overconfident.

'Hope this helps.
Logged

lankford

  • Newbie
  • *
  • Offline Offline
  • Posts: 1
    • http://www.sundogimage.com
RAID 5 recommendations please
« Reply #23 on: April 08, 2007, 10:42:36 pm »

I am very pleased with the wiebetech RAIDTech 800, has usb 2.0, firewire 400 and 800.  I am using mirrored setup with two 750 gig and transfer times are fast enough for me. For another assurance I use a western digital 500 gig external drive that you can get at best buy, etc.  (pics in 3 places)
Logged
Stacy Lankford

Ed Foster, Jr.

  • Full Member
  • ***
  • Offline Offline
  • Posts: 219
    • http://www.edfoster.net
RAID 5 recommendations please
« Reply #24 on: April 09, 2007, 10:31:30 am »

Quote
Ouch, I think I see why its so inexpensive and what might be an issue. The ONLY port out is a SATA cable, no Firewire or USB so I'm not sure if I use one bay as a backup, I can boot from it.

Also, is having the single port a bad option?
[a href=\"index.php?act=findpost&pid=111230\"][{POST_SNAPBACK}][/a]
While one might want the option for multiple ports for other needs, when I researched this and set up mine last year I found the following:  SATA connections provides the fastest throughput over USB or FW 400 and 800; external USB and FW drives proved unreliable (for me) and, at that time, you could not boot from the externals utilizing USB or FW connections.  You can boot from your external SATA connected drives because, in essence, they are like your internals in terms of protocol.   As for drives, WD's & Maxtors I have had problems with over the years, but Seagates have been reliable for me, so I chose Seagate's Barracuda 7200.10 series.  Today you can find them at less than $150.00 for 500 gb and 750 gb for around $250.0.

Ed
Logged
Ed Foster, Jr.
[url=http://www.edfoster.

cmburns

  • Newbie
  • *
  • Offline Offline
  • Posts: 47
RAID 5 recommendations please
« Reply #25 on: April 09, 2007, 12:21:45 pm »

My personal experience is you're increasing your chances of losing data by going to Raid setup. In my case I had a Raid 5+1 array. It was run from a dedicated Raid controller card and it had an onboard battery pack that went dead and corrupted the array. Of course I thought it was 20 other things until I figured out that was the problem.(I had to buy another controller card and when it worked I realized it was the original card, until then it looked like so many other things were the problem) I also never got it to work as I hoped, where I would have 3 sets of drives, 2 active, and one out of the computer as a backup. I hoped to be able to pull out one set every week, put the other one in and have it autoupdate the older array. That just never worked right. You also have a lot more noise and heat. Make sure you have a helluva case with lots of fans, those hard drives will get hot hot especially all packed together. My setup was fairly robust corporate stuff and when it failed I was left not trusting raid.

Always go with the KISS method, keep it simple stupid. Raid is not simple. There's way more that can go wrong and you're lured in with this false sense of security. "I'll be ok if a drive or even two fail." Yeah but what if your power fails, do you have a backup that will run the computer for a while. FYI I did, but still the onboard battery pack got me. What if you're not there when the power fails and you don't get it shut down before your backup runs out, etc., just a lot that can go wrong. To me most consumer raid stuff today is made to access your data fast, not as any kind of backup solution. It's for video editing where you need to keep a ton of data on the computer, have fast scratch disks etc, but you can bet that data is also backed up safely somewhere else.
KISS method for me means backing up to an external hard drive. As another poster said, esata is where it's at now for speed, but USB2.0 is much more universal. I can take the USB enclosure to any computer I have and copy to or from. I never bother screwing the enclosure together, because i'm always swapping hard drive in and out of it.
I also burn 2 sets of DVD's and store in 2 seperate places, One offsite. Thus far my worse case has been a hard drive failure, and out of 50 or so DVD's that was used to restore it, 2 had a handful of unreadable files on them. The 2nd set of dvd's read those files just fine and I then remade the bad dvd's. The point is, all of this stuff fails so have multiple redundancy.

I organize my stuff in a folder by date so it might say "2007-04-08 Easter at so and so's.", I then run a dos command (in windows click start, then run, then type in "cmd" to get a dos prompt. Then d: to get to the root directory of the D drive for instance.)
dir /B *.* > c:\list.txt
 that spits out all the folders and files in the root directory of whatever drive you're on to a text file in the root of the c drive. If you want all the file names in the subdirectories, basically everything on the disk, add a /S to that.

I then cut and paste that into a master text file. I put the hard drive into the little static proof bag it came in, and put a sticker on the outside with a number. So then I can look at my text file(or better yet search through it) and see that hard drive so and so has this or that shoot on it. Get it out, plug into enclosure and have the files copied back to the main computer in minutes.

I've read several advice things that say it's good to do a scandisk on your hard drives, say every 6 months. The longer they sit not being used the more likely they'll get errors. They have error correction built into them so when you run the scandisk the hard drives will find tiny errors and self correct them using it's parity data.

The problem I have is when a hard drive fails, it's really getting to be a chore to back up from dvd's. A 500gb drive means you're going to be spending a lot of time copying the contents of 100+ dvd's over. Luckily I have 4 computers around the house with dvd drives, but that's still a day of poking dvd's in and out and then copying to a replacement drive in an enclosure. Hopefully another year or so and we'll have Blu Ray or HD DVD writable disks that will hold a lot more data.
Logged

Nill Toulme

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 738
    • http://www.toulmephoto.com
RAID 5 recommendations please
« Reply #26 on: April 09, 2007, 01:07:06 pm »

That's a fairly anecdotal reason for dumping RAID.  Sort of like "I had a car once, and it broke.  You'll never see me in one of those things again."

As for backing up to DVD's or other optical media, I have to say I just don't get it.  Backing up is bad enough, but what are you going to do if you ever have to do a complete restore?  You can get a nice 500GB Seagate USB2 external HD for 150 bucks; for 300 clams you can have set-it and-forget-it dual redundant backups of your entire data set.

What is the attraction of DVD backup?  What am I missing?

Nill
~~
www.toulme.net
Logged

cmburns

  • Newbie
  • *
  • Offline Offline
  • Posts: 47
RAID 5 recommendations please
« Reply #27 on: April 09, 2007, 02:16:44 pm »

For me the plusses of the dvd backup is longevity and cheapness. Hard drives are not made to last. They're made to be obsolete and replaced by bigger hard drives in a few years. Buy quality DVD's and you'll be ok for a much longer time period than hard drives and that's if you use good hard drives. Do you? I know for DVD's buy Taiyo Yuden and forget it. For hard drives you have to wait for a drive to be out for a while then do some digging to find reviews and failure rates etc. Maybe a Seagate so and so .9 is great and then the newer .10 version is crap.(I just ran into this as I saw a drive at a fantastic price, the drive has horrible reviews though so I payed a little more for a more reliable model)

Take a look at this
http://labs.google.com/papers/disk_failures.pdf

Their measured baseline failure rate is about 8% after 2 years and that's across all models. Buy a bad model and it will be much higher. Also note that drives with little use fail at a higher rate than those that see regular use.
Cost. you can buy hard drives for around 25 cents/gb, so a terrabyte of storage will set you back $250. No biggie in this line of work, but the same terrabtyte of DVD's would be about 1/3 as much, they just take a lot more of your time.

As for my raid failure being anecdotal, of course. But it was supposed to be perfect. I did tons of research, bought the best stuff, scsi drives, a stud raid controller card, power backup, etc. and it get's taken out by a failing battery. There's just too much that can go wrong with it. Nothing is bullet proof, just do multiple backups and hope you don't have acts of God at your place and your offsite storage as well.

Raid is just not a good backup solution. Use it as a huge volume for Lightroom, ok, just have the whole thing backed up somewhere else.

Oh another consideration, what if your motherboard fails after a year or two? This also has happened to me, simply a capacitor going bad, they buy the cheapest ones possible to keep cost down. How will you get your data off your array? Will there still be that model of motherboard to buy? Does the same company have a newer motherboard with raid controller that is backwards compatible etc., maybe buy 2 motherboards. When my raid controller failed I was lucky to find a used one on Ebay. Computer stuff goes obsolete so fast.
Logged

Nill Toulme

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 738
    • http://www.toulmephoto.com
RAID 5 recommendations please
« Reply #28 on: April 09, 2007, 02:49:30 pm »

Quote
...Raid is just not a good backup solution. ...
RAID is not a backup solution, period.  RAID has many advantages, notably among them for purposes of the present discussion avoiding drive failure in the first place, thus avoiding downtime and the need for painful restores.  As I mentioned above, I've had a drive fail in my RAID.  I simply replaced it and let the array rebuild itself, meanwhile carrying on business as usual.  Yes RAID controllers fail.  Motherboards fail, everything fails.  It's just a matter of playing the odds (i.e., the odds that a drive will fail being MUCH higher than that a controller will fail), and being prepared with backups when you lose.

Quote
...Oh another consideration, what if your motherboard fails after a year or two? This also has happened to me, simply a capacitor going bad, they buy the cheapest ones possible to keep cost down. How will you get your data off your array? ...
That's what your backups are for.  And when you hope you've got one on something other than DVD's.

Nill
~~
www.toulme.net
Logged

Nill Toulme

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 738
    • http://www.toulmephoto.com
RAID 5 recommendations please
« Reply #29 on: April 09, 2007, 02:55:03 pm »

p.s.

Quote
For me the plusses of the dvd backup is longevity and cheapness. ... Buy quality DVD's and you'll be ok for a much longer time period than hard drives and that's if you use good hard drives. Do you? I know for DVD's buy Taiyo Yuden and forget it. ...
That remains to be seen, does it not?  You're not relying on one copy of each of those DVD's, are you?  What's your regimen for checking every one of them periodically to see if they still work?

Nill
~~
www.toulme.net
Logged

digitaldog

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 20944
  • Andrew Rodney
    • http://www.digitaldog.net/
RAID 5 recommendations please
« Reply #30 on: April 09, 2007, 02:56:24 pm »

Quote
RAID is not a backup solution, period. ~~
www.toulme.net
[a href=\"index.php?act=findpost&pid=111520\"][{POST_SNAPBACK}][/a]

Amen! I didn't think it needed mentioning but not a bad idea. My interest in a RAID isn't to forgo backing up (I'm super anal about that) but rather to have one less area to worry about. Plus, once my RAID bucket fills (using Peter Krogh's term), I'd like one drive for offsite back up and one on site just to access the files if necessary. IOW, I have to constantly do backups to two sets of drives anyway which is real time consuming. Three copies is even better (and more work).

I've been on the Mac since 1988 and I guess I'm lucky but I've never had a drive fail. Doesn't stop me from backing up quite often. Maybe that's the key <g>.

Ideally my goal is to have two drives (mirror) the current Lightroom Library I'm working on. So dual buckets. Once filled, one goes offsite or in a media safe, the other can be popped into an empty shuttle in case I want to pull off a file or print something. That's why I'd really like a four or five bay system. Two for the current Raid 1, one maybe for the system back up and one to pop drives into just to access preexisting data.
Logged
http://www.digitaldog.net/
Author "Color Management for Photographers".

nma

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 312
RAID 5 recommendations please
« Reply #31 on: April 09, 2007, 02:59:32 pm »

Quote
That's a fairly anecdotal reason for dumping RAID.  Sort of like "I had a car once, and it broke.  You'll never see me in one of those things again."

As for backing up to DVD's or other optical media, I have to say I just don't get it.  Backing up is bad enough, but what are you going to do if you ever have to do a complete restore?  You can get a nice 500GB Seagate USB2 external HD for 150 bucks; for 300 clams you can have set-it and-forget-it dual redundant backups of your entire data set.

What is the attraction of DVD backup?  What am I missing?

Nill
~~
www.toulme.net
[a href=\"index.php?act=findpost&pid=111501\"][{POST_SNAPBACK}][/a]

Nill,
cmburns provided more than anecdotal advice. Even mirrored RAID 5 is not a sufficient archival solution. It is possible to have failures that corrupt both arrays.  As part of my professional work we have been using RAID 5 and NAS for about 10 years.  This equipment is supposed to be more reliable than ordinary consumer solutions. Single drive failures have not been a big problem. The gotchas we have encountered included failures of the RAID controllers, causing disk array corruption and failure of the dual power supplies to survive an electrical spike, again causing corruption of the disk array.  Once the disk array is corrupted, the solution is to reformat. The rule of thumb is to replace RAID 5 after about 3 years. As noted by cmburns, if it becomes necessary to replace the computer and or RAID controller all bets are off.  Three years in the computer business is a long time. Parts become unavailable, companies go out of business, etc.  I can offer no fool proof solutions.
As I wrote earlier, tape back up has a longer history and thus we know it can outlast disk storage. DVD as an archival medium is not well established and the capacity is small.  Finally, no backup solution can be relied on unless you have demonstrated that you can recover your data.  There have been more than one data center  who was embarrassed to find that their back up was no good.

I hope this is food for thought.
Logged

Nill Toulme

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 738
    • http://www.toulmephoto.com
RAID 5 recommendations please
« Reply #32 on: April 09, 2007, 03:09:05 pm »

Andrew I'm probably stating the obvious (again...), but be thoughtful about a "mirror" solution just as you are with RAID.  A true mirror will still subject you to all the other possible causes of data loss, other than drive failure, just as a simple RAID will.  IOW, mirroring is not truly a backup solution either.

Nill
~~
www.toulme.net
Logged

digitaldog

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 20944
  • Andrew Rodney
    • http://www.digitaldog.net/
RAID 5 recommendations please
« Reply #33 on: April 09, 2007, 03:12:48 pm »

Quote
Andrew I'm probably stating the obvious (again...), but be thoughtful about a "mirror" solution just as you are with RAID.  A true mirror will still subject you to all the other possible causes of data loss, other than drive failure, just as a simple RAID will.  IOW, mirroring is not truly a backup solution either.

Nill
~~
www.toulme.net
[{POST_SNAPBACK}][/a]

Right. I will of course continue to back up. Old habits die hard.

As to DVD's, I got this email from my bud and tech editor Karl Lang awhile back and the product does look useful:

Quote
[a href=\"http://www.truedisc.com/]http://www.truedisc.com/[/url]

Hey guys,

This was just released. The marketing hype is too much to take, 
however the basic idea is really, really great.

Record a standard 9660 ISO CD/DVD. Don't fill it all the way up leave 
some room.

Use the free space to store redundant checksum data. Then if 
something happens to the disk use their recovery software to re-
extract the file data with the redundant data. Up to 90% of the 
original could be damaged and still recover the file.

If this becomes a standard format we can all sleep better at night. I 
think I may buy it just to support the idea. If I was Adobe or Apple, 
I would buy this company right now. It's such a simple and easy idea, 
at the OS level it would be awesome.

Karl

My only beef with DVD is it's not large enough.
Logged
http://www.digitaldog.net/
Author "Color Management for Photographers".

Nill Toulme

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 738
    • http://www.toulmephoto.com
RAID 5 recommendations please
« Reply #34 on: April 09, 2007, 03:13:24 pm »

Quote
Nill,
cmburns provided more than anecdotal advice. Even mirrored RAID 5 is not a sufficient archival solution. It is possible to have failures that corrupt both arrays.  ...[{POST_SNAPBACK}][/a]
Exactly.  I suspect you were typing this as I was typing my "RAID is not a backup solution, period" post above.  What I was really taking exception to was cmburns's statement that "...you're increasing your chances of losing data by going to Raid setup."  I think that's simply wrong (well, unless you're talking about RAID 0... ).

Nill
~~
[a href=\"http://www.toulme.net]www.toulme.net[/url]
« Last Edit: April 09, 2007, 03:19:44 pm by Nill Toulme »
Logged

Nill Toulme

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 738
    • http://www.toulmephoto.com
RAID 5 recommendations please
« Reply #35 on: April 09, 2007, 03:17:43 pm »

Quote
...My only beef with DVD is it's not large enough.
[{POST_SNAPBACK}][/a]
Right... even with checksum data. ;-)  

It's feeding the dadburn things into the drive that's so daunting, especially if you contemplate the restore of a large amount of data.  Remember LP changers?  If you could just stick 500 of the little suckers into some sort of auto loader, I suppose I might be moderately interested in them.

Nill
~~
[a href=\"http://www.toulme.net]www.toulme.net[/url]
Logged
Pages: 1 [2]   Go Up