Luminous Landscape Forum

Equipment & Techniques => Cameras, Lenses and Shooting gear => Topic started by: Gregory on April 06, 2007, 09:35:26 am

Title: RAID 5 recommendations please
Post by: Gregory on April 06, 2007, 09:35:26 am
[span style=\'font-size:8pt;line-height:100%\'](if this is the wrong forum to ask this question, please forgive me. I checked the forum descriptions but couldn't find one that directly related to storage equipment. The description for the "Managing Megabytes" forum states "Discussion regarding The Great Luminous Landscape 2006 State-of-The-Art Shootout"!)[/span]

hello all.

I need to back up my photo library (and other computer files). I've had too many incidents with DVDs and they take too long to back up to anyway so I've decided to go with RAID 5.

Preferably, I'd like:I don't want OEM drives because they only offer 1 year warranty.
I don't want Lacie because I've had too many problems with their equipment.

Any recommendations?

kind regards,
Gregory
Title: RAID 5 recommendations please
Post by: wolfnowl on April 06, 2007, 03:30:10 pm
I'd recommend Infrant but they're networked, not firewire.  I typed Raid 5 firewire 800 into Google and got 1.29 million hits.  You might want to look up Wiebetech (http://www.wiebetech.com/products/rt5.php (http://www.wiebetech.com/products/rt5.php) or http://www.wiebetech.com/products/rt5x2.php (http://www.wiebetech.com/products/rt5x2.php) if you're really thinking ahead) as I've heard good things about them.

Pity you couldn't stick in an eSATA card as your read/write times would be MUCH faster...

Mike.
Title: RAID 5 recommendations please
Post by: Jack Flesher on April 06, 2007, 04:31:16 pm
I was just in my local Fry's electronic supply store.  They had Seagate 500G SATA drives on sale for $119 each.   I have for a long time been preaching RAID 1-1 as a back-up strategy.  In effect I have two redundant copies of my current working image drive, one stored off-site and backed up monthly or after any large shoot, the second on-site as fully redundant back-up, and then keep two identical drives of my historical images, one on-site and one offsite.  At current prices, even keeping 3 equivalent drives for all my images ($360 for 500/500/500) my storage cost is well under $1 per gig ($360/500G = 72 cents per double-redundant Gig of stored data).  Dropping to a single-redundant strategy, your cost is only 48 cents per Gig.  And this is only getting cheaper as time passes.  

FWIW, I also recently picked up a handy little $30 Vantec device to support this strategy -- a SATA/IDE  to USB2 adapter, model CR-ISATAU2.  With this device you can transfer data to a 3.5" or 5.25" SATA OR IDE drive via a USB port and you don't need to have the drive in an enclosure!  Just pull the drive out of its factory cushion pack, plug the adapter onto it, load it with data, un-plug and put the drive back in its factory cushion pack for long-term, off-site storage.  

My .02 only,
Title: RAID 5 recommendations please
Post by: Nill Toulme on April 06, 2007, 05:44:00 pm
What kind of thruput do you get with that little gizmo Jack?

Nill
~~
www.toulme.net (http://www.toulme.net)
Title: RAID 5 recommendations please
Post by: digitaldog on April 06, 2007, 07:22:54 pm
Check out the products from these guys:

https://www.g-technology.com/index.cfm (https://www.g-technology.com/index.cfm)

I'm looking at the G-Safe. I did purchase their small driver (Mini) for travel and it's a beautifully built and quite fast unit.
Title: RAID 5 recommendations please
Post by: Dave Carter on April 06, 2007, 08:07:10 pm
HI,
Yea, I have been using G-RAIDs for quite some time.  I always have two hooked up to my computer via 400 firewire.  They are all 500 Gb (each housing cotains 2 drives) and are the older G-Raid not G-Raid2's
I save data to each one so I have a working housing and a backup housing.  Works quite well and I sleep nights.  Only takes seconds to change housings.  I probably shouldn't say this, but I have never had a problem with them.

I think their new G-SAFE will be a better way to go as soon as it is out and running well.
Title: RAID 5 recommendations please
Post by: digitaldog on April 06, 2007, 08:14:13 pm
Quote
Yea, I have been using G-RAIDs for quite some time.  I always have two hooked up to my computer via 400 firewire.  They are all 500 Gb (each housing cotains 2 drives) and are the older G-Raid not G-Raid2's
[a href=\"index.php?act=findpost&pid=111081\"][{POST_SNAPBACK}][/a]

I'm ready to go for the G-Safe but have yet to find out the cost of the swapable drives. So can you swap out the drives from your G-Raids and what's the cost of the shuttles (if that's the right term)?
I can't find anything on their site about cost of additional drives. I'd like to have at least a 2nd drive that is mirrored but in a fire proof safe. Then when the first group fills up, I'd like to be able to add two more.
Title: RAID 5 recommendations please
Post by: Nill Toulme on April 07, 2007, 08:38:59 am
The G-Safe looks to me like any other external 2-fer in a RAID-1 arrary with the signal addition of the words "for photographers" attached to it.  How is it better than any other external RAID-1 or, more importantly, than simple multiple redundant external single firewire (or eSATA) drives?

The problem I have with RAID-1 arrays is that while they might be "fail-proof" they're not really "fool-proof."  Yes if one drive dies you have the other — but any OTHER problem that causes an untoward write or delete to the disk — virus, operator headspace error, you name it — does so, by definition, to both drives simultaneously, and there goes your data.  On the downside, they double your drive cost and physical space requirements.

The way I've analyzed this is this:

The first goal of data protection should be not to lose the data in the first place.  That means good hygiene practices and, it seems to me, RAID-5 for your live data.  RAID-5 means if you lose a drive, you keep chugging along with no restore necessary.  For those of us approaching a TB of data or more, that's important, as a complete restore from external HD can take hours and hours.  (I don't even want to think about a complete restore from optical media!)  RAID-6 would provide a little bit of extra comfort here, but that's a cost-benefit tradeoff that, at least here at the low end of the food chain, comes out better IMO in favor of RAID-5.

After that, you're talking about multiple redundant backups to recover from that catastrophic loss.  At this point, yes a RAID-1 at each of the multiple redundant levels would provide some modest extra comfort over a single drive there, but less so than simply doubling the number of levels, which is also less expensive because those cool little RAID boxes aren't.

In practice, this means my live data (which, happily at this stage, is still ALL my data) resides on a 1.1TB RAID-5 in my main graphics machine.  Downloaded cards get automatically duplicated to an external firewire drive, as they're downloaded, so I have at least two copies from the moment the image leaves the camera.  The RAID-5 gets backed up, nightly and automatically, to two other external firewire drives (Seagate 400GB & 750GB).  Those drives in turn get backed up every few weeks or so to their twins that live off-site (at my next-door neighbor's house).

Perfect?  Of course not.  But it seems to me that I've maximized my data security in the most cost-effective (and time-efficient) way possible.  I'm subject to frequent lapses of judgment and insight though, so if I've missed something please let me know.  (For example, I worry about backing up the backups *from* the backups, rather than from the live data, but I do that without deletes, so the backup-backups are larger than the backups, if you see what I mean.  I think this guards better against the occasional stupid inadvertent delete on my part, which is probably the greatest threat to my data.)

Nill
~~
www.toulme.net (http://www.toulme.net)
Title: RAID 5 recommendations please
Post by: Dave Carter on April 07, 2007, 11:04:08 am
Digital dog
       I do not switch out the drives in my G-RAIDS.  I have purchased them over time, so just use them as complete units. (I know this is probably more expensive - - but ,it sure is easy).

I think Nill has some good points to consider.  Probably a RAID-5 tower is a good way to go.  And to be safe, back it up as well.

I wonder what happened to G-RAID's tower that had five drives in it that were configured to RAID-5?  I just noticed that it is no longer for sale on their web site.  Did thay have problems with it?  I was also thinking about it.  I will try to check with them or VideoGuys next week.
Title: RAID 5 recommendations please
Post by: digitaldog on April 07, 2007, 11:44:02 am
What I'm looking for is a hot swappable system such that I can duplicate my Lightroom library on the fly, having two clones. Then when they fill, I'd like to be able to have one on line, one off (in a fire proof safe). Of course, if I need to update the LR database, I'm out of sync again.

Right now I just use two (or more) sets of external drives and I have to keep syncying them up which is a lot of work (even with a software utility that only updates newer changes).

Of course, having multiple small external drives is useful for working with multiple computer systems. I routinely move an external firewire drive from desktop to laptop.

Nill, I agree with your points. What are you currently using?
Title: RAID 5 recommendations please
Post by: RicAgu on April 07, 2007, 12:22:52 pm
The question for the new Millenium!

I have really battled with this using all sorts of LaCie, Firewire Direct, Weibetech, xRaid and various other drives and housings.

At the end of the day I discovered a great low cost housing from Weibetech for $89.00 that takes firewire 800 and SATA and Sata 500GB drives from Htachi and Seagate for $119.00.  Then I discovered a great little piece of Software called Softraid ( http://www.softraid.com/ (http://www.softraid.com/) ).  I like the software raid because there is no need to upgrade the chipset on the hardware raid housing, technology does not get old in software and need to be sent for an upgrade that will cost you and X amount of money.  Newer technology drives won't work in the housinig, etc.. etc..

LaCie is great on a whim and needing a quick drive to transfer stuff.  My problem with them is they have failed quite regularly and had a lot of Maxtor drives in them which were notorious for failing.  You cannot reuse the housing for anything.  But their customer service adequate and OK at best.

I demand Hitachi, IBM or Seagate drives with any drive I use.  My main concern now with Seagate is that they bought Maxtor.  In the eight years of buying drives.  The only ones that have failed have been Maxtor.  Out of the 15-20 Maxtor drives i have owned all but two have failed.  Not cool!  Luckily it wasn't arm failure that can destroy the platter and I was able to have them salvadged (before I had redundant backs ups).

Firewire Direct SUCKS.  Their customer service sucks, their RAIDs fail all the time.  I have three different housings and they have all failed and the people do not stand behind their product.  They are rude on the phone and full of EGO.

The Apple XRaid is awesome but the price is INSANE.  I sold it and bought another tower and a ton of drives with Weibetech housings.  Took four of my LaCie 250g housings and placed them in the D2 rack and stripped them together to make a 1tb Scratch disk.  Therefore they are not crucial for anything.

Have double Weibetech housings on every 750g and the working 750G RAIDed through Softraid.  Then I bought a simple Weibetech dock/dongle that lets me buy a drive, remove it from its travel crate, plug it into the double array of each drive and SR backs its up in the background.  When it is done I unplug it, put it back in its travel crate and it comes home and sits in a drawer with gaffers tape and a date and name.

Weibetech makes amazing stuff!  Their customer service is second to none.  They fix and deal with stuff the same day and ship it right out and send advance exchanges on some things.  They have upgraded me for free on some things just because I am such a regular customer.  For me there is no better company.

I do hope Apple gets in the game and designs some cool housings and possibly a sled system like what works inside the new MacPro.  But what I have discovered is the new GTech Raid set ups Raid Mini and Raid2.  My only concern is that I can't swap drives on my own if I want to, like the Weibetech housings.

Right now I have three 750 sets going for images. One 750 set for my iTunes and one 750 set for silly iPhoto point and shoot stuff.  That is 15 750 drives and it cost me an eigth of what major raid systems would cost me.  The minute I introduce a new drive to particular drive set Softraid takes care of it.  I can have ten drives backing up if I want.


Just my two cents.


Best of Luck

 
Title: RAID 5 recommendations please
Post by: Nill Toulme on April 07, 2007, 01:51:18 pm
Andrew, I'm bottom-of-the-line all the way.  ;-)  My internal 1.1TB RAID is just four Hitachi 400GB SATA drives running off the onboard nVidia RAID controller built into my ASUS mobo.  I know a dedicated RAID controller would be better/faster, but this is working fine for me now.  I actually had one of the drives die on me; I replaced it and the RAID rebuilt itself with never a hiccup, so it really works.  No downtime, no painful restore!

The external drives are inexpensive Seagate firewire externals, a couple of 400 GB and a pair of 750's, plus one smaller one that gets the automatic backup dumps when Downloader Pro downloads my cards.

I back up nightly, automatically, using FolderClone (whose website seems to be down at the moment), and back up the backups every few weeks manually using the same utility.  (That would ideally of course be done weekly, but that's much easier said than done by a slug like me.)

Nill
~~
www.toulme.net (http://www.toulme.net)
Title: RAID 5 recommendations please
Post by: Nill Toulme on April 07, 2007, 01:53:18 pm
Quote
...  My main concern now with Seagate is that they bought Maxtor.  In the eight years of buying drives.  The only ones that have failed have been Maxtor.  Out of the 15-20 Maxtor drives i have owned all but two have failed.  Not cool!  ...

Ric, I think we all hope that Seagate bought Maxtor solely for the purpose of putting it out of our misery.  ;-)

Nill
~~
www.toulme.net (http://www.toulme.net)
Title: RAID 5 recommendations please
Post by: RicAgu on April 07, 2007, 04:05:10 pm
I hope so!  



Quote
Ric, I think we all hope that Seagate bought Maxtor solely for the purpose of putting it out of our misery.  ;-)

Nill
~~
www.toulme.net (http://www.toulme.net)
[a href=\"index.php?act=findpost&pid=111178\"][{POST_SNAPBACK}][/a]
Title: RAID 5 recommendations please
Post by: digitaldog on April 07, 2007, 04:31:44 pm
Is it possible to have say a four bay system where two are Raid 1 (mirror) and the other two are single (JBOD)? That would be ideal for me.
Title: RAID 5 recommendations please
Post by: RicAgu on April 07, 2007, 04:59:45 pm
Yes,

you could either get it from Weibetech ( weibetech.com )or anyone else who has housings and raid set ups.  The silversata from Weibetech is pretty awesome.  The pre-set it up as you wish before shipping.  Or you can get four drives, a four drive housing system and Softraid.

Below are companies that are amazing.

www.weibetech.com

www.softraid.com

www.granitedigital.com


Quote
Is it possible to have say a four bay system where two are Raid 1 (mirror) and the other two are single (JBOD)? That would be ideal for me.
[a href=\"index.php?act=findpost&pid=111201\"][{POST_SNAPBACK}][/a]
Title: RAID 5 recommendations please
Post by: Ed Foster, Jr. on April 07, 2007, 05:23:33 pm
Quote
Is it possible to have say a four bay system where two are Raid 1 (mirror) and the other two are single (JBOD)? That would be ideal for me.
[{POST_SNAPBACK}][/a] (http://index.php?act=findpost&pid=111201\")

Andrew,
That is exactly the setup I use with a 4 drive SataVault from Coolgear. IMHO, it is an excellent housing with good cooling that has 4 swappable bays that I outfitted it with 4 750 gb SATA Seagate drives. Using Mac's software raid capabilities, two drives are set as a  RAID 1 and the other two are for archiving.  They are connected via a SATA expansion card.  Consequently, the throughput is excellent.  Total cost was less than $1,600.00   See: [a href=\"http://www.cooldrives.com/u6-4s-s2-4-bay-sata-enclosure.html]cooldrives 4 bay SATA Vault[/url]

Regards,
Ed
Title: RAID 5 recommendations please
Post by: digitaldog on April 07, 2007, 05:31:43 pm
Thanks guys. I see that Sonnet technologies seems to support this as well with either a 4 or 5 bay unit and even has the Express Card for the MacBook I'd be using as well. The 5 bay unit runs around $435 on Amazon which isn't expensive (minus drives).

http://www.amazon.com/gp/product/B000F1YJ7...d=1OX2JNK67L6EU (http://www.amazon.com/gp/product/B000F1YJ74/ref=wl_it_dp/103-1073389-3964666?ie=UTF8&coliid=I3O3XA7XD481GQ&colid=1OX2JNK67L6EU)

So aside from not using Maxtor and (according to my wife) Western Digital, what's tops for SATA drives?
Title: RAID 5 recommendations please
Post by: Nill Toulme on April 07, 2007, 06:21:12 pm
MacGurus (http://www.macgurus.com/productpages/sata/SATAEnclosures.php) offers a lot of solutions along these lines.  Any experience with their stuff?

Nill
~~
www.toulme.net (http://www.toulme.net)
Title: RAID 5 recommendations please
Post by: digitaldog on April 07, 2007, 07:01:04 pm
Quote
Thanks guys. I see that Sonnet technologies seems to support this as well with either a 4 or 5 bay unit and even has the Express Card for the MacBook I'd be using as well. The 5 bay unit runs around $435 on Amazon which isn't expensive (minus drives).
[a href=\"index.php?act=findpost&pid=111209\"][{POST_SNAPBACK}][/a]

Ouch, I think I see why its so inexpensive and what might be an issue. The ONLY port out is a SATA cable, no Firewire or USB so I'm not sure if I use one bay as a backup, I can boot from it.

Also, is having the single port a bad option?
Title: RAID 5 recommendations please
Post by: RicAgu on April 07, 2007, 07:12:47 pm
I always like multiple options for connecting.  I had a bad firewire experience years ago and now always want at leaset three connection options.  Firewire 400, 800 (x2) and USB so kind of like a quad connect.  They do make a real quad connect that includes SATA II.

Best of luck
Title: RAID 5 recommendations please
Post by: Dennishh on April 07, 2007, 08:28:01 pm
I’ve been using eSata for about three months now and find it to be the best thing going. I’m on a PC with windows XP, so you guys with Macs should have no problems with this set up. I use two Sonnet Fusion 500p five bay enclosures http://www.sonnettech.com/product/fusion500p.html (http://www.sonnettech.com/product/fusion500p.html) One eSata cable connects five drives in each box. I use two wd raptors at raid 0 and three wd 500 gig drives for a raid 5 array. This box costs $400 the drives are about $150.00. These are all running from a $50 two port card. http://www.usb-ware.com/pci-express-2-port-esata-ii-card.htm (http://www.usb-ware.com/pci-express-2-port-esata-ii-card.htm) The key to this whole set up is the silicon image 3132 chipset. You will need an e-sata card for Mac, I think pc express makes the same card for Mac. I had a problems with the array always having to rebuild and found out the culprit was the type of hard drive. The drives for desktops are not good with these enclosures. The western Digital WD5000ys drives are made for raid external enclosures. I have had no rebuilds at all after starting to use these drives. http://www.newegg.com/Product/Product.asp?...C-AdwordsFeeder (http://www.newegg.com/Product/Product.asp?Item=N82E16822136032&ATT=22-136-032&CMP=KNC-AdwordsFeeder)
This set up is so fast!!!
Dennis
Boston
Title: RAID 5 recommendations please
Post by: nma on April 07, 2007, 09:28:47 pm
Quote
[span style=\'font-size:8pt;line-height:100%\'](if this is the wrong forum to ask this question, please forgive me. I checked the forum descriptions but couldn't find one that directly related to storage equipment. The description for the "Managing Megabytes" forum states "Discussion regarding The Great Luminous Landscape 2006 State-of-The-Art Shootout"!)[/span]

hello all.

I need to back up my photo library (and other computer files). I've had too many incidents with DVDs and they take too long to back up to anyway so I've decided to go with RAID 5.

Preferably, I'd like:
  • A minimum of 3 active drives + 1 spare drive
  • FireWire 800 AND 400. USB not necessary. ie; stand-alone RAID without the need for a RAID card (I'm using a Core Duo 2 iMac and a MacBook Pro).
  • Hotswap (albeit optional because I'm not running a business. I can shut the RAID down if I need to.)
  • Compatible with commonly available 3.5" drives (eg, Barracudas from Seagate because I can get 5-year warranties when I buy these locally here in Hong Kong).
I don't want OEM drives because they only offer 1 year warranty.
I don't want Lacie because I've had too many problems with their equipment.

Any recommendations?





kind regards,
Gregory
[a href=\"index.php?act=findpost&pid=110970\"][{POST_SNAPBACK}][/a]

Dear All,

We all face serious difficulties in safegaurding our photo data, but Raid 5 is not a backup solution or archival. Professionally, we find them to be convenient, but unreliable. They only provide recovery if a single drive fails. Other failures modes causing data loss cannot be recovered. And this statement is based on experience, not theory. Examples include problems with the raid controller that corrupt the array as well as problems with the power supplies.  

Better solutions involve multiple disk copies,  tape backup, mirrored raid 5, etc.  No backup strategy should be relied on unless it is tested.  Simple is best.  Things are changing very quickly. No one actually knows how long these elctromagnetic devices will work. Can we be sure that a drive left on the shelf for two years will function reliably? I don't think so.  

Soultions involving backup on AIT-tape are rated at ~ 20 years in controlled environements.  There is much more expereince with tape. It is not very convenient nor cheap, but it works.  Still, there are pitfalls with tape, as well: Proprietary recovery sofware may  cease to be supported. Tapes have to be rewritten as they age.  As you change computers or operating systems you have to ensure that software and drivers exist.  


All this means that one must think carefully and not be impulsive or overconfident.

'Hope this helps.
Title: RAID 5 recommendations please
Post by: lankford on April 08, 2007, 10:42:36 pm
I am very pleased with the wiebetech RAIDTech 800, has usb 2.0, firewire 400 and 800.  I am using mirrored setup with two 750 gig and transfer times are fast enough for me. For another assurance I use a western digital 500 gig external drive that you can get at best buy, etc.  (pics in 3 places)
Title: RAID 5 recommendations please
Post by: Ed Foster, Jr. on April 09, 2007, 10:31:30 am
Quote
Ouch, I think I see why its so inexpensive and what might be an issue. The ONLY port out is a SATA cable, no Firewire or USB so I'm not sure if I use one bay as a backup, I can boot from it.

Also, is having the single port a bad option?
[a href=\"index.php?act=findpost&pid=111230\"][{POST_SNAPBACK}][/a]
While one might want the option for multiple ports for other needs, when I researched this and set up mine last year I found the following:  SATA connections provides the fastest throughput over USB or FW 400 and 800; external USB and FW drives proved unreliable (for me) and, at that time, you could not boot from the externals utilizing USB or FW connections.  You can boot from your external SATA connected drives because, in essence, they are like your internals in terms of protocol.   As for drives, WD's & Maxtors I have had problems with over the years, but Seagates have been reliable for me, so I chose Seagate's Barracuda 7200.10 series.  Today you can find them at less than $150.00 for 500 gb and 750 gb for around $250.0.

Ed
Title: RAID 5 recommendations please
Post by: cmburns on April 09, 2007, 12:21:45 pm
My personal experience is you're increasing your chances of losing data by going to Raid setup. In my case I had a Raid 5+1 array. It was run from a dedicated Raid controller card and it had an onboard battery pack that went dead and corrupted the array. Of course I thought it was 20 other things until I figured out that was the problem.(I had to buy another controller card and when it worked I realized it was the original card, until then it looked like so many other things were the problem) I also never got it to work as I hoped, where I would have 3 sets of drives, 2 active, and one out of the computer as a backup. I hoped to be able to pull out one set every week, put the other one in and have it autoupdate the older array. That just never worked right. You also have a lot more noise and heat. Make sure you have a helluva case with lots of fans, those hard drives will get hot hot especially all packed together. My setup was fairly robust corporate stuff and when it failed I was left not trusting raid.

Always go with the KISS method, keep it simple stupid. Raid is not simple. There's way more that can go wrong and you're lured in with this false sense of security. "I'll be ok if a drive or even two fail." Yeah but what if your power fails, do you have a backup that will run the computer for a while. FYI I did, but still the onboard battery pack got me. What if you're not there when the power fails and you don't get it shut down before your backup runs out, etc., just a lot that can go wrong. To me most consumer raid stuff today is made to access your data fast, not as any kind of backup solution. It's for video editing where you need to keep a ton of data on the computer, have fast scratch disks etc, but you can bet that data is also backed up safely somewhere else.
KISS method for me means backing up to an external hard drive. As another poster said, esata is where it's at now for speed, but USB2.0 is much more universal. I can take the USB enclosure to any computer I have and copy to or from. I never bother screwing the enclosure together, because i'm always swapping hard drive in and out of it.
I also burn 2 sets of DVD's and store in 2 seperate places, One offsite. Thus far my worse case has been a hard drive failure, and out of 50 or so DVD's that was used to restore it, 2 had a handful of unreadable files on them. The 2nd set of dvd's read those files just fine and I then remade the bad dvd's. The point is, all of this stuff fails so have multiple redundancy.

I organize my stuff in a folder by date so it might say "2007-04-08 Easter at so and so's.", I then run a dos command (in windows click start, then run, then type in "cmd" to get a dos prompt. Then d: to get to the root directory of the D drive for instance.)
dir /B *.* > c:\list.txt
 that spits out all the folders and files in the root directory of whatever drive you're on to a text file in the root of the c drive. If you want all the file names in the subdirectories, basically everything on the disk, add a /S to that.

I then cut and paste that into a master text file. I put the hard drive into the little static proof bag it came in, and put a sticker on the outside with a number. So then I can look at my text file(or better yet search through it) and see that hard drive so and so has this or that shoot on it. Get it out, plug into enclosure and have the files copied back to the main computer in minutes.

I've read several advice things that say it's good to do a scandisk on your hard drives, say every 6 months. The longer they sit not being used the more likely they'll get errors. They have error correction built into them so when you run the scandisk the hard drives will find tiny errors and self correct them using it's parity data.

The problem I have is when a hard drive fails, it's really getting to be a chore to back up from dvd's. A 500gb drive means you're going to be spending a lot of time copying the contents of 100+ dvd's over. Luckily I have 4 computers around the house with dvd drives, but that's still a day of poking dvd's in and out and then copying to a replacement drive in an enclosure. Hopefully another year or so and we'll have Blu Ray or HD DVD writable disks that will hold a lot more data.
Title: RAID 5 recommendations please
Post by: Nill Toulme on April 09, 2007, 01:07:06 pm
That's a fairly anecdotal reason for dumping RAID.  Sort of like "I had a car once, and it broke.  You'll never see me in one of those things again."

As for backing up to DVD's or other optical media, I have to say I just don't get it.  Backing up is bad enough, but what are you going to do if you ever have to do a complete restore?  You can get a nice 500GB Seagate USB2 external HD (http://www.newegg.com/Product/Product.aspx?Item=N82E16822148172) for 150 bucks; for 300 clams you can have set-it and-forget-it dual redundant backups of your entire data set.

What is the attraction of DVD backup?  What am I missing?

Nill
~~
www.toulme.net (http://www.toulme.net)
Title: RAID 5 recommendations please
Post by: cmburns on April 09, 2007, 02:16:44 pm
For me the plusses of the dvd backup is longevity and cheapness. Hard drives are not made to last. They're made to be obsolete and replaced by bigger hard drives in a few years. Buy quality DVD's and you'll be ok for a much longer time period than hard drives and that's if you use good hard drives. Do you? I know for DVD's buy Taiyo Yuden and forget it. For hard drives you have to wait for a drive to be out for a while then do some digging to find reviews and failure rates etc. Maybe a Seagate so and so .9 is great and then the newer .10 version is crap.(I just ran into this as I saw a drive at a fantastic price, the drive has horrible reviews though so I payed a little more for a more reliable model)

Take a look at this
http://labs.google.com/papers/disk_failures.pdf (http://labs.google.com/papers/disk_failures.pdf)

Their measured baseline failure rate is about 8% after 2 years and that's across all models. Buy a bad model and it will be much higher. Also note that drives with little use fail at a higher rate than those that see regular use.
Cost. you can buy hard drives for around 25 cents/gb, so a terrabyte of storage will set you back $250. No biggie in this line of work, but the same terrabtyte of DVD's would be about 1/3 as much, they just take a lot more of your time.

As for my raid failure being anecdotal, of course. But it was supposed to be perfect. I did tons of research, bought the best stuff, scsi drives, a stud raid controller card, power backup, etc. and it get's taken out by a failing battery. There's just too much that can go wrong with it. Nothing is bullet proof, just do multiple backups and hope you don't have acts of God at your place and your offsite storage as well.

Raid is just not a good backup solution. Use it as a huge volume for Lightroom, ok, just have the whole thing backed up somewhere else.

Oh another consideration, what if your motherboard fails after a year or two? This also has happened to me, simply a capacitor going bad, they buy the cheapest ones possible to keep cost down. How will you get your data off your array? Will there still be that model of motherboard to buy? Does the same company have a newer motherboard with raid controller that is backwards compatible etc., maybe buy 2 motherboards. When my raid controller failed I was lucky to find a used one on Ebay. Computer stuff goes obsolete so fast.
Title: RAID 5 recommendations please
Post by: Nill Toulme on April 09, 2007, 02:49:30 pm
Quote
...Raid is just not a good backup solution. ...
RAID is not a backup solution, period.  RAID has many advantages, notably among them for purposes of the present discussion avoiding drive failure in the first place, thus avoiding downtime and the need for painful restores.  As I mentioned above, I've had a drive fail in my RAID.  I simply replaced it and let the array rebuild itself, meanwhile carrying on business as usual.  Yes RAID controllers fail.  Motherboards fail, everything fails.  It's just a matter of playing the odds (i.e., the odds that a drive will fail being MUCH higher than that a controller will fail), and being prepared with backups when you lose.

Quote
...Oh another consideration, what if your motherboard fails after a year or two? This also has happened to me, simply a capacitor going bad, they buy the cheapest ones possible to keep cost down. How will you get your data off your array? ...
That's what your backups are for.  And when you hope you've got one on something other than DVD's.

Nill
~~
www.toulme.net (http://www.toulme.net)
Title: RAID 5 recommendations please
Post by: Nill Toulme on April 09, 2007, 02:55:03 pm
p.s.

Quote
For me the plusses of the dvd backup is longevity and cheapness. ... Buy quality DVD's and you'll be ok for a much longer time period than hard drives and that's if you use good hard drives. Do you? I know for DVD's buy Taiyo Yuden and forget it. ...
That remains to be seen, does it not?  You're not relying on one copy of each of those DVD's, are you?  What's your regimen for checking every one of them periodically to see if they still work?

Nill
~~
www.toulme.net (http://www.toulme.net)
Title: RAID 5 recommendations please
Post by: digitaldog on April 09, 2007, 02:56:24 pm
Quote
RAID is not a backup solution, period. ~~
www.toulme.net (http://www.toulme.net)
[a href=\"index.php?act=findpost&pid=111520\"][{POST_SNAPBACK}][/a]

Amen! I didn't think it needed mentioning but not a bad idea. My interest in a RAID isn't to forgo backing up (I'm super anal about that) but rather to have one less area to worry about. Plus, once my RAID bucket fills (using Peter Krogh's term), I'd like one drive for offsite back up and one on site just to access the files if necessary. IOW, I have to constantly do backups to two sets of drives anyway which is real time consuming. Three copies is even better (and more work).

I've been on the Mac since 1988 and I guess I'm lucky but I've never had a drive fail. Doesn't stop me from backing up quite often. Maybe that's the key <g>.

Ideally my goal is to have two drives (mirror) the current Lightroom Library I'm working on. So dual buckets. Once filled, one goes offsite or in a media safe, the other can be popped into an empty shuttle in case I want to pull off a file or print something. That's why I'd really like a four or five bay system. Two for the current Raid 1, one maybe for the system back up and one to pop drives into just to access preexisting data.
Title: RAID 5 recommendations please
Post by: nma on April 09, 2007, 02:59:32 pm
Quote
That's a fairly anecdotal reason for dumping RAID.  Sort of like "I had a car once, and it broke.  You'll never see me in one of those things again."

As for backing up to DVD's or other optical media, I have to say I just don't get it.  Backing up is bad enough, but what are you going to do if you ever have to do a complete restore?  You can get a nice 500GB Seagate USB2 external HD (http://www.newegg.com/Product/Product.aspx?Item=N82E16822148172) for 150 bucks; for 300 clams you can have set-it and-forget-it dual redundant backups of your entire data set.

What is the attraction of DVD backup?  What am I missing?

Nill
~~
www.toulme.net (http://www.toulme.net)
[a href=\"index.php?act=findpost&pid=111501\"][{POST_SNAPBACK}][/a]

Nill,
cmburns provided more than anecdotal advice. Even mirrored RAID 5 is not a sufficient archival solution. It is possible to have failures that corrupt both arrays.  As part of my professional work we have been using RAID 5 and NAS for about 10 years.  This equipment is supposed to be more reliable than ordinary consumer solutions. Single drive failures have not been a big problem. The gotchas we have encountered included failures of the RAID controllers, causing disk array corruption and failure of the dual power supplies to survive an electrical spike, again causing corruption of the disk array.  Once the disk array is corrupted, the solution is to reformat. The rule of thumb is to replace RAID 5 after about 3 years. As noted by cmburns, if it becomes necessary to replace the computer and or RAID controller all bets are off.  Three years in the computer business is a long time. Parts become unavailable, companies go out of business, etc.  I can offer no fool proof solutions.
As I wrote earlier, tape back up has a longer history and thus we know it can outlast disk storage. DVD as an archival medium is not well established and the capacity is small.  Finally, no backup solution can be relied on unless you have demonstrated that you can recover your data.  There have been more than one data center  who was embarrassed to find that their back up was no good.

I hope this is food for thought.
Title: RAID 5 recommendations please
Post by: Nill Toulme on April 09, 2007, 03:09:05 pm
Andrew I'm probably stating the obvious (again...), but be thoughtful about a "mirror" solution just as you are with RAID.  A true mirror will still subject you to all the other possible causes of data loss, other than drive failure, just as a simple RAID will.  IOW, mirroring is not truly a backup solution either.

Nill
~~
www.toulme.net (http://www.toulme.net)
Title: RAID 5 recommendations please
Post by: digitaldog on April 09, 2007, 03:12:48 pm
Quote
Andrew I'm probably stating the obvious (again...), but be thoughtful about a "mirror" solution just as you are with RAID.  A true mirror will still subject you to all the other possible causes of data loss, other than drive failure, just as a simple RAID will.  IOW, mirroring is not truly a backup solution either.

Nill
~~
www.toulme.net (http://www.toulme.net)
[{POST_SNAPBACK}][/a] (http://index.php?act=findpost&pid=111532\")

Right. I will of course continue to back up. Old habits die hard.

As to DVD's, I got this email from my bud and tech editor Karl Lang awhile back and the product does look useful:

Quote
[a href=\"http://www.truedisc.com/]http://www.truedisc.com/[/url]

Hey guys,

This was just released. The marketing hype is too much to take, 
however the basic idea is really, really great.

Record a standard 9660 ISO CD/DVD. Don't fill it all the way up leave 
some room.

Use the free space to store redundant checksum data. Then if 
something happens to the disk use their recovery software to re-
extract the file data with the redundant data. Up to 90% of the 
original could be damaged and still recover the file.

If this becomes a standard format we can all sleep better at night. I 
think I may buy it just to support the idea. If I was Adobe or Apple, 
I would buy this company right now. It's such a simple and easy idea, 
at the OS level it would be awesome.

Karl

My only beef with DVD is it's not large enough.
Title: RAID 5 recommendations please
Post by: Nill Toulme on April 09, 2007, 03:13:24 pm
Quote
Nill,
cmburns provided more than anecdotal advice. Even mirrored RAID 5 is not a sufficient archival solution. It is possible to have failures that corrupt both arrays.  ...[{POST_SNAPBACK}][/a] (http://index.php?act=findpost&pid=111527\")
Exactly.  I suspect you were typing this as I was typing my "RAID is not a backup solution, period" post above.  What I was really taking exception to was cmburns's statement that "...you're increasing your chances of losing data by going to Raid setup."  I think that's simply wrong (well, unless you're talking about RAID 0... ).

Nill
~~
[a href=\"http://www.toulme.net]www.toulme.net[/url]
Title: RAID 5 recommendations please
Post by: Nill Toulme on April 09, 2007, 03:17:43 pm
Quote
...My only beef with DVD is it's not large enough.
[{POST_SNAPBACK}][/a] (http://index.php?act=findpost&pid=111533\")
Right... even with checksum data. ;-)  

It's feeding the dadburn things into the drive that's so daunting, especially if you contemplate the restore of a large amount of data.  Remember LP changers?  If you could just stick 500 of the little suckers into some sort of auto loader, I suppose I might be moderately interested in them.

Nill
~~
[a href=\"http://www.toulme.net]www.toulme.net[/url]