Luminous Landscape Forum

Equipment & Techniques => Medium Format / Film / Digital Backs – and Large Sensor Photography => Topic started by: amsp on October 14, 2007, 11:35:05 am

Title: Hard earned advice against RAID...
Post by: amsp on October 14, 2007, 11:35:05 am
Well, it's official, I just lost most of my work   About half a year ago I bought a hardware RAID solution with two hot-swappable drives, thinking that my worrying days were over. Instead the very solution that was supposed to protect my files ended up destroying them. It started when one of the drives failed, with a "bad block" error. I googled it and found it would resolve it self with a zero-out format. After formatting the drive I inserted it into the RAID casing and to my horror I see it starting to copy the empty drive to the drive with my files. I try to abort it but it was too late. Desperate I tried using data recovery software only to find that it is total bs, nothing was retrieved.

So, a word of advice, stay clear of RAID and just back up your files on two drives your self. And never completely trust any hardware, however much it's supposed to "protect" your files.

P.s. Please refrain from any hindsight advice, I'm depressed enough as it is. Thanks.
Title: Hard earned advice against RAID...
Post by: jonstewart on October 14, 2007, 11:49:04 am
Quote
Well, it's official, I just lost most of my work   About half a year ago I bought a hardware RAID solution with two hot-swappable drives, thinking that my worrying days were over. Instead the very solution that was supposed to protect my files ended up destroying them. It started when one of the drives failed, with a "bad block" error. I googled it and found it would resolve it self with a zero-out format. After formatting the drive I inserted it into the RAID casing and to my horror I see it starting to copy the empty drive to the drive with my files. I try to abort it but it was too late. Desperate I tried using data recovery software only to find that it is total bs, nothing was retrieved.

So, a word of advice, stay clear of RAID and just back up your files on two drives your self. And never completely trust any hardware, however much it's supposed to "protect" your files.

P.s. Please refrain from any hindsight advice, I'm depressed enough as it is. Thanks.
[a href=\"index.php?act=findpost&pid=145881\"][{POST_SNAPBACK}][/a]

This following is not hindsight advice for you, but a suggestion to others who don't want what has happened to you, to happen to them. Otherwise, there's be no point in your post, if it's not a learning experience for all.....I would say most feel your pain and loss!

But you should also have been doing a weekly (if not daily) backup to external drive, or better, tape, ideally using software designed for the purpose. If not, then a manual copy (although this allows little of no version control).

Your real mistake was in how you handled things when the drive failed. You should have left the failed drive out, and ran on the good one with the express purpose of backing up the whole system. Then when it's backed up, you start recovering. I think your data was lost because you tried to rebuild the 'wrong way around'...sometimes the 'good' disk needs to be reset in the raid bios to be the master before recovering.

I can never understand why people (generally) are so trusting of goods that are typically now made down to a cost level, rather than up to a quality level.

As for me, I have raid on the server, and backups weekly, negs for all projects are stored on dvd for short term recovery, should something happen. Thats a total of 5 copies at any one time. (Yeah, and I'm not paranoid!)
Title: Hard earned advice against RAID...
Post by: EricWHiss on October 14, 2007, 12:11:05 pm
I am going to back up tonight!  

Sorry to hear about your data loss!  

You didn't tell us what RAID setting you had.  As far as I know RAID boxes only allow a drive to go down and swap it out with only two drives if it is set to mirror mode.  There's like 20 RAID configurations....If you have 3 or more drives then you can swap out a drive in some of the other configurations.

Even then your data is not safe. You need to keep a backup in a second physical location.
Title: Hard earned advice against RAID...
Post by: Graham Mitchell on October 14, 2007, 12:13:59 pm
I've had a drive in my RAID go down. I just replaced the drive and rebuilt the array. Easy.

Sorry to hear you had problems but perhaps it was user error?

As someone already pointed out, all projects should be backed up onto DVD separately and preferably stored in a different location to protect you against fire, theft, etc.
Title: Hard earned advice against RAID...
Post by: Gary Ferguson on October 14, 2007, 12:17:59 pm
Quote
Well, it's official, I just lost most of my work   
[a href=\"index.php?act=findpost&pid=145881\"][{POST_SNAPBACK}][/a]

I'm signing off now to spend the rest of the day burning DVD's.
Title: Hard earned advice against RAID...
Post by: feppe on October 14, 2007, 12:47:17 pm
Quote
So, a word of advice, stay clear of RAID and just back up your files on two drives your self. And never completely trust any hardware, however much it's supposed to "protect" your files.
[a href=\"index.php?act=findpost&pid=145881\"][{POST_SNAPBACK}][/a]

Sound advice. I'd like to point out never trust any single piece of hardware: backup up to separate devices and preferably locations (off-site backups).

But most importantly: RAID is not meant or suitable for backup; it's meant for maximum uptime.
Title: Hard earned advice against RAID...
Post by: Monito on October 14, 2007, 01:01:30 pm
Quote
I'm signing off now to spend the rest of the day burning DVD's. [a href=\"index.php?act=findpost&pid=145895\"][{POST_SNAPBACK}][/a]

Right idea, wrong approach.  You are right, burning DVDs is labor intensive.

Copy your whole disk to an external disk with Acronis or similar disk imaging software, then turn it off and keep it off almost all the time (to extend life).  Then copy it again to a second external disk and put that in your safety deposit box or offsite somewhere, in case of fire or burglary.  I read a report that Francis Ford Coppola's main drive with 15 years work and his backup drive sitting beside it were stolen recently.  If you like you can additionally copy key files like password lists, addresses and your 200 best Raw files to a DVD.
Title: Hard earned advice against RAID...
Post by: Steve Kerman on October 14, 2007, 01:11:23 pm
Quote
Copy your whole disk to an external disk with Acronis or similar disk imaging software,...
[a href=\"index.php?act=findpost&pid=145900\"][{POST_SNAPBACK}][/a]

I very strongly recommend EMC Retrospect backup software.  I've had many backup systems over the years, and this is the only package I've ever used that makes incremental backups that are reasonable to use if you every need to restore.
Title: Hard earned advice against RAID...
Post by: feppe on October 14, 2007, 01:14:45 pm
Quote
Copy your whole disk to an external disk with Acronis or similar disk imaging software, then turn it off and keep it off almost all the time (to extend life).  Then copy it again to a second external disk and put that in your safety deposit box or offsite somewhere, in case of fire or burglary.
[a href=\"index.php?act=findpost&pid=145900\"][{POST_SNAPBACK}][/a]

HDDs are meant and designed to be used, not idled, and having them idle is not a good idea. Studies suggest that HDDs usually fail in the first few months of use. If you idle your backup HDDs you will always use it within this failure-prone period, thus reducing the reliability of your backups. Anecdotally, I have HDDs in my box right now which have been on 24/365 in constant, hard swapping use for years. I've read and heard several stories about people going back to their offsite HDDs after months of disuse, only for them to never wake up again.

While offsite backups is an advisable strategy, keep the above in mind.
Title: Hard earned advice against RAID...
Post by: jonstewart on October 14, 2007, 01:15:47 pm
Quote
Right idea, wrong approach.  You are right, burning DVDs is labor intensive.

Copy your whole disk to an external disk with Acronis or similar disk imaging software, then turn it off and keep it off almost all the time (to extend life).  Then copy it again to a second external disk and put that in your safety deposit box or offsite somewhere, in case of fire or burglary.  I read a report that Francis Ford Coppola's main drive with 15 years work and his backup drive sitting beside it were stolen recently.  If you like you can additionally copy key files like password lists, addresses and your 200 best Raw files to a DVD.
[a href=\"index.php?act=findpost&pid=145900\"][{POST_SNAPBACK}][/a]

Turning on and off a drive is not going to extend it's life. It's more likely to cause a problem, since the maximum load on the motor is when it's spinning up.

Agree with offsite (or secure) backup, but I failed to mention that DVD's should ONLY be used for short term storage and not long term. For long term, tape is arguable the best, and most reliable solution.

When you consider the value of the work that is stored and the cost of the equipment we use, I cannot understand why professional photographers do not have a file server separate from their workstation / laptop. Have raid in the server, and if you really want, create a local synced copy of all or specific work. You then have automatically 3 copies, on two different computers. Add tape / external hard drive as 4th copy, and dvd for short term backup as 5th copy (...and I always write two copies of the dvd...thus 6 copies in total).

Do it this way, and you have the maximum backup for the minimum investment in time... Your time is worth something isn't it? Don't spend any more of it doing stuff that doesn't (directly) make money.
Title: Hard earned advice against RAID...
Post by: mattlap2 on October 14, 2007, 01:34:05 pm
Quote
Well, it's official, I just lost most of my work   About half a year ago I bought a hardware RAID solution with two hot-swappable drives, thinking that my worrying days were over. Instead the very solution that was supposed to protect my files ended up destroying them. It started when one of the drives failed, with a "bad block" error. I googled it and found it would resolve it self with a zero-out format. After formatting the drive I inserted it into the RAID casing and to my horror I see it starting to copy the empty drive to the drive with my files. I try to abort it but it was too late. Desperate I tried using data recovery software only to find that it is total bs, nothing was retrieved.

So, a word of advice, stay clear of RAID and just back up your files on two drives your self. And never completely trust any hardware, however much it's supposed to "protect" your files.

P.s. Please refrain from any hindsight advice, I'm depressed enough as it is. Thanks.
[a href=\"index.php?act=findpost&pid=145881\"][{POST_SNAPBACK}][/a]

I would also urge you to quickly search out a professional recovery service.   There are many companies that specialize in this sort of work and have the ability to do many things you do not.

It will probably cost you quite a bit ... bit put a value on how much that lost work is worth to you.
Title: Hard earned advice against RAID...
Post by: jing q on October 14, 2007, 01:50:42 pm
I had a 1tb external hard drive that was on RAID 0...and one drive had problems and to my horror the data on the 2nd drive disappeared too. since then I've gone back to simply having single drives and having backups of those single drives.

I'll stay with being simple in such situations.
Title: Hard earned advice against RAID...
Post by: feppe on October 14, 2007, 02:11:55 pm
Quote
I had a 1tb external hard drive that was on RAID 0...and one drive had problems and to my horror the data on the 2nd drive disappeared too. since then I've gone back to simply having single drives and having backups of those single drives.

I'll stay with being simple in such situations.
[a href=\"index.php?act=findpost&pid=145915\"][{POST_SNAPBACK}][/a]

That's a perfect example of applying a totally incorrect tool for backup, and/or not understanding the tool you're using. RAID 0 is striped, ie. there's no redundancy whatsoever. This means the RAIDed drive is twice as likely to fail than a "normal" drive - not exactly a backup-worthy choice. The data on the 2nd drive didn't disappear, it was still there, but due to striping it's next to impossible to recover in any meaningful way. It should only be used for swapping, as you will lose all your data if only one of the drives fail - unless you want to shell big euros for data recovery services.

--

I'm quite surprised to hear these RAID horror stories pop up every month or so. Don't salespeople educate their customers as to what RAID is supposed to be used - again, NOT for backup -, or don't people read up on the products they use for backup? I bet most of us spend hours and hours reading reviews about our next lens selection. If people spent even a fraction of that time learning about proper backup technologies, we'd have less of these unfortunate stories.
Title: Hard earned advice against RAID...
Post by: jing q on October 14, 2007, 02:15:11 pm
Quote
That's a perfect example of applying a totally incorrect tool for backup, and/or not understanding the tool you're using. RAID 0 is striped, ie. there's no redundancy whatsoever. This means the RAIDed drive is twice as likely to fail than a "normal" drive - not exactly a backup-worthy choice. The data on the 2nd drive didn't disappear, it was still there, but due to striping it's next to impossible to recover in any meaningful way. It should only be used for swapping, as you will lose all your data if only one of the drives fail - unless you want to shell big euros for data recovery services.

--

I'm quite surprised to hear these RAID horror stories pop up every month or so. Don't salespeople educate their customers as to what RAID is supposed to be used - again, NOT for backup -, or don't people read up on the products they use for backup? I bet most of us spend hours and hours reading reviews about our next lens selection. If people spent even a fraction of that time learning about proper backup technologies, we'd have less of these unfortunate stories.
[a href=\"index.php?act=findpost&pid=145923\"][{POST_SNAPBACK}][/a]

yup of course I understand now.
But tell that to me when all I was looking for was another external backup hard drive and ended up being given a RAID device.
Yes salesmen don't give good advice alot of times.
Yes unfortunately I rather read about photography then backup technology...heh
Title: Hard earned advice against RAID...
Post by: digitaldog on October 14, 2007, 02:58:24 pm
Right now I'm using a Raid 1 (mirror) for my LR libraries, but have 2 clones on 'standard' external FireWire drives. One I use on location, then back that up to the Raid when I return. Can someone alert me to why this system isn't pretty safe? At any one time, I have at last 3 clones (counting the Mirror as two) and a forth that is a clone plus the new stuff. Seems pretty rock solid.
Title: Hard earned advice against RAID...
Post by: feppe on October 14, 2007, 03:27:42 pm
Quote
Right now I'm using a Raid 1 (mirror) for my LR libraries, but have 2 clones on 'standard' external FireWire drives. One I use on location, then back that up to the Raid when I return. Can someone alert me to why this system isn't pretty safe? At any one time, I have at last 3 clones (counting the Mirror as two) and a forth that is a clone plus the new stuff. Seems pretty rock solid.
[a href=\"index.php?act=findpost&pid=145936\"][{POST_SNAPBACK}][/a]

I'm no expert, but sounds pretty good to me. One thing I'd do is to actually do a test recovery of all your data. This is especially true if you use backup software or hardware, or zip them up, instead of plain old mirroring.

User error is quite common - probably more common than hardware failure -, so knowing how to recover in case of such failure is worthwhile. Backing up the wrong data, overwriting to an incorrect drive, overwriting old backups with corrupted files while backing up, or recovering the wrong data is just as catastrophic as a failed HDD.
Title: Hard earned advice against RAID...
Post by: digitaldog on October 14, 2007, 03:32:39 pm
Quote
I'm no expert, but sounds pretty good to me. One thing I'd do is to actually do a test recovery of all your data. This is especially true if you use backup software or hardware, or zip them up, instead of plain old mirroring.

If I'm cloning, I use SuperDuper. It only backs up changes so its pretty fast. Its been a life saver in the past when I've had to reclone say an entire boot drive so I'm pretty confident in the product.

If I want to sync folders (not clone the entire enchilada) I use Chronosync which seems to do the job.

I'd agree however that user error is probably the biggest issue here (cloning the wrong drive).
Title: Hard earned advice against RAID...
Post by: Steve Kerman on October 14, 2007, 03:33:38 pm
Quote
Right now I'm using a Raid 1 (mirror) for my LR libraries, but have 2 clones on 'standard' external FireWire drives. One I use on location, then back that up to the Raid when I return. Can someone alert me to why this system isn't pretty safe? At any one time, I have at last 3 clones (counting the Mirror as two) and a forth that is a clone plus the new stuff. Seems pretty rock solid.
[a href=\"index.php?act=findpost&pid=145936\"][{POST_SNAPBACK}][/a]
I see two big holes: 1) The house (or studio) burns down, taking everything with it.  2) Burglers take the computer and all the backups.

An off-site backup is generally considered essential for a really secure backup plan.
Title: Hard earned advice against RAID...
Post by: digitaldog on October 14, 2007, 03:36:56 pm
Quote
I see two big holes: 1) The house (or studio) burns down, taking everything with it.  2) Burglers take the computer and all the backups.

An off-site backup is generally considered essential for a really secure backup plan.
[a href=\"index.php?act=findpost&pid=145949\"][{POST_SNAPBACK}][/a]

I keep one drive in a Fire Proof safe (in the home). Its actually unlocked! I'd rather have the bad guys take the stuff inside that they think is valuable (the wife's jewelry which ain't that valuable) and hopefully leave the media alone. But then with three dogs, they have to GET to the safe.

In the end, the images have no value to anyone but me. I don't make them any more for money. But I'm still paranoid about keeping them safe and sound.

I'd agree however, an off site copy would be advisable.
Title: Hard earned advice against RAID...
Post by: Monito on October 14, 2007, 04:01:33 pm
Quote
HDDs are meant and designed to be used, not idled, and having them idle is not a good idea. Studies suggest that HDDs usually fail in the first few months of use. If you idle your backup HDDs you will always use it within this failure-prone period, thus reducing the reliability of your backups. Anecdotally, I have HDDs in my box right now which have been on 24/365 in constant, hard swapping use for years. I've read and heard several stories about people going back to their offsite HDDs after months of disuse, only for them to never wake up again. [a href=\"index.php?act=findpost&pid=145905\"][{POST_SNAPBACK}][/a]
There is some truth to what you say, but external drives are a different issue from internal drives.  Internal drives are cooled by the computer systems cooling system (fans).  External drives have only passive cooling, are mounted vertically, and experience has shown that they are not as durable as internal drives when kept on 24/7.

Note that I advised double backups (two external drives, one of them offsite), and a third backup (DVD) for only the most valuable data and images.

I keep my internal drives on 24/7, my main external drive goes on only when I download photos from the memory card (automatically backing up via Alan Light's DIM program).  My secondary backup disk is turned on even less often to copy disk images.
Title: Hard earned advice against RAID...
Post by: MarcRochkind on October 14, 2007, 04:36:58 pm
Quote
But most importantly: RAID is not meant or suitable for backup; it's meant for maximum uptime.
[a href=\"index.php?act=findpost&pid=145896\"][{POST_SNAPBACK}][/a]
Everyone should pay a lot of attention to what feppe said. RAID 0, which is not even R.A.I.D., is for performance (it reduces, rather than increases, reliability). The others are to make recovery faster in the event of hard drive failure.

None of them protect against most of the causes of data loss, which can be grouped into these categories: hardware or software failure (other than single drive), electrical surge, human error, theft, physical destruction (fire, explosion), and area-wide disasters (flood, earthquake, bombing). To protect against some of these, one or more locally-stored backups will work. For physical destruction and area-wide disasters, you need physical separation (e.g., data stored online, at a neighbor's house, across town).

It's not only making the copies that matters, but where those copies are stored.

--Marc
ImageIngester.com
Title: Hard earned advice against RAID...
Post by: andybuk99 on October 14, 2007, 05:04:05 pm
Quote
There is some truth to what you say, but external drives are a different issue from internal drives.  Internal drives are cooled by the computer systems cooling system (fans).  External drives have only passive cooling, are mounted vertically, and experience has shown that they are not as durable as internal drives when kept on 24/7.
[a href=\"index.php?act=findpost&pid=145957\"][{POST_SNAPBACK}][/a]

That is just not true. Internal drives get next to no cooling from a standard pc fan setup, therefore an external drive within a normal enclosure would be no better or worse than an internal drive.

External drives are not all mounted vertically, in fact most if not all are designed to lay vertical OR horizontal. Any way why would that make any difference to the drives stability?

Oh and the majority of internal/external drives are exactly the same just in different enclosures.
Title: Hard earned advice against RAID...
Post by: Monito on October 14, 2007, 06:01:17 pm
Different configurations result in different wear on bearings and different gravitational effects on heads.  Warranties on external drive units (incl. enclosure) tend to be shorter than internals.  Further, my standard issue PC has two quiet fans, one of which draws air directly through the drive bay.

However, the important thing is to make backups, and backups to external hard drives are the most efficient and flexible, especially when compared to labor intensive DVDs (loading, unloading, stacking, labelling, sorting).  Any drive might fail at any time, running or not, so reliability and MTBF are value issues, not preventative or prophylactic.

Another reason for keeping an external drive off-line is in case of viruses or malware or trojans doing a nasty on the disk.  Once you take it offline (disconnect the USB), it is up to you whether you turn it off or not, but many external drive units are powered via the USB, or have automatic shutdown.

Nothing said here convinces me to alter the way I run: internal drives on 24/7, double backups to external drives (one for offsite), external on only when needed, single backup DVD disk for only the most important data and images, duplicate backup DVD disk for offsite.
Title: Hard earned advice against RAID...
Post by: ericstaud on October 14, 2007, 07:10:41 pm
Miss-labeled thread really.  After reading advice all over the internet that RAID IS NOT A BACKUP, the title should be "Hard earned advice against working without backups".

Raid 5 is great for:
-Creating one large volume instead of several small ones.
-Providing more up-time during drive failures (to keep a web-store alive for example).
-Increasing speed when using many drives (using an Apple X-SERVE RAID with 14 drives and a fibre channel card will give you blazing fast speeds that single drives don't come close to).
-heating your office
-raising the noise floor in a room so that your G5 doesn't sound so loud anymore.

I come home with a job.....
1- put the job on the RAID
2- backup the RAID
3- Process the files
4- backup the RAID
5- retouch the files
6- backup the RAID

I want it backed up frequently enough that if the RAID controller fails that I only loose a few hours of my time at most.

There should be one backup set on site for mission critical work.
There should be another backup set safe from Fire/Theft/earthquakes as well.

It is a huge pain in the butt to be sure.  Most friends and family I know do not put much consideration into securing their family photos, work, quicken files, emails and the like.  It never gets easy to hear that someone else lost all the photos from their kids first 5 years.
Title: Hard earned advice against RAID...
Post by: KAP on October 15, 2007, 10:14:24 am
Quote
Well, it's official, I just lost most of my work   About half a year ago I bought a hardware RAID solution with two hot-swappable drives, thinking that my worrying days were over. Instead the very solution that was supposed to protect my files ended up destroying them. It started when one of the drives failed, with a "bad block" error. I googled it and found it would resolve it self with a zero-out format. After formatting the drive I inserted it into the RAID casing and to my horror I see it starting to copy the empty drive to the drive with my files. I try to abort it but it was too late. Desperate I tried using data recovery software only to find that it is total bs, nothing was retrieved.

So, a word of advice, stay clear of RAID and just back up your files on two drives your self. And never completely trust any hardware, however much it's supposed to "protect" your files.

P.s. Please refrain from any hindsight advice, I'm depressed enough as it is. Thanks.
[a href=\"index.php?act=findpost&pid=145881\"][{POST_SNAPBACK}][/a]

This is one of the reasons I use Aperture and the vault system, I back up to 3 other drives (4 drives total) in rotation. I never have more than two drives connected at one time.
In short I have the main drive, when I add files to that I plug in another drive, I then ask Aperture to update it, Apertures adds any alterations on the main drive to the back up drive. Next time I back up to a different drive and so on. My theory being not backing up all the drives at once gives me a chance if I've copied a virus or other problem to find it and correct it.
I don't like the idea of Raid, I feel more secure with independant back ups.

Kevin.
Title: Hard earned advice against RAID...
Post by: jonstewart on October 15, 2007, 10:21:46 am
Quote
I don't like the idea of Raid, I feel more secure with independant back ups.

Kevin.
[a href=\"index.php?act=findpost&pid=146104\"][{POST_SNAPBACK}][/a]

Yes, but, I think this is the general point here. RAID is not really about backup; it's more about uptime reliability and easier disaster recovery (assuming you don't rebuild the new clean drive to the old one with the data!)
Title: Hard earned advice against RAID...
Post by: luong on October 15, 2007, 01:29:32 pm
A few years ago, I bought a Miglia box that was supposed to do the same thing as your unit. However, upon deployment, it simply killed the drives in it. This made me think about the liabilities associated with using additional levels of hardware/software. Since then I have stayed with simple backups.

The main problem with RAID 1/5 is that it gives a false sense of security. In the computer lab where I used to work, we had the controller in a RAID 5 box fail. All the drives were toasted.
Title: Hard earned advice against RAID...
Post by: santa on October 15, 2007, 02:11:06 pm
Neither RAID0 or RAID1 are of any use for data backup. Raid1 helps in the event of a HD failure, but if a directory structure is corrupted or the user deletes half his hard drive there is no protection there. I have books on my shelf dealing with the simple subject of backup.
Title: Hard earned advice against RAID...
Post by: jpjespersen on October 15, 2007, 02:13:13 pm
I just purchased a Drobo robot system.  I hope that it works better than your raid did.
Title: Hard earned advice against RAID...
Post by: feppe on October 15, 2007, 02:35:17 pm
Quote
I just purchased a Drobo robot system.  I hope that it works better than your raid did.
[a href=\"index.php?act=findpost&pid=146144\"][{POST_SNAPBACK}][/a]

Wow, thanks for the heads up. That sounds a lot like Infrant's ReadyNAS NV+, but with an intuitive "user-interface": no interface. Please share your experiences with us. I've been looking for a good NAS which doesn't require me to plonk thousands, and can add drives as I go. Drobo sounds like it fits the bill perfectly.
Title: Hard earned advice against RAID...
Post by: Jonathan Wienke on October 15, 2007, 03:13:33 pm
To reiterate a few points:

RAID 0 is more likely to kill your data  than a single drive. With 2 single drives, if a drive fails you lose half your data. With a 2-drive RAID 0 array, if either drive failes you are f***ed.

RAID 1 (mirroring) is slightly better; if you lose a drive you can still access your data. But you're susceptible to user error (rebuilding from blank drive instead of data drive) even if you change out the bad drive.

RAID 5 is the best option overall. You get data protection even if a drive fails, and when changing a bad drive, it is impossible to rebuild the new drive over all the old ones with data. It's also the most efficient option, with only one drive in the array needed to provide redundancy instead of data protection. The more drives in the array, the smaller the percentage of overhead needed to provide redundancy.

RAID provides protection against drive failure (except RAID 0, which increases the probability of data loss due to drive failure), but is not a substitute for having backups of all of your data, preferably in geographically separate protected locations.
Title: Hard earned advice against RAID...
Post by: rethmeier on October 15, 2007, 04:16:19 pm
Is anybody using a Mac Pro with the Raid Card from Mac?
I'm in the process of purchasing a new Mac Pro and I noticed that there is a Raid Card available.
It is pricey ,however if it protects your files,who cares.
Also,does anybody know,how much Ram is needed to run CS3?
Cheers,
Willem.
Title: Hard earned advice against RAID...
Post by: jpop on October 15, 2007, 04:32:45 pm
There are two kinds of data.  Data that's been backed up and data that hasn't been lost yet.

RAID level 1/5 solutions are fault tolerant but by no means a replacement for a back up.
Title: Hard earned advice against RAID...
Post by: Steve Kerman on October 15, 2007, 04:39:59 pm
Quote
It is pricey ,however if it protects your files,who cares.
[a href=\"index.php?act=findpost&pid=146180\"][{POST_SNAPBACK}][/a]
I'm wondering If you've read the rest of the thread?      The point is that while RAID 1/5 do in theory give you small bit of protection, the word "small" is to be emphasized.  RAID is not intended as a replacement for a backup solution; it is intended to be an inexpensive, high-uptime alternative to expensive server drives.  The redundancy added to improve uptime is not foolproof--data can most certainly be lost, as people here have poignantly testified.
Title: Hard earned advice against RAID...
Post by: digitaldog on October 15, 2007, 04:45:07 pm
Quote
I'm wondering If you've read the rest of the thread?      The point is that while RAID 1/5 do in theory give you small bit of protection, the word "small" is to be emphasized.  RAID is not intended as a replacement for a backup solution; it is intended to be an inexpensive, high-uptime alternative to expensive server drives.  The redundancy added to improve uptime is not foolproof--data can most certainly be lost, as people here have poignantly testified.
[a href=\"index.php?act=findpost&pid=146186\"][{POST_SNAPBACK}][/a]

Well said! It would be totally foolish for anyone to rely on any single system. I guess one could say three separate drives with identical data is 'as good' as a Raid 1 and a 2nd back up. Its a lot less work to use the Raid system however. But the bottom line is, you need multiple copies of important data. 2, 3, who's to say? You can't be too rich, too thin or have too many backups.
Title: Hard earned advice against RAID...
Post by: rethmeier on October 15, 2007, 04:52:09 pm
Thanks for the advise Guys.
I did read the whole posts and realize back-ups are the final solution.
My question was,would that Raid card for the Mac Pro be useful?
Cheers,
Willem.
Title: Hard earned advice against RAID...
Post by: feppe on October 15, 2007, 04:55:52 pm
Quote
Thanks for the advise Guys.
I did read the whole posts and realize back-ups are the final solution.
My question was,would that Raid card for the Mac Pro be useful?
Cheers,
Willem.
[a href=\"index.php?act=findpost&pid=146190\"][{POST_SNAPBACK}][/a]

If you're not that familiar with RAIDs to begin with, something like the Infrant NV+ or the Drobo mentioned above are quite a bit easier to use than a RAID card - not to mention less prone to user error. Drobo appears to even build itself up in case of drive failure, so you don't have to do it manually, like with most (all?) RAID cards.
Title: Hard earned advice against RAID...
Post by: rethmeier on October 15, 2007, 06:24:40 pm
Thanks Feppe!(Bedankt)
I'll go the Drobo route and will use Super Duper for back-up.
Cheers,
Willem
Title: Hard earned advice against RAID...
Post by: pprachun on October 15, 2007, 09:35:42 pm
Quote from: rethmeier,Oct 15 2007, 08:16 PM
Is anybody using a Mac Pro with the Raid Card from Mac?   I'm in the process of purchasing a new Mac Pro and I noticed that there is a Raid Card available.  It is pricey ,however if it protects your files,who cares.
Also,does anybody know,how much Ram is needed to run CS3?
Cheers, Willem.

YES --
I have the MAC RAID Card on my MAC Pro and you're damn right it's pricey; but aren't all those pretty MACs?  I use it for mirroring (RAID 1) for my System drive for security.  All my photo data is on a second RAID 0 (2x750) drive giving me a fast 1.5TB drive.  The SECURITY for my data is 6, yes six, external drives that get rotated.

I have virtually the same setup with my PC: RAID 1 and RAID 0.  And my $280 PC motherboard has the RAID on board -- no extra $1000 option.

RAID is NOT about security, any more than a computer or a drive is.  Security is about REDUNDANCY -- REDUNDANCY -- REDUNDANCY.

BTW: Windows (Vista 32), beats the MAC Pro for speed, usability, and extent of photography software.
Title: Hard earned advice against RAID...
Post by: Frank Doorhof on October 16, 2007, 03:07:59 am
I never run raid for my photography for a simple reason.

WHEN something breaks there is no problem with raid as long as you do what is needed, if you insert the new drive it should mirror the old drive if configured correctly.

HOWEVER when you get a lighting strike it happens often that BOTH drives are affected, and you loose everything.

What I do is much more failsafe (except for EMF).
I own two external drive cases from Addonics, both can house 6 Sata drives with a portmultiplier to one cable.

One case is on when my PC is on and contains drives with old work and a drive with new work.

After every shoot I will power up case 2 and backup the new sessions (retrospect) or added stuff to other folders.
After that it powers down again.

Also once every year I make a backup on an USB drive and this one is stored in my parents house.

Meaning only with a severe EMF load I will loose everthing from that year.

I don't trust DVD's or CD's, I did backups on them a few years ago and although they are still being read you don't want to see the error logs
Title: Hard earned advice against RAID...
Post by: PatrikR on October 16, 2007, 08:14:03 am
Quote
For long term, tape is arguable the best, and most reliable solution.

[{POST_SNAPBACK}][/a] (http://index.php?act=findpost&pid=145906\")

Hi,

What kind of tape are you using? LTO, VXA, AIT, DAT or what? I called HP and they recommended LTO tape systems for longterm storage, like 30 years, but wouldn't confirm or know if their drives would work with Macs. The LTO is reliable and fastest tape and most popular in servers they told.

The VXA seems very durable but slow. There's a demo where they cook, freeze and dunk VXA tapes into hot coffee and so on with 100% recovery. Impressive. [a href=\"http://www.exabyte.com/technology/tested/index.cfm]VXA[/url]

Do you use the tape on a Mac or PC? Most professional tape drives are SCSI or SAS... Since there are no LTO kits for Mac does anybody know if they work with Mac Pro if I buy a $600 Atto Scsi card?

These investments are just growing and growing. Now I'm wondering if I need to invest almost $4000 US to get a tape drive, scsi cards, software and tape media to back up my drives. This is getting very expensive when it seems that there's always need for new equipment. But back ups are a must.

Patrik
Title: Hard earned advice against RAID...
Post by: Ray on October 16, 2007, 09:39:36 am
Quote
I don't trust DVD's or CD's, I did backups on them a few years ago and although they are still being read you don't want to see the error logs
[a href=\"index.php?act=findpost&pid=146296\"][{POST_SNAPBACK}][/a]

Are you suggesting, Frank, that those errors degrade the image in any visible way?

I'm surprised that after 10-12 years of backing up thousands of images on CDs and DVDs, I've never had any major problem retrieving data, even from the very earliest CDs, many of which were non-descript, bargain blanks.

Stories of unreadable discs due to bit rot seem very odd to me. When people have trouble like that, I can't help wondering what the circumstances really are. Perhaps they've used labels with corrosive glue. Perhaps they've stored the discs in a room with other corrosive chemicals. Perhaps they happen to live in a heavily industrialised, polluted city like Beijing. Perhaps they accidentally left the unreadable disc(s) in a parked car to bake in the sun, or perhaps more likely, the unreadable disc(s) were never recorded properly in the first instance.

I've had a number of hard drive failures during that 12 year period, but not one CD or DVD failure of a disc which I know was recorded properly in the first instance. I have had trouble, of course, getting a successful recording (haven't we all) and still have a stack of toasters about a foot high.
Title: Hard earned advice against RAID...
Post by: rainer_v on October 16, 2007, 11:29:59 am
i back up with two discs which i have in seperate locations, and i have backups of all stuff on 2 sets of dvds, also on two places. actual images go in a raid in my g5. on my laptop i copy them manually on external drives if on location.
i trust more in harddrives and dvds. i dont know about the tape storages, but some years ( 10 or so ) ago i stored a lot of ( digital ) music data on DAT backup tapes and nearly one of them was readable some years later. most CD backups of that time stil are readable. i listened that tape is not good for long term because the magnetic information fades away slowly by slowly. also this never was a great problem with analog data, with digital it can become one.
Title: Hard earned advice against RAID...
Post by: Frank Doorhof on October 16, 2007, 12:08:05 pm
Digital = digital so the quality is the same.

The alarm went of when I used an old CD and it took me 20 minutes to read the whole CD while normally that's less than a minute.
Everything was 100% ok.
But when I checked the error correction it was enough to let me destroy my CDs and got the third harddrive which now replaces my DVDs

And to be honest backuping on DVD is a big problem, I shoot in one session between 1-2GB so that would mean ALOT of DVDs
Harddrive space is cheap and very reliable.
Title: Hard earned advice against RAID...
Post by: feppe on October 16, 2007, 12:09:08 pm
Quote
Are you suggesting, Frank, that those errors degrade the image in any visible way?

I'm surprised that after 10-12 years of backing up thousands of images on CDs and DVDs, I've never had any major problem retrieving data, even from the very earliest CDs, many of which were non-descript, bargain blanks.

Stories of unreadable discs due to bit rot seem very odd to me. When people have trouble like that, I can't help wondering what the circumstances really are. Perhaps they've used labels with corrosive glue. Perhaps they've stored the discs in a room with other corrosive chemicals. Perhaps they happen to live in a heavily industrialised, polluted city like Beijing. Perhaps they accidentally left the unreadable disc(s) in a parked car to bake in the sun, or perhaps more likely, the unreadable disc(s) were never recorded properly in the first instance.

I've had a number of hard drive failures during that 12 year period, but not one CD or DVD failure of a disc which I know was recorded properly in the first instance. I have had trouble, of course, getting a successful recording (haven't we all) and still have a stack of toasters about a foot high.
[a href=\"index.php?act=findpost&pid=146351\"][{POST_SNAPBACK}][/a]

My experience is the polar opposite: I've had one HDD fail out of 20 (guesstimate) I've used, while I have whole spindles of DVDs which are unreadable. And my HDDs are on 24/365 and in constant use for years. The DVDs are the cheapest I can find, but I store them appropriately.

But that's not the reason why I back up to HDDs. Backing up 100+ gigs of images and 100+ gigs of music to DVDs is about as appealing as pouring battery acid on my eyes. While backing up a few DVDs a week incrementally is ok - ie. if you don't have to start from zero, and/or only backup the latest work on DVDs -, it still leaves verification.

Verifying those 200+ gigs of data every month is a colossal waste of time. With HDDs I can do full bit-to-bit verification each and every time I backup with only a marginal increase in time. And if you don't verify your backups, you're likely to get two shocks when you lose your data: one when you do, and second when your backups are DOA. And test recoveries are much faster, also. HDDs over DVDs as backups are a no-brainer.
Title: Hard earned advice against RAID...
Post by: Steve Kerman on October 16, 2007, 12:12:21 pm
From PatrikR's VXA link:
Quote
VXA is the only tape format in the world to write and read data on tapes in individually addressed data Packets.
I have to say that I am absolutely dumbfounded to read that.

Let me tell you a story that I thought was ancient history:

Many years ago I had a QIC tape backup system from Emerald Systems.  To give some idea of the timeframe, the tapes each held 60 megabytes.  I used their tape system to back up the development files of my company.

One day while editing a file, I happened to destroy it.  So, I went to the most recent backup tape to attempt to recover a recent version of the file.

A short way into reading the backup tape, the drive reported an unrecoverable read error.

At which time it stopped.

It didn't continue reading past said read error, which was on a part of the tape that didn't contain my lost file.  It simply stopped, and refused to read anything at all past that first error on the tape.

After some angry calls to Emerald Systems, I was told that the design of the QIC tape standard was such that drives couldn't read past an error.

Frankly, I thought that I was being fed a line of BS, and never again did any business with Emerald Systems.

As an engineer in the microelectronics field, I am absolutely dumbfounded to read that not only was what Emerald said about QIC tapes apparently true, but that this design flaw has continued to exist in the multiplicity of tape standards that have been developed since that time.  It seems to me to be utterly obvious that a backup tape system has to be able to ignore bad spots on the tape and recover all the readable data that is beyond the bad spot.  How this situation could have been designed in in the first place, and then continued to exist for these many years, is stupidity of a level that exceeds my ability to comprehend.


Reviewing what I just wrote, I see that it was quite a rant!  I guess the loss of that data after having spent thousands of dollars on a backup system that didn't work still stings.    I guess the moral of this story is not to trust your precious data to backup tapes that can't be read if there are any bad spots.  Which would appear to imply that VXA tapes are the only tapes which might be suitable.
Title: Hard earned advice against RAID...
Post by: Dustbak on October 16, 2007, 01:50:04 pm
You would not believe how many companies are extremely rigid in backing up every bit of data but have never performed a (full) recovery as a test.

Many of them are in for a real surprise whenever they need to recover. Most of the time separate data is very well doable but when it comes to recovering servers with stuff like active directory or exchange db servers it is a totally different story.
Title: Hard earned advice against RAID...
Post by: Monito on October 16, 2007, 04:33:37 pm
Quote
You would not believe how many companies are extremely rigid in backing up every bit of data but have never performed a (full) recovery as a test. Many of them are in for a real surprise whenever they need to recover. Most of the time separate data is very well doable but when it comes to recovering servers with stuff like active directory or exchange db servers it is a totally different story.[a href=\"index.php?act=findpost&pid=146400\"][{POST_SNAPBACK}][/a]
Good point.  Another point in favor of external drives is that one should every few years (5?) copy everything off of the old drives onto the current sweet spot capacity drive, verify the integrity of the data, and then box and put into storage the old drives.

Suppose that two years ago, a 200 GB external drive cost $200.  I wouldn't be surprised if three years from now a 2,000 GB (2 TB) drive will cost $200.  So copy ten of your shelf archive drives onto the 2 TB drive.  Verify.   Get your corresponding ten offsite archive drives and copy them onto a second 2 TB drive.  Verify it and take it and the ten old drives offsite again.

This way you keep the bits and the technology current.  If any one of the 20 drives fails to deliver its precious images, you can save the day with the other copy before you have a duplicate failure ten or 15 years down the line.
Title: Hard earned advice against RAID...
Post by: Steve Kerman on October 16, 2007, 04:47:29 pm
Good point about saving old archives.  I've been through a couple of episodes of going to IT with, "Remember Frank, who left two years ago, who designed the FPGA for the ZZXY system that we're selling million$ of each month?  Well, it seems he didn't properly release the source code to Document Control before he left, so we need to recover the copies that were on his PC, which you archived when he left."  To which IT responds, "Oh, we only keep those past-employee archives for a year; we recycled that tape a long time ago."  
Title: Hard earned advice against RAID...
Post by: nicolaasdb on October 17, 2007, 12:48:56 am
one word.....backup!!!!!!!!

a raid is not a backup ever!!!!

but  feel your pain!! try to have your drives recovered.. there are companies that can get your info back,,,,but it is expensive
Title: Hard earned advice against RAID...
Post by: Ray on October 17, 2007, 02:25:23 am
Quote
My experience is the polar opposite: I've had one HDD fail out of 20 (guesstimate) I've used, while I have whole spindles of DVDs which are unreadable. And my HDDs are on 24/365 and in constant use for years. The DVDs are the cheapest I can find, but I store them appropriately.
[a href=\"index.php?act=findpost&pid=146374\"][{POST_SNAPBACK}][/a]

That seems very strange indeed. Whole spindles of DVDs that are unreadable? Have you investigated possible causes?

I could appreciate that you might have unwittingly bought whole spindles of reject and faulty DVDs from some unscrupulous dealer, perhaps DVDs that were missing their protective coating.

I could also appreciate that you might have been in a hurry and recorded whole spindles of DVDs without verification of data (which can sometimes take as long as the initial recording process), or that you might have recorded the unreadable DVDs with incompatible burner and/or software which caused the discs to be unreadable on other burners.

I've also experienced inexplicable, apparent unreadability of the occasional disc on other DVD players. It's happened a couple of times that RAW images I've recorded to DVD on my laptop when travelling, have been unreadable on a particular desktop computer back home but perfectly readable on the laptop and also perfectly readable on another desktop computer with a more up-to-date DVD reader/burner.

I've also tried to play the occasional pirated DVD movie (bought in Asian countries - can I say this without incriminating myself?) which would not play well (or at all) on my stand-alone DVD player, but which would play apparently perfectly on my computer, presumably because the computer DVD player and software had better error correction.

It's so easy after reading horror stories about bit rot and general articles about optical media not being as reliable as they're cracked up to be, to make the first assumption when things go wrong that it's the disc itself which is faulty or unrealiable. I could almost bet my bottom dollar that in 9 cases out of 10, the causes of the problems are user error and/or incompatible software and hardware.

So let's be clear about this, Feppe, just for the record. Are you saying that you've got whole spindles of DVDs that you know were initially recorded properly and were readable after recording, on at least one machine, that were placed in proper storage for a period of time and subsequently became unreadable on any of several computers that you tried; that is, unreadable period?
Title: Hard earned advice against RAID...
Post by: SeanPuckett on October 17, 2007, 01:30:56 pm
Images are business data.  Treat it like any other business treats its data -- or should.  If you're not doing what the banks do, you're taking the same risks with your life.

Fast, redundant online storage -- RAID 5 at a minimum.
Reliable nearline storage -- External hard disk backups, done daily.
Permanent off-site storage -- Archived data on hard disks or DVDs somewhere else.

Redundant online storage protects you from hardware failure.
Nearline storage with daily backups protects you from user failure.
Off-site storage protects you from acts-of-god.

There is no alternative; there is nothing less responsible to your future self and business interests than to shirk competent and adequate data protection.  No excuses.

(Guess what industry *I* used to be in.)
Title: Hard earned advice against RAID...
Post by: rainer_v on October 17, 2007, 02:11:55 pm
Quote
From PatrikR's VXA link:

I have to say that I am absolutely dumbfounded to read that.

Let me tell you a story that I thought was ancient history:

Many years ago I had a QIC tape backup system from Emerald Systems.  To give some idea of the timeframe, the tapes each held 60 megabytes.  I used their tape system to back up the development files of my company.

One day while editing a file, I happened to destroy it.  So, I went to the most recent backup tape to attempt to recover a recent version of the file.

A short way into reading the backup tape, the drive reported an unrecoverable read error.

At which time it stopped.

It didn't continue reading past said read error, which was on a part of the tape that didn't contain my lost file.  It simply stopped, and refused to read anything at all past that first error on the tape.

After some angry calls to Emerald Systems, I was told that the design of the QIC tape standard was such that drives couldn't read past an error.

Frankly, I thought that I was being fed a line of BS, and never again did any business with Emerald Systems.

As an engineer in the microelectronics field, I am absolutely dumbfounded to read that not only was what Emerald said about QIC tapes apparently true, but that this design flaw has continued to exist in the multiplicity of tape standards that have been developed since that time.  It seems to me to be utterly obvious that a backup tape system has to be able to ignore bad spots on the tape and recover all the readable data that is beyond the bad spot.  How this situation could have been designed in in the first place, and then continued to exist for these many years, is stupidity of a level that exceeds my ability to comprehend.
Reviewing what I just wrote, I see that it was quite a rant!  I guess the loss of that data after having spent thousands of dollars on a backup system that didn't work still stings.    I guess the moral of this story is not to trust your precious data to backup tapes that can't be read if there are any bad spots.  Which would appear to imply that VXA tapes are the only tapes which might be suitable.
[a href=\"index.php?act=findpost&pid=146375\"][{POST_SNAPBACK}][/a]

this is exactly what happened to my DAT backup tapes. fortunately i had also CDs, one of the first you could burn yourself..... some were not working but most dd although i didnt care at al how i strored them, because i thought i have everything save on tapes.
Title: Hard earned advice against RAID...
Post by: feppe on October 17, 2007, 03:16:23 pm
They were not backup disks, so I didn't verify them. The most likely reason is:

Quote
I've also experienced inexplicable, apparent unreadability of the occasional disc on other DVD players. It's happened a couple of times that RAW images I've recorded to DVD on my laptop when travelling, have been unreadable on a particular desktop computer back home but perfectly readable on the laptop and also perfectly readable on another desktop computer with a more up-to-date DVD reader/burner.
[a href=\"index.php?act=findpost&pid=146563\"][{POST_SNAPBACK}][/a]

They were readable, but that was with the burner I burned them with. Now I have a new burner, which refuses to read a few spindles - but works on most of the brands burned on the old recorder.

In any case, it doesn't matter. I've given the reasons why HDD kicks DVDs' ass when it comes to backing up. Having to worry about unreadable media on some readers is yet another reason to avoid DVDs.
Title: Hard earned advice against RAID...
Post by: Bruce MacNeil on October 17, 2007, 03:52:30 pm
My 2 cents/

I use a large raid with Aperture.

2 raids really. One for library - one for vault.

When I work on a new project - I export that Aperture project to external bare drives that are kept off-site. These drives use MS-ds file system so I can read them with a Mac or windows machine.

I have experiment on the reliability by:

1) rebuilding library from vault
2) reading raw data from the archive drives with Max and with PC.

Things seem to work and I have everything to date always live.

The raids are 8TB each and I buy 500 GB external drives for about $100.

I also use subsumation.
Title: Hard earned advice against RAID...
Post by: Fred Ragland on October 17, 2007, 04:24:48 pm
Quote
...I also use subsumation.
[a href=\"index.php?act=findpost&pid=146711\"][{POST_SNAPBACK}][/a]
Hmm...subsumation, to ??????
Title: Hard earned advice against RAID...
Post by: Bruce MacNeil on October 17, 2007, 06:53:06 pm
Quote
Hmm...subsumation, to ??????
[a href=\"index.php?act=findpost&pid=146719\"][{POST_SNAPBACK}][/a]

In  the sense that the child subsumes the parent. Drive capacity adequate for 2 years of work. Then, in 2 years the new purchase (there is always a new purchase) will be large enough to contain the older system with tremendous room to spare.

In this manner I have every dig files created since 1995 ready and at hand.
Title: Hard earned advice against RAID...
Post by: Ray on October 18, 2007, 01:30:05 am
Quote
They were readable, but that was with the burner I burned them with. Now I have a new burner, which refuses to read a few spindles - but works on most of the brands burned on the old recorder.

Just as I thought. There's an important distinction to be made between hardware  failure or material breakdown/instability and software/firmware incompatibility. Computers are always giving trouble for inexplicable reasons. One usually succeeds in fixing the problems through software/firmware updates or, in the case of an unreadable CD/DVD, simply by replacing the DVD drive.

I've been storing images on CDs and DVDs for about 15 years. In fact, I was getting my old Kodachromes transferred to Kodak's Photo-CD system before I'd bought my first computer. All those early recordings are perfectly readable on my current computers. But that has not always been the case. The second CD drive I bought, upgrading a 4x CD reader to a 20x reader about 10 years ago, refused to read many of those images. Naturally, I assumed the discs were already beginning to suffer from bit rot, although I couldn't see any blemishes on the surface, and of course I complained to Kodak on their toll-free number before I tried the discs on a friend's computer and discovered the problem did not lie with the discs but with the CD drive.

Quote
In any case, it doesn't matter. I've given the reasons why HDD kicks DVDs' ass when it comes to backing up. Having to worry about unreadable media on some readers is yet another reason to avoid DVDs.

No, I think you're wrong. It does matter. So far you've confirmed that you've had one hard drive failure but have not confirmed that you've had any DVD failures. For all you know, those apparently unreadable discs, which were once readable on the drive that burned them, are still in perfect condition.

I assure you it's far more worrying to have images stored on a medium which is rapidly deteriorating physically than it is to have images which may occasionally be subject to software incompatibilities during attempted access.
Title: Hard earned advice against RAID...
Post by: feppe on October 18, 2007, 02:03:37 am
Quote
No, I think you're wrong. It does matter. So far you've confirmed that you've had one hard drive failure but have not confirmed that you've had any DVD failures. For all you know, those apparently unreadable discs, which were once readable on the drive that burned them, are still in perfect condition.

I assure you it's far more worrying to have images stored on a medium which is rapidly deteriorating physically than it is to have images which may occasionally be subject to software incompatibilities during attempted access.
[a href=\"index.php?act=findpost&pid=146822\"][{POST_SNAPBACK}][/a]

As said, I've given the list of reasons why HDDs are better than DVDs as backup earlier, and the reasons are none of the ones you list above - and are more pertinent.

Besides, NTFS and FAT32 are industry standards, and a HDD can be read by PCs, Mac or Linux machines. But once again, that's not really a problem as one can migrate all the backups to new media when it becomes available in a matter of minutes. Unlike DVDs, which will take hours and hours.
Title: Hard earned advice against RAID...
Post by: Ray on October 18, 2007, 02:51:06 am
Quote
As said, I've given the list of reasons why HDDs are better than DVDs as backup earlier, and the reasons are none of the ones you list above - and are more pertinent.

Besides, NTFS and FAT32 are industry standards, and a HDD can be read by PCs, Mac or Linux machines. But once again, that's not really a problem as one can migrate all the backups to new media when it becomes available in a matter of minutes. Unlike DVDs, which will take hours and hours.
[a href=\"index.php?act=findpost&pid=146831\"][{POST_SNAPBACK}][/a]

Fair enough! We all have our preferences for a variety of reasons, some of which might be sheer convenience. I've no argument there. I'm just concerned with the facts.

My claim is, I haven't lost any data that has been correctly recorded on optical media since I first started using it 15 years ago. But I have lost data due to failure of hard drives. So have you, it seems.

You claim you have a number of DVDs that are not readable on your current DVD drive but you haven't confirmed whether this is due to DVD drive/software incompatibility or deterioration of the disc.

My concern is you are confusing issues; the convenience and time-saving factor of backing up to hard drives as opposed to the ultimate, long term reliability of optical media.
Title: Hard earned advice against RAID...
Post by: Frank Doorhof on October 18, 2007, 03:02:57 am
Let me clearify something

We own a PC store (well actually my wife runs it since I'm full time into photography), the people coming to us with defective DVDR's and CDR's are much more than the defective HDDs.

Same goes for tape.
Tape works when it's stored dry and on a constant temperature.
And you will often need the EXACT same tapereader.

HDDs are in my opinion (and that's with 15 years experience in selling IT) the best bet for failsafe and future proof backup.

But as with ALL backups it's only safe if checked at least yearly (refreshed on HDDs) AND if stored powerless.
And of course it's best to keep at least two copies from older work.

So old work backuped up in your own house and on a location away.
New work backuped up in your own house.
Title: Hard earned advice against RAID...
Post by: Ray on October 18, 2007, 03:31:08 am
Quote
We own a PC store (well actually my wife runs it since I'm full time into photography), the people coming to us with defective DVDR's and CDR's are much more than the defective HDDs.

Frank,
What are you saying precisely? Customers are returning prerecorded DVDs and CDs that won't play on their machines? Customers are returning blanks which fail to record on their burners? Or customers are returning their own recordings that used to be okay but no longer are able to be read due to disc deterioration?

Quote
But as with ALL backups it's only safe if checked at least yearly (refreshed on HDDs) AND if stored powerless.
And of course it's best to keep at least two copies from older work.

Well, that's a rather onerous chore, isn't it? The great thing about optical media is, if you forget or find it inconvenient to do that annual renewal, you'll still be okay. Probably the worst that could happen is you might find an odd 10 year old CD that's become corrupted due to physical deterioration and/or faulty manufacture and you almost lost 700mb of data, but didn't because you'd already re-recorded the data to DVD about 5 years ago.  
Title: Hard earned advice against RAID...
Post by: Frank Doorhof on October 18, 2007, 01:41:44 pm
What I meant is what I wrote down

We hear ALOT of complaints about media not reading correct anymore after 4-5 years, some even after 2 years.
I made some backups on the EXTREME expensive TDK goldplated 100year warranty CD-R's, they retailed in that time for arround $20.00 per piece.

I have used 10.
And from those 10 I now have SERIOUS problems reading them with 5.
That's a 50% loss of data.
They are still read by the way without problems but it's not normal that a CD with 600MB will take arround 15 minutes to read.

When I started using DVD-Rs it became noticably worse. Some DVD-Rs were allready showing read errors after 1 year.

With HDDs I know for sure I have at least two backups and because of the size and price it's easy to make double backups.
A 500GB HDD is copied over night and put in storage.
A second 500GB HDD is copied the other night and put online for incremental backups.

For the $120.00 a 500GB retails now that's a no brainer.
If I were to backup on DVDRs I would need ALOT, and I mean a WHOLE LOT
Title: Hard earned advice against RAID...
Post by: Ray on October 18, 2007, 02:15:49 pm
Frank,
Sure is a mystery to me. What are you guys doing??

As I said, I've also had problems with the occasional disc not reading, or reading very slowly then grinding to a halt, but it has always proved to be caused by an incompatible (or faulty, whatever) drive. Those early Kodak CDs, from which it used to take a full 2 minutes on my first computer for an 18mb image to be read and displayed, can now be read at lightning speed on my current XP64 desktop, about 4 or 5 seconds as I recall.

Fortunately, I've always been in the habit of creating space on my hard drives as they fill up with images, by transferring the data to CD and later DVD.
Title: Hard earned advice against RAID...
Post by: Monito on October 18, 2007, 03:26:35 pm
How many DVDs and CDs do you have, Ray?  How do you file them?  In stacks?
Title: Hard earned advice against RAID...
Post by: LasseDPF on October 19, 2007, 04:07:42 pm
Anyone have any experience with the iomega rev drives ?
Title: Hard earned advice against RAID...
Post by: Jonathan Wienke on October 20, 2007, 06:09:10 am
I wouldn't recommend ANY proprietary drive technology for backups. Does anyone have a working reader for a Syquest SparQ drive??? With industry standard drives like SATA, IDE, etc. you have a pretty decent chance of reading the drive in 10 yeard even if the manufacturer goes out of business...
Title: Hard earned advice against RAID...
Post by: jonstewart on October 20, 2007, 06:14:55 am
Quote
I wouldn't recommend ANY proprietary drive technology for backups. Does anyone have a working reader for a Syquest SparQ drive??? With industry standard drives like SATA, IDE, etc. you have a pretty decent chance of reading the drive in 10 yeard even if the manufacturer goes out of business...
[a href=\"index.php?act=findpost&pid=147351\"][{POST_SNAPBACK}][/a]

I agree with you Jonathan!

However, believe it or not, I do have a working Sparq drive and 5 disks (at least they were working 9 months ago when I tested them).... and no, I'd never dream of using that technology again, for exactly the reasons above.

I'd never again buy into proprietary hardware technology.
Title: Hard earned advice against RAID...
Post by: Eric Myrvaagnes on October 20, 2007, 01:53:28 pm
Quote
I wouldn't recommend ANY proprietary drive technology for backups. Does anyone have a working reader for a Syquest SparQ drive??? With industry standard drives like SATA, IDE, etc. you have a pretty decent chance of reading the drive in 10 yeard even if the manufacturer goes out of business...
[a href=\"index.php?act=findpost&pid=147351\"][{POST_SNAPBACK}][/a]
And I've got an 8-inch floppy disk around here somewhere. I think it was supposed to hold maybe 120KB of data. I don't have a drive for it, however.  
Title: Hard earned advice against RAID...
Post by: pss on October 20, 2007, 02:17:40 pm
i have used raid 5 for years and it is the best/safest backup method i know....DVDs are useless, after a couple of years they become unreadable....even the manufacturers will agree with that....does not mean that they will all be bad, but you know the one you need will be, plus 4.5 or even 9gb don't really do that much....blueray might be better, but there is still the disc/storage/life issue...
so i have a raid 5 box with 3TB and a couple of mirrored raids....if any of the harddrives fail, i am safe....which is my biggest concern....i swap the HDs about every 2 years anyway to get bigger ones, so the drives never get that old....i have never had a drive fail, but i know one day it will.....
my personal data/day today files are backed up onto a mirrored raid and also online.....i am in the process right now to backup my entire image library online....most important files first....offline storage is the best solution.....all the backup in the world won't help if the drives are sitting next to the computer when the fire hits/water hits/theft happens/whatever.....
with the prices of online storage coming way down i will probably have my most important files with 2 different services....just to make sure....

i stay away from proprietory tapes/drives....anyone old enough to remember syquest? anyway, they all get phased out at one point or another....

my raid 5 is a nitroav box...highly recommended....
Title: Hard earned advice against RAID...
Post by: DarkPenguin on October 20, 2007, 03:56:47 pm
Quote
I agree with you Jonathan!

However, believe it or not, I do have a working Sparq drive and 5 disks (at least they were working 9 months ago when I tested them).... and no, I'd never dream of using that technology again, for exactly the reasons above.

I'd never again buy into proprietary hardware technology.
[a href=\"index.php?act=findpost&pid=147353\"][{POST_SNAPBACK}][/a]

If you have the last working drive on the planet you can probably become rich reading peoples orphaned backups and burning them to dvd.  Just an off topic thought.
Title: Hard earned advice against RAID...
Post by: geesbert on October 20, 2007, 05:49:42 pm
i am not going to bore you with how i am backing up my files, it is a bit like some of you do...

but one thing is for shure, my personal pictures, especially of my kids, are all getting printed. those prints might be a bit faded in 30 years, but my kids will probably be able to look at them without understanding my backup system.

i just made a very usable scan of a 45 year old picture of my parents, it was a very bad B&W print where the photographer used a bad fixer, but after a bit of work it is now a very printable digital file.
Title: Hard earned advice against RAID...
Post by: Ray on October 21, 2007, 02:53:02 am
Quote
How many DVDs and CDs do you have, Ray?  How do you file them?  In stacks?
[a href=\"index.php?act=findpost&pid=147001\"][{POST_SNAPBACK}][/a]

Thousands, Monito. I used to store the early CDs in the cases they came in, placed in CD racks. Then I discovered zipped wallets with plastic sleeves, very cheap, made in China, but ideal for my purposes. I have various sizes ranging from 12 sleeves to 96. Those that hold 96 DVDs would be about the size of the average 400-500GB external hard drive of a couple of years ago and hold about as much data. I store these wallets on a shelf like books.

When Blu-ray discs become more affordable and multi layered and dual sided, a 96 disc wallet should be able to hold about 10 terabytes of data. That might be sufficient for all the images I have taken in my entire life, but if I'm extremely prolific in my old age, then 2 book-size wallets should be enough.  
Title: Hard earned advice against RAID...
Post by: neil snape on October 21, 2007, 02:44:26 pm
Great thread that immediately scared me into backing up everything on all the computers. I'll just have make it a scheduled routine. Andrew uses SuperDuper. I used Carbon Copy Cloner. I hope to find a way to do incremental backups scheduled weekly. Is this alright or should I use something like Retrorespect?
Title: Hard earned advice against RAID...
Post by: Eric Myrvaagnes on October 21, 2007, 04:27:28 pm
Quote
Great thread that immediately scared me into backing up everything on all the computers. I'll just have make it a scheduled routine. Andrew uses SuperDuper. I used Carbon Copy Cloner. I hope to find a way to do incremental backups scheduled weekly. Is this alright or should I use something like Retrorespect?
[a href=\"index.php?act=findpost&pid=147647\"][{POST_SNAPBACK}][/a]
For years my "backup protocol" was that I would back up everything whenever I heard that a friend had lost data from a crashed hard disk.    

These days I use a variety of external HDs pretty much as others have described.
Title: Hard earned advice against RAID...
Post by: rethmeier on October 21, 2007, 06:23:34 pm
What about that new thing called Time Machine in Leopard?
It also copies and back-ups.
I'll hook my Drobo on to it.
Cheers,
WR.
Title: Hard earned advice against RAID...
Post by: neil snape on October 23, 2007, 06:47:23 am
Quote
What about that new thing called Time Machine in Leopard?
It also copies and back-ups.
I'll hook my Drobo on to it.
Cheers,
WR.
[a href=\"index.php?act=findpost&pid=147684\"][{POST_SNAPBACK}][/a]
It's a good thing the Time Machine feature. Hope it works better than XP's recovery that has never worked on my PC. It seems to be limited to a limited range of devices but I'm still in the camp of multiple redundant back up disks over raid anyway. Suppose I should recopy all my CD's and DVDs before they too are unreadable.
Title: Hard earned advice against RAID...
Post by: larryg on November 17, 2007, 09:32:18 pm
Quote
I wouldn't recommend ANY proprietary drive technology for backups. Does anyone have a working reader for a Syquest SparQ drive??? With industry standard drives like SATA, IDE, etc. you have a pretty decent chance of reading the drive in 10 yeard even if the manufacturer goes out of business...
[a href=\"index.php?act=findpost&pid=147351\"][{POST_SNAPBACK}][/a]


You are so right I had the Syquest (still have it somewhere) but would have no idea where the software to run it is.  I also did the Bernouli drive and the Jazz Drives with skuzzi hookup   Not sure they would work now.  

Standardization would be a better long term solution