Tape backup for MacOS X 10.4 ?

[Christopher Sanderson]

My bus strategy for video is to keep the i/o on a single fast channel free of other demands.

All apps, utilities etc. live internally; all captured & edited video lives externally on the FW800 drives. I found with FW400 that if multiple demands were made on a FW bus, the signal would often fail. This strategy also allows the video to be portable between machines if I need to edit the same video on different computers.

Chris S

[Concorde-SST]

Hello all,

thanks for all the great informations - I now agree that tape backup
does make sense mostly for enterprise users who do that regularly
and aren´t that restricted by something like a private budget...

I´ve heard about tape "bleeding" - but not in that phrase - thanks
- I forgot that and can agree with that.

Could this happen to HDD´s too? Does anyone has some advice
for storing HDD´s (off-line)? Do they lose magnetization (?) or
could they "bleed" too? Are they being remagnetized automatically
or not?! Would a complete three-way-backup do the trick
(copy the full harddisk to another one with same or greater
size, then copy it back to the original one)?

So I will look around after a new LaCie harddisk - I found them
very reliable to work with an apple computer - since I own one
1 TB (FW800) and four 80GB pocket drives for travelling (FW400).

Chris, I´m shocked about your experience with Retrospect - I was
thinking of buying this software but your experience kept me from
doing this. I´d try the somewhat more simple and hopefully more
reliable CCC software.

Of course a RAID would do the trick too - if you can spare the
money and space (be aware of the noise and the heat-generation!!
A friend of mine works as a system operator for a middle sized
company and says the raid arrays produce so much heat that they
need to be in an special climatized room).

To Andy:

I can´t recommend you to use your internal HDD for HD editing - I´d do it like
Chris mentioned. I was doing that with my photos a while ago but found it
much safer and faster to leave the internal HDD for the OS only and the programs.
Its wise not to fill up the internal HDD more than half of its size - so the HDD
stays fast.

But it depends on your desired use of HD video editing or still photography. If you
do it relatively occassionally, then go for it, otherwise use one or more external
drives. This has become clear to me more than before - thanks to this thread.

so guys,

thanks a lot for all the good advice and
a happy shooting this weekend - fall
colours should be great over the big
pond...

all the best,


Andreas
Nuernberg,
Germany.

[David Mantripp]

This is sadly not the case. Tape cartridges are more prone to wear out with use and become unreadable than hard drives, and are also more likely to deterirate in storage. Tape requires physical contact between the R/W heads and the tape, which means wear occurs on the surface of the medium each time the tape is read or written.This is sadly not the case.

Jonathan, I do sometimes wonder how you managed to become an expert on everything under the sun. I wish I had your IQ and memory.

However, in this case, as in many others, I suspect, you're talking total rubbish. In long term data storage (e.g scientific data, which I do actually know about), DAT and Exabyte rule supreme.  You may have a sort of a point on tape wear, but you see, tapes are not cycled very often.  Disk drives do fail, often abruptly. So do disk controller electronics. And interfaces come and go.  An Exabyte drive for MacOS will cost around $2000, which, compared to an ever expanding RAID array, is not so bad.

Disks are for online data, essentially, not archival. Digital Tape, with a good cycling strategy, is a safer long term bet.  And far, far more scaleable in terms of cost, physical space and data space.

[pobrien3]

I have to defend Jonathan here (not nice to say he's talking total rubbish; he isn't). Large volume backup and data storage is perfectly fine on disk. Tape is still used in many organisations but this is transitional.  Backup to tape is linear, failure rate is not insignificant, and it's not cheap. It is inefficient and slow. Amazon.com now operate (I believe) the world's largest commercial database, which they run on a grid / RAC architecture: they have to run 24/7, they don't back up to tape. Storage arrays when properly managed using modern manifestations of RAID-type technology don't need backup as they have load-balanced failover.  You can take entire disk sections out with an axe and still lose nothing. For business continuity / DR the biggest organisations will replicate the storage arrays remotely: often (and definitely in the case of the global banks) in real time.  This is all done on disk.  Oracle Corporation operate and host extremely large databases and storage arrays, which they operate 24/7 with zero downtime for backup / replication / patching / maintenance. Know who replaced SUN as their prime hardware platform?  Dell. I don't believe you'll find a tape drive in Oracle's Austin datacentre, and I can tell you this first hand.

Disks are for online, not archival? Says who? Old thinking.  I've been bitten by tape failures far more often than disk failures.

You can easily and cheaply buy the hardware for a RAID array, and hard disks are cheap and plentiful. Rely on one disk and in the event of a failure you'll get hurt: invest in a sensible storage strategy and you'll easily cope with disk failure.

I can't leave without making a further remark: Jonathan is RIGHT to make sure he knows about data storage - it's his livelihood.  In the recent past a photographer who negligently stored film media would end up with damaged originals, even if he could find them. Big bucks were spent on archival storage for film media. For digital, he HAS to understand digital storage: why are you surprised?

Jonathan may not be expert on everything under the sun, but I for one think far too many 'professional' photographers are anything but professional in this modern era.  Love him or loathe him, Jonathan certainly is professional.

[pobrien3]

Of course a RAID would do the trick too - if you can spare the money and space (be aware of the noise and the heat-generation!!  A friend of mine works as a system operator for a middle sized company and says the raid arrays produce so much heat that they need to be in an special climatized room).

At home, I have a total of approx 8Tb of disk storage (not all of it online) in external racks, and operate RAID 5. It generates only as much heat as you would expect from the number of disks you have 'live' - RAID in itself generates no heat.  This is not an expensive option: you can buy the requisite hardware online.

[Concorde-SST]

pobrien,

about the RAID + heat - of course I meant the harddisks! They do
produce some heat - especially the ones with high spin rpm and
since they have a BIG RAID (my friend can´t tell it for security
reasons, how big and how it is exactly built in his company).

How did you construct your RAID? RAID 5 is the one in which
you have more than one copy?

Also - I´d like to know how you "archive" your non-live disks -
just leave them in the stack or store it somewhere "safe"?

a great weekend,

Andreas.

[kenstrain]

A variation on Jonathan's suggestion: I have a local RAID array (0.8TB RAID in the PC) and do incremental weekly backups to one of two sets of external drives. These are kept in buildings a few miles apart.  The drives are replaced every 18 months with ones at the price sweet-spot (~250GB now) and the old drives stored for 18 months too. (So far I've found a use for all the old drives after that period.)  This approach is not very expensive (I'm backing up about 0.5TB at present.)

Big RAID arrays are great if done well.  At work (not photography) we learned a lesson when a few TB RAID5 array controller died and took out the whole file system.  12 hours later the hardware was fixed under contract, 12 days later the system was up and running (we did not have recent backups).  Also our disk failure rate (hitachi, but I don't think that matters) is high compared to MTBF.  We suspect poor cooling in the RAID box. If you use RAID as the final resort do it well.

Ken

[David Mantripp]

whatever.  There is too much opinion stated as fact from some people here, and that is a disservice to fellow forum participants.  Dell & Amazon may well run multi-million air conditioned multi terrabyte multi redundant backup RAID arrays in offsite data warehouses, but I'm not totally sure how relevant that is to a single amateur, semi-pro or pro photographer looking for a reliable, scaleable and affordable data archival system, which at the same time doesn't need a data tech to run it.  I'm not sure what the point about speed is here. So what if it takes all night to do a backup ?  If you want a flexible strategy, by all means keep copies of important files on seperate disks, or indeed a small RAID sytem.  There are also some fairly breathtaking views on what constitutes "cheap" around here, but then again I wouldn't be surprised if the majority of Canon 1Ds' are used to take photos of cats and grandchildren.

I think the original point of this thread was to investigate tape storage as an option. It is a viable option, and it verges on arrogant, and near-total (I'll concede a fraction) rubbish to arbitrarily rule it out.

[kenstrain]

Whatever, David?

I thought the point of the thread was to avoid 200 DVDs, with tape as the method that had been "heard about".

Is it too unhelpful to suggest alternatives that seem to work?

Natually my earlier post was factual. I agree with your point about RAID, which is why I mentioned the RAID problem and the lesson we learned.  (We now have an off-site tape backup.)  As you imply, the relevance of any of this to an individual depends on their specific requirements, speed, cost per GB etc. But I suggest the best answers to those points may be found on fora dealing with mass storage (I'm sorry, but I don't know of one I'd trust enough to recommend.)

When I looked at tape storage as an alternative I was not convinced tht it was a good long-term solution for my modest needs.  Spinning media seemed the best compromise given the extremely low cost and high speed, that won't be true for everyone.

If you were not criticising my post you could have been more specific (no offense taken, of course).

Ken

[Concorde-SST]

David,

I think you are exaggerating a little. That some people
(myself included) veer off somewhat from the original
topic is in my opinion normal and good - at least I find
this. I now have a greater grasp over this background.

It helped me and hopefully other non-responding readers
to shape their eventual decision for a "safe" backup solution.

And yes, I do take besides of my professional assignments
photos of my family and my late terrier dog and am proud
to have high quality photos of those who are my family
and people whom I care most of with the best camera I
have.

"Rubbish" is a quite stark choice of describing some
people´s lines - I think its not polite but thats all
about talking to each other to find a solution to a
question I started.

Remember, there are no rubbish questions, only
rubbish answers. :-)

Anyway, no offense taken on my part.

Have a good weekend!

Andreas
Nuernberg,
Germany.
:blues:

[David Mantripp]

Ok, ok, I'll shut up. Sorry Jonathan. I'm a bit grumpy due to an enforced stay at home after an unscheduled visit to hospital earlier this week. Sorry.

[Concorde-SST]

David,

no problem, that could happen to me too! Sorry to hear of your
difficulties - get well soon!! After all, we´re just humans with
too much technical things around us....

all the best,

Andreas.

[Concorde-SST]

Just for the general information:

here´s the link for LaCie´s RAID S-ATA Array:

LaCie

best,

Andreas.

[Jonathan Wienke]

Jonathan, I do sometimes wonder how you managed to become an expert on everything under the sun. I wish I had your IQ and memory.

Prior to doing professional photography, I spent 7 years as the head programmer and network andministrator for a marketing company, and had hands-on experience with a few different tape backups. I had several unhappy experiences attempting to restore backups from tapes that were unreadable only a few months after being written, as well as the hassles associated with automated backups erroring out due to tape write errors. Eventually I gave up on tape altogether, and built a pair of RAID servers that replicated the data to each other, and got a drive enclosure to regularly back up critical data to portable hard drives for off-site storage. This pretty much eliminated the problems associated with the tapes.

[61Dynamic]

Could this happen to HDD´s too?

No. HDs store information on metal patters and they do not come in contact with one-another. HDs do not "bleed."

Do they lose magnetization

Yes but it would take a good 400 years (from what I remember off-hand) or more before that would happen.


Look on the bright side DMR, you don't have to stay in the hospital and eat that cheap green jello.

[pobrien3]

Andreas, I have an on-board RAID controller in my PC and four external 8-bay SCSI racks. Only one of the racks is 'live' at a time, disks can be hot-swapped within the rack in the event of failure.  I mostly use Seagate 350GB disks in the newer racks: best to use identical disks throughout. I've been using this setup for about four years, and in that time have only had two, separate disk failures (both 200GB 10,000RPM Maxtors, oddly enough). I was able to simply replace them, and nothing was lost. The controller and enclosures are Taiwanese-branded items I have only ever seen for sale in HK, China and Taiwan, and they have been very reliable.  I don't store copies off site as I'm not commercially dependent on the safety of this stuff, but I would be unhappy if I lost anything!  I also store video as well as photos on there, so I need a lot of space.

I have also had too many occurences (in work) of being unable to get tape backups restored, so there's no way I'd use them for my personal stuff. I've also discovered to my cost that CD and DVD media are not as robust over time as we were once led to believe, so for me HDD storage is the only option for now.

RAID 5 isn't quite mirroring - have a look at the table on http://www.midwestdatarecovery.com/raid-array-types.html to see the differences.  You need a minimum of three disks for it; maximum depends on the specifications of your controller.

My biggest complaint about my setup is the noise from the SCSI disks and fans. If I were to re-build this today I'd look at newer setups you can get using S-ATA, and perhaps find a better quality case with improved noise control.

PS - I took a picture of my little girl holding her cat with my 1DsMkII over the weekend - should I expect a visit from the authorities? ::
PPS - David, sorry to hear you've been below par - I wish you good health.

[David Mantripp]

seems to be some confusion here between mirroring and archive.  I would have said that rAID mirroring is appropriate when you want 24/7 access to data / services, but it doesn't help you if you accidentally delete a file.  Archiving on the other hand does.

I'm not going to say anything else on this topic, apart from to observe that tape backup, like everything else, can be done well or badly, and just because you throw a heap of money at it it doesn't mean it's done well. Not including redundancy in a tape backup strategy is living very dangerously.  Also, in over 10 years, when I was using professionally in large volume, I never had a DAT (DDR) tape failure. Ever. I did have a couple of drive failures though (Sun units). No big deal, from the point of view of data integrity

[Lust4Life]

Greetings,
A word of caution on the Mac G-5 OS 10.xxx - there is reportedly an engineering oversight in the hardware as it relates to Firewire writes - they are VERY slow compared to putting in a SCSI card and going to an external RAID - then run SATA drives in the RAID cabinet.
In short, avoid Firewire external drives.
Jack

[David Mantripp]

Can you provide any hard evidence for this ? I'm using external FW400 & FW800 drives on a Mac G5, and I can't say I notice them being particularly slow...

[Concorde-SST]

Jack,

well, I don´t agree with your opinion. It could be - since I´m not particularly
good at this depth of technicality.

My FW800 and 400 drives are extremely fast, I regularly have 88MB/s transfer
rates from the 800 drive - I don´t consider that as very slow. Maybe you´re
used to an array of SCSI320 RAID stripe set?!

best,

Andreas.  :blues: