Pages: [1] 2   Go Down

Author Topic: External HD v. DVD for Archiving  (Read 12458 times)

Jonathan Wienke

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 5829
    • http://visual-vacations.com/
External HD v. DVD for Archiving
« on: April 23, 2005, 05:36:25 pm »

I use external HDs; CD/DVD is too capacity-challenged to be practical in many instances. I have an external USB enclosure with a drive dock so I can easily swap drives in and out of the enclosure. To add capacity, I simply buy an additional cheap IDE drive and a $12 dock tray to mount it in.

As to the aliens...
Logged

Tim Gray

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 2002
    • http://www.timgrayphotography.com
External HD v. DVD for Archiving
« Reply #1 on: April 24, 2005, 11:46:52 am »

USB isn't the issue - I can easily remove the internal IDE drives and plug them into something else - firewire if that's what I want.  The issue is what if IDE's aren't supported at some time in the future?  Any technology solution will be obsolete someday but I'm happy to worry about that when the time comes.  I'm not archiving for future generations - I'm only archiving for the period of time (next 20 - 25 years) when I'll give a dam if I loose the data.  I tend to upgrade my hardware every 2 or 3 years - that should keep me current engough, as long as I updarade (if necessary) the archive technology at the same time.
Logged

61Dynamic

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 1442
    • http://
External HD v. DVD for Archiving
« Reply #2 on: April 24, 2005, 06:43:00 pm »

Quote
As to the aliens...
Good thing Jark doesn't have a LL account otherwise he would probe you...

Quote
I'm not protecting against fire or theft since I keep them at home - #1 risk I'm managing is drive/media failure.

I've seen hard drives that have survived house fires. The electronics get fried, but it's possable for the platters to come out unscathed and the drive rebuilt. Of course you'll pay a couple grand to have the thing rebuilt from a service like DriveSavers.

Quote
The issue is what if IDE's aren't supported at some time in the future?  Any technology solution will be obsolete someday but I'm happy to worry about that when the time comes.

Luckfully IDE has been around for such a long time that I'm sure there will be a way to get the drive working one way or another for some time to come. It certainly could go obsolete, but it's not as likely to be as difficult to get going as an interface that hasn't been around as long would be.

Quote
Is there any substance to this claim or is it pure BS based on the fact that there will be no one around in 100 years to complain or complain to?

Probably BS. The question is, how did they test that if they did at all? Untill some third-party tests DVDs similar to how CD-R/RWs have been tested, DVD longevity is nothing but speculation.
Logged

Stealthfixr

  • Newbie
  • *
  • Offline Offline
  • Posts: 25
External HD v. DVD for Archiving
« Reply #3 on: April 24, 2005, 11:19:05 pm »

I keep images on my primary desktop computer 300GB HD, and on a 300GB backup external HD, and burn to DVD.  If I move the images off of HD's altogether, then I burn them to another (2nd) DVD as well.
Logged

firestarter

  • Newbie
  • *
  • Offline Offline
  • Posts: 1
External HD v. DVD for Archiving
« Reply #4 on: April 28, 2005, 08:44:36 pm »

Quote
My next HD purchase will be one of the new LaCie 1 TB units.
What, one of those big LaCie 1TB drives that are actually 4 low-quality Maxtor mechanisms in one box?

That use disk striping to increase the access speed (data is spread across all 4 disks) but at the same time quadrupling your risk of disk failure? (Loose one disk, loose them all).

One of those wouldn't be my choice of a backup medium.
Logged

nochance44

  • Newbie
  • *
  • Offline Offline
  • Posts: 3
External HD v. DVD for Archiving
« Reply #5 on: April 23, 2005, 12:13:39 pm »

My new 160GB external HD is a delight after using hundreds of CD's for image storage and never bothering to browse through to see what I might want to work on.  My intent was to start with the external HD and then add a DVD writer for duplicate archiving.  Now I'm wondering about just adding a second external HD duplicating the first instead of buying DVD writer.

My question regards the reliability of external HD's v. DVD's for long term archiving.

Thank you for any information.

Bob
Logged

Gabe

  • Full Member
  • ***
  • Offline Offline
  • Posts: 146
External HD v. DVD for Archiving
« Reply #6 on: April 23, 2005, 12:45:07 pm »

I'll second Bob's advice -- I like HDs because they make it easy to move the data around really quickly

OTOH those aliens can be pretty resourceful.. best practice would probably be to make friends and let them keep a copy
Logged

Bobtrips

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 679
External HD v. DVD for Archiving
« Reply #7 on: April 23, 2005, 06:49:22 pm »

So you keep a set of HDs in a separate physical location and rotate drives for fire safety?

(Used to that in the olden days when the only affordable way to back up was on video tape.  A,B,C copies.  Always one in the second locale.)
Logged

ausoleil

  • Newbie
  • *
  • Offline Offline
  • Posts: 38
External HD v. DVD for Archiving
« Reply #8 on: April 24, 2005, 11:30:52 am »

I will agree with Tim, save for one caveat:  external USB drives are like any technological solution:  you need to keep the solution holistic to protect against the future, and you need to evolve your backup mass storage as contemporary technology moves away from the techniques you are using today.

What does that mean?

From Part 1 Of a Backup Discussion On My Blog

USB is a wonderful thing -- but it is not guaranteed to be around forever.  Even now, the move towards Firewire is gathering steam, and eventually, it is reasonable to say that something even beyond USB will overwhelm USB and eliminate it from the market.

That will not happen today, or even in the next couple of years, but it is a fact that the past is littered with orphaned and obsoleted technologies.  Consider 5.25" floppies -- do you ever see a computer sold today with one included?  How rare are the drives?  They were state of the art in the 1980's, and lots of data was archived upon them then.  Now, if you need to access the data, you will have a lot of trouble finding a computer to access the files and import them into a contemporary form.

Further, consider two obsoleted hard drive technologies:  MFM and RLL.  If you have a RLL drive today, you simply CANNOT mount it in a modern computer.  IDE passed them a long time ago.  IDE is now evolving towards SATA and I predict in ten years, an IDE drive will be orphaned by whatever has takenm over the marketplace.

Obviously, images are longer lasting than any of that.  You will want to see those photos you took at Sentinel Dome in Yosemite twenty years from now.  Sentinel Dome will change and you will want to remember the picture you took of the tree that's now gone, for example.  I use this because I took a "past and future" photo of the dead carcass of the Jeffrey pine tree that Ansel Adams made in the 60's.  It is gone now and the photograph is irreplaceable:  you cannot photograph what is not there.

The same thing will happen to USB.  That is why there is a need to preserve a holistic solution.  For example, I have an old IBM XT computer with a 10 MB harddrive that has one of the first ehternet cards in it.  The XT also has two of the aforementioned 5.25 inch drives.  I use it from time to time, perhaps twice a year, to export data through the ethernet card and onto CDR for a customer.  I charge $10-$20 a disk, depending on if I have to also break out old software that can 'rescue' a damaged 5.25 floppy.

People could and should have spared themselves that cost and trouble by copying their info to 3.5" floppies when they overtook the older 5.25" inch ones in the late 80's and early 90's.  Then, they should have moved that data onto CDR when it more or less overwhelmed the 3.25 floppy.  Data preservation is an ongoing thing, and expect today's solution to need upkeep through updating tommorow.

And for those of you who do not, I have a Pentium 4 computer already stored away with a stash of software to save your data in ten years.  :D

Gone are the days of preserving images via their negatives in inert sleeves.
Logged

ausoleil

  • Newbie
  • *
  • Offline Offline
  • Posts: 38
External HD v. DVD for Archiving
« Reply #9 on: April 24, 2005, 12:23:29 pm »

Tim, read the part about MFM and RLL drives.  Trust me, I make a nice living rescuing data for people.  I also backup data for a 2 billion dollar business segment for a Fortune 100 company.  You are very wise to keep your computers up to date, as well as using two drives.  I just wish people were more careful about backups as you are...I hear a very sad (read: expensive) story at least once a week.

And oh, by the way, don't ever drop a drive.  I'll almost guarantee you a DOA drive if you drop it from waist level, on or off.

By the way, the failure rate of hard drives is a well established and very well known number:  it is precisely 100% with a six sigma of 0.00 percent.  :p

Finally, something else to keep in mind is that new software tends to orphan file formats over time.

Obsolete File Formats

Quote
I got a bit of a shock last night while pulling the last bits of my digital image collection into a Photoshop Elements catalog. PS Elements lets you browse to a location on disk and pull in all the image files it finds, after which you organise them as appropriate. So, I dropped a load of images into a folder and pointed PS Elements at it. It didn't find any image files it understood. What? Why not? These were images from a PhotoCD I had made years ago, and the PS Elements organiser doesn't understand the PhotoCD file format, apparently. Fortunately, the editor itself does understand the format so a batch conversion soon dealt with the problem, but it seems the writing is on the wall for the PhotoCD file format.

The morale of this story? Don't write stuff to CDs and assume that as long as you have a working CD drive you'll be able to read the contents. Periodically check that the files on your archive CDs can still be read by your current software of choice. And even if they can, keep a copy in a different format just to make sure.
Logged

ausoleil

  • Newbie
  • *
  • Offline Offline
  • Posts: 38
External HD v. DVD for Archiving
« Reply #10 on: April 25, 2005, 01:22:14 pm »

The majority of hard drive failures are mechanical in nature.  A drive not spinning and correctly stored is safe from all but fire, flood and theft, which realistically are the genuine threats to archival drives.

Compared to tape, hard drives are rapidly becoming a replacement option.  Tape drives are exhoribitantly expensive, as are the tapes themselves.  I am lucky in that I got a decent tape drive from E-Bay cheap and recovered several dozen tapes from my company when they obsoloeted a drive technology.  Problem is, even with 60GB tapes, it is hard to adequately back all of my photos up -- so I use the tapes for system recoveries and for data backups where the data are not alrwady committed to another archival schema -- in other words, stuff I am working on now.

Hard drives with mechanical failure can still be retrieved.  There are companies that remove the platters (the internals of the hard drive where the data are actually stored) and mount them in working drives.

Don't try this at home, since you need a clean room to ensure success.  And bring your Visa card -- it ain't cheap.

OnTrack -- one of many

If that fails, for example, with a fire, the drive platters can be mounted in a scanning electron microscope and restored.  This is possible, mind you, but it is beyond the means of an average photographer and is almost exclusively the provenance of criminal computer forensics.

As for the lifespan of media, this is actually testable and there are procedures for it.  Add to it, the data generated by the simulations match pretty well with real-world experience.  This is a routine QA check for almost any major media manufacturer.

Description of NIST Testing Protocol

The bottom line is that the prudent photographer will never have anything to worry about, and those that are not will.  IDE is less troublesome than SCSI, as pointed out above, but when the industry moves away from it and the equipment using it obsoletes, it will be time to move on and not count on a grandfathered solution.

The more I think about what Tim said about online storage the more I like it.  Since I am about to have a couple servers colocated, I may have to put a couple 320's in one of them and do a simple offsite mirror.
Logged

Ray

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 10365
External HD v. DVD for Archiving
« Reply #11 on: April 27, 2005, 11:49:15 pm »

Last time I checked, recording stuff on DVD was definitely the cheapest option. I've also been worried about eventual obsolescence of the IDE HD format. Oddly enough, over the past 11 years or so that I've been doing digital photography, I've had more hard drives fail than CD discs.

As an amateur I've no doubt got a different perspective than some of you. The best of my slides taken years ago were first scanned by Kodak. It wouldn't worry me if any of those became unreadable because I rescanned them all myself some time later when I got my first scanner and realised I could do a better job than Kodak had done.

When I subsequently upgraded my scanner to a 4000dpi unit, I rescanned them yet again. I'm currently transferring stuff on older CDs to DVDs and I would expect in just a few years from now when the high definition DVD format becomes an affordable reality, that I'll transfer the whole lot to that new format.

I wonder if one can get too worried about this issue. When in Italy, at the end of each day, I burned all my RAW images on CD, with my laptop. I then worried about the possibility of my laptop getting stolen and being left with just one copy of the images, or possibly both the laptop and CDs stolen, so I started burning 2 copies of the images. I'm now in the process of transferring all those images (about 45 CDs) to DVD, which effectively means I've already got 3 copies of each RAW images before I've even started processing them.
Logged

Tim Gray

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 2002
    • http://www.timgrayphotography.com
External HD v. DVD for Archiving
« Reply #12 on: April 24, 2005, 12:50:08 pm »

Maybe the issue isn't the life of the media/technology - it's the narrowness of our current bandwidth.  I can imagine a not too distant future where I'll pay to "vault" my images and just upload to a remote commercial site.
Logged

Robert Spoecker

  • Full Member
  • ***
  • Offline Offline
  • Posts: 162
External HD v. DVD for Archiving
« Reply #13 on: April 24, 2005, 03:49:07 pm »

I'm glad I'm old enough not to have to worry about extremely long archiving.  I'm backing my raw files on Tayo Yuden DVD's.  The hype is that these are archival with an "expected" life of at least 100 yrs. under "reasonable" conditions.  Is there any substance to this claim or is it pure BS based on the fact that there will be no one around in 100 years to complain or complain to?  I really do like 10 cents per gig for a hassle free storage option and I've experienced no glitches so far.  My converted working files go on hard disk (duplicate) for easier access, etc.
Logged

sergio

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 666
    • http://www.sergiobartelsman.com
External HD v. DVD for Archiving
« Reply #14 on: April 24, 2005, 09:03:29 pm »

Using HDs as archival storage is simply not safe enough for me. IF the drive fails, 250 gbs worth of images are gone forever. HDs are good for fast retrieval. The archival storage thing is an unresolved problem, at least in an efficient and economical way.

 
Logged

Bobtrips

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 679
External HD v. DVD for Archiving
« Reply #15 on: April 24, 2005, 10:20:55 pm »

I'm with didger.  Images on two different HDs.  And two CD backups (trying to get my d### DVD burner to work).

One set of CDs is about 40 miles from here.  The second set is about 250 miles away.

Remember the JFK photos what were lost on 9/11?  Stuff happens.  Houses burn down.  Angry "next ex-wives" can wreak havoc.

But for those who don't want to mess with DVDs, how about a 'buddy system'?  Keep a HD at a friend's house and keep one of theirs at  yours.  Do downtime backups over the net.
Logged

Bobtrips

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 679
External HD v. DVD for Archiving
« Reply #16 on: April 23, 2005, 12:37:01 pm »

My advice?

Both.

External HD for convenient backup.  

DVD for safe storage far from your computer/house.  Gives you the theft/fire/alien abduction protection that you don't achieve with a hard drive sitting on your desk.
Logged

Tim Gray

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 2002
    • http://www.timgrayphotography.com
External HD v. DVD for Archiving
« Reply #17 on: April 24, 2005, 08:18:21 am »

I switched to a pair of 250 gig Seagate USBs after I noticed some of my "archive" cds starting to deteriorate - even some commercial rescue software couldn't retrieve some of the files on some of the disks - I estimate I lost about 50 images over 85 cds.  I use Syncback to keep the drives in sync.  I'm not protecting against fire or theft since I keep them at home - #1 risk I'm managing is drive/media failure.
Logged

ausoleil

  • Newbie
  • *
  • Offline Offline
  • Posts: 38
External HD v. DVD for Archiving
« Reply #18 on: April 24, 2005, 02:40:39 pm »

Tim, that's too far a throw.  Many a company is already doing stuff like that, and even are outsourcing "disaster recovery" systems such that they have software and data stored offsite.

Seeing those firms move into the consumer marketplace is not at all a far stretch, but until then, data backup strategies that folks can depend on now are prudent.  Yours makes a lot of sense, but folks with piles of CDs and nothing else are almost guaranteed to be sorry.  They and the folks who are not transcoding.
Logged

didger

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 2030
External HD v. DVD for Archiving
« Reply #19 on: April 24, 2005, 09:44:01 pm »

Quote
IF the drive fails, 250 gbs worth of images are gone forever
No matter how you archive, you're out on a limb if you don't make two copies.  I do that for my hard drive material and DVD backup.  That means I actually have 4 copies of every image, two hard drive backups for converted images and two raw DVD copies.

A bit OT, but incidentally, back when we had our big earthquake here some years ago, I had a big heavy clunky 1GB external drive (at that time about max size and worth about $2000) and it fell onto bare concrete from about head level and it kept right on working.  How many times could you get away with that?

If a hardrive fails and the material is important enough, you can generally have most of the data recovered, but that takes a cleanroom facility and gets very expensive.  Better to do dual backups.
Logged
Pages: [1] 2   Go Up