Luminous Landscape Forum

Equipment & Techniques => Computers & Peripherals => Topic started by: nemophoto on January 27, 2008, 02:57:39 pm

Title: Next step for archive storage
Post by: nemophoto on January 27, 2008, 02:57:39 pm
Years ago, I use to be able to store entire shoots (I often shoot week long catalogs, as well as advertising and editorial) on a few CD's. When those started to grow to eight or ten disks, I started using DVD's. Then, I migrated to DVD+R DL disks. Now that I own both verisons of the Mark III, I'm averaging 20GB/day in just RAW files. (I just returned from a shoot with 97GB to archive off a pocket drive.)

Has anyone explored the 25GB Blu-ray disks and drives? Are they worth it, or should I give in and buy more external hard drives? (I already have three that I use regularly for images I consider "active".) I thought of editing each of my shots down to 20-30 images to save space, but that's almost as time consuming as backing up the images. My thoughts are that the Blu-ray disks physically take up less space than yet another hard drive (as well as yet another power adapter on a hard drive, though it would only be connected while backing up). At the end of a year, that's a lot of hard drives sitting on the shelf.

Thanks for the thoughts.

Nemo
Title: Next step for archive storage
Post by: Chris_Brown on January 27, 2008, 03:20:09 pm
Have you considered a tape drive (http://store.apple.com/1-800-MY-APPLE/WebObjects/AppleStore.woa/wa/RSLID?s=priceHL&fnode=home/shop_mac/mac_accessories/storage&nplm=TM191LL/A&mco=EECD6583) with backup software?
Title: Next step for archive storage
Post by: Jonathan Wienke on January 27, 2008, 08:55:57 pm
Quote
My thoughts are that the Blu-ray disks physically take up less space than yet another hard drive

Nope. By the time you put them in protective cases, even the slim ones, 10-20 Blu-Ray discs are going to take up more space than an 3.5-inch hard drive. A lot more.
Title: Next step for archive storage
Post by: Farkled on January 29, 2008, 03:59:27 am
I'm not going to trust any optical medium for more than a year and mostly not at all.  Same goes for tape.  If that solves your problem, then go for it.  Don't forget the verify process after a write to either tape or optical.
Title: Next step for archive storage
Post by: Ray on January 29, 2008, 07:00:13 am
Quote
My thoughts are that the Blu-ray disks physically take up less space than yet another hard drive (as well as yet another power adapter on a hard drive, though it would only be connected while backing up). At the end of a year, that's a lot of hard drives sitting on the shelf.
[a href=\"index.php?act=findpost&pid=170025\"][{POST_SNAPBACK}][/a]


Nemo,
It's amazing how competitive these different methods of storage can be. I think Johnathan is right that the weight and bulk of current one-sided recordable Blu-ray discs is going to be greater than using the latest WD Passport pocket drives, but perhaps by not as much as he thinks.

I don't think it's necessary to put either DVD or Blu-ray blanks in individual plastic cases. I use 24 disc wallets with plastic sleeves. The wallets cost about $2, made in China, and when full of discs you have something about the weight and bulk of a novel made from quality paper.

My WD pocket hard drive holds 200GB, weighs little more than 100gms and measures about 5"x3"x1/2". It's a little beauty in piano black finish. It cost me less than $1 per GB. I see that Western Digital now have a 320GB model, similar size and weight, that costs around 80 cents per GB.

I believe TDK are selling one-sided Blu-ray blanks for about $20. 24 of them will give you about 600GB of storage and weigh as much as 4 of these pocket hard drives with about the same bulk. Two of these 320GB pocket drives will give you about the same storage as 24 Blu-ray discs, and oddly enough will cost about the same price, ie. $20x24=$480 for Blu-ray, and 600x80 cents=$480.

Net result; same price but double the bulk and weight for the Blu-ray discs.
Title: Next step for archive storage
Post by: Jonathan Wienke on January 29, 2008, 09:58:50 am
Quote
It's amazing how competitive these different methods of storage can be. I think Johnathan is right that the weight and bulk of current one-sided recordable Blu-ray discs is going to be greater than using the latest WD Passport pocket drives, but perhaps by not as much as he thinks.

Not when you consider that 3.5" drives are now available in 500GB and 1TB capacities. 40 discs, even if not in any sleeve or case or wallet, are going to take up far more space than a 3.5" drive, even in a bulky ruggedized enclosure.

Quote
My WD pocket hard drive holds 200GB, weighs little more than 100gms and measures about 5"x3"x1/2". It's a little beauty in piano black finish. It cost me less than $1 per GB. I see that Western Digital now have a 320GB model, similar size and weight, that costs around 80 cents per GB.

I believe TDK are selling one-sided Blu-ray blanks for about $20. 24 of them will give you about 600GB of storage and weigh as much as 4 of these pocket hard drives with about the same bulk. Two of these 320GB pocket drives will give you about the same storage as 24 Blu-ray discs, and oddly enough will cost about the same price, ie. $20x24=$480 for Blu-ray, and 600x80 cents=$480.

Net result; same price but double the bulk and weight for the Blu-ray discs.

And that's comparing to 2.5" drives which cost more per GB than 3.5" drives. With 3.5" drives, the cost advantage swings in favor of the hard drives as well as bulk. Toss in the uncertainty of the longevity of Blu-Ray writable discs vs hard drive, and IMO the choice is a no-brainer.
Title: Next step for archive storage
Post by: Ray on January 29, 2008, 11:25:47 am
Quote
Not when you consider that 3.5" drives are now available in 500GB and 1TB capacities. 40 discs, even if not in any sleeve or case or wallet, are going to take up far more space than a 3.5" drive, even in a bulky ruggedized enclosure.
And that's comparing to 2.5" drives which cost more per GB than 3.5" drives. With 3.5" drives, the cost advantage swings in favor of the hard drives as well as bulk. Toss in the uncertainty of the longevity of Blu-Ray writable discs vs hard drive, and IMO the choice is a no-brainer.
[a href=\"index.php?act=findpost&pid=170585\"][{POST_SNAPBACK}][/a]

I believe the 3.5" external drives which normally include a separate AC/DC converter and power cord would be both heavier and bulkier than the equivalent Blu-ray storage, but no doubt cheaper. At least that's the situation with my external 500GB hard drives, a LaCie Big Disk and a WD. They are each about as bulky as a 96 disc wallet. 96x25=2.4TB, so you've basically got twice the bulk and weight at probably half the price.

My LaCie Big Disc is heavier than a 96 disc wallet but I think my WD 500GB drive is lighter. But 2 or 3 of them, which is what you'd need to equal 2.4TB would be heavier.

By far the lightest and most compact storage would be the WD Passport pocket drives. They just plug into the USB socket on the laptop. But you pay for the miniaturisation.

When double-sided recordable Blu-ray discs become available about June this year, each storing about 50GB, that 96 disc wallet with the bulk and weight of a Photoshop manual (perhaps not even as heavy) will store a staggering 4.8TB. That should be enough for the most prolific photographer   .
Title: Next step for archive storage
Post by: Jonathan Wienke on January 29, 2008, 12:23:18 pm
Quote
I believe the 3.5" external drives which normally include a separate AC/DC converter and power cord would be both heavier and bulkier than the equivalent Blu-ray storage, but no doubt cheaper. At least that's the situation with my external 500GB hard drives, a LaCie Big Disk and a WD. They are each about as bulky as a 96 disc wallet. 96x25=2.4TB, so you've basically got twice the bulk and weight at probably half the price.

You must have some pretty wacky enclosures then. I have a Buffalo TeraServer NAS with 4 3.5" drives in it that is less bulky than a 96-disc CD case. with 1TB drives I can have 3TB of RAID5 storage with protection against drive failure. And you're still not addressing the long-term viability of writable BluRay discs, which is a questionable proposition at best.
Title: Next step for archive storage
Post by: Ray on January 29, 2008, 01:58:00 pm
Quote
You must have some pretty wacky enclosures then. I have a Buffalo TeraServer NAS with 4 3.5" drives in it that is less bulky than a 96-disc CD case. with 1TB drives I can have 3TB of RAID5 storage with protection against drive failure.[{POST_SNAPBACK}][/a] (http://index.php?act=findpost&pid=170645\")

Why would you think LaCie and Western Digital products are wacky? The drives came with their own enclosures which are about the same volume as one of my 96 disc wallets, maybe a bit slimmer. If what you've got is similar to the products on this link [a href=\"http://www.beachaudio.com/Buffalo-Technology/Hs-Dh1000gl-p-120504.html]http://www.beachaudio.com/Buffalo-Technolo...l-p-120504.html[/url]
then it weighs at least 1.5Kg without power adapters, and I believe those contain just one 3.5" drive. Put 4 in the case and you've definitely got something heavier than a cloth, zipped wallet containing 96 Blu-ray discs.

But as I said, it certainly looks like the cheaper alternative.

Quote
And you're still not addressing the long-term viability of writable BluRay discs, which is a questionable proposition at best.

I've addressed this issue, with regard to CD and DVD longevity numerous times. I've never had any problems, except with some burners that very occasionally won't read data written on a different burner. Changing the burner or using another burner has fixed the problem.

If you follow some basic principles of common sense, you should be all right. But of course there are no guarantees.

(1) Always verify the data after it's written.

(2) Store the discs in an environment free of strong chemical odours.

(3) Do not expose the discs to extremes of temperature and humidity and especially avoid leaving them out in the sun or baking in a car.

(4) Avoid using adhesive labels, unless you are very sure the chemicals in the adhesive will not react with the protective coating on the disc. I never use adhesive labels.

(5) Always use a soft tipped marker designed for writing on optical discs.

(6) If a disc fails that you know was recorded properly, do not fly into a fit of rage and assume that the disc is suffering from bit rot and that the manufacturer should be sued. Check to see if the disc reads okay in another drive or two.

I could add, always buy the most expensive and reliable media, but the fact is I haven't done that and I've still had no permanent loss of data, just occasional minor hiccups of drive incompatibility.

Personally, I think those WD Passport pocket drives are ideal for travelling light. I don't think you'll get 320GB in a smaller, lighter package. Just 100gms for 320GB of storage. Amazing!
Title: Next step for archive storage
Post by: luong on January 29, 2008, 02:30:17 pm
Burning DVDs is a waste of time when the HDD are less than 20 cents per GB.
Title: Next step for archive storage
Post by: Ray on January 29, 2008, 09:32:39 pm
Quote
Burning DVDs is a waste of time when the HDD are less than 20 cents per GB.
[a href=\"index.php?act=findpost&pid=170693\"][{POST_SNAPBACK}][/a]

You mean it's a waste of money. The cost of hard drive storage has nothing to do with the time it takes to burn a DVD. But actually, DVDs now cost less than 20 cents a GB.

However, the OP's question has been answered. Current one-sided Blu-ray discs do not physically take up less space than the most compact pocket hard drives, but the double sided ones would probably be lighter and more compact, but more expensive, than a desktop external 3.5" HD with power adaper.

Unless it's your policy always to have a back-up of RAW images on optical media, I think a couple of those 320GB pocket drives are ideal. When you get back to base, you can transfer the images to a more substantial RAID system, whatever.
Title: Next step for archive storage
Post by: Jonathan Wienke on January 30, 2008, 09:24:56 am
Quote
Why would you think LaCie and Western Digital products are wacky?

The 3.5" single-drive enclosures I'm familiar with are about the size of a 5.25" CD-ROM drive, plus the power brick. The unit I have is an older version of the Buffalo TeraStation (http://www.buffalotech.com/products/network-storage/terastation/terastation-live/), which is about the same height and width as a stack of 96 discs, but is deeper and of course rectangular, approximately double the volume, but has an internal power supply so no brick. It holds 4x3.5" drives and offers RAID5 drive failure protection and gigabit ethernet connectivity. Given the instant availability of the data (no digging through the discs to find the right one), the inordinate amount of time involved in burning, labeling, indexing, and filing the discs, they make little sense as a backup option.

Time is valuable, wasting time is no different than wasting money, especially in a business setting.
Title: Next step for archive storage
Post by: luong on January 30, 2008, 01:09:42 pm
Quote
You mean it's a waste of money. The cost of hard drive storage has nothing to do with the time it takes to burn a DVD. But actually, DVDs now cost less than 20 cents a GB.
]

No, that's what I meant. HDDs are and has always been by far the most efficient storage technique, but when the price was so much higher than other media, one could hesitate to use them. Now that the price has fallen that low, no matter the price of other media, there is little reason not to use them if one values his time only a little bit. Time is money anyways :-)
Title: Next step for archive storage
Post by: Ray on January 30, 2008, 10:48:24 pm
Quote
Time is valuable, wasting time is no different than wasting money, especially in a business setting.
[a href=\"index.php?act=findpost&pid=170966\"][{POST_SNAPBACK}][/a]

Quote
No, that's what I meant. HDDs are and has always been by far the most efficient storage technique, but when the price was so much higher than other media, one could hesitate to use them. Now that the price has fallen that low, no matter the price of other media, there is little reason not to use them if one values his time only a little bit. Time is money anyways :-)

Well, of course. One can't argue with the fact that storage on a hard drive is more efficient. If lower cost was the only reason to burn to DVDs, then any current cost advantage of DVDs (Blu-rays have no cost advantage) does not warrant the extra stuffing around of organising folders into 4.2 GB packages and labelling discs.

A reason for archiving on optical media is to have a secondary back-up which one can store in a separate place away from one's studio, and more or less forget about it. A sort of insurance package in the event one's studio gets a direct lightning hit or burns down or gets burgled.

Whether or not you think this is worthwhile will to some extent depend on your past experience. Since I have never had the experience of discovering that a 10 year old CD that once was readable is no longer readable, I feel more comfortable having all my RAW images on DVDs. I don't feel comfortable about leaving a hard drive on a shelf for 10 years and then expecting it to work flawlessly.

On this current trip, I'm burning my RAW files to DVD first then copying the data to to my 200GB pocket drive before deleting them from my lapatop. When I get back to base, I'll probably confirm the fact that those DVDs are readable by transferring the data from them to my main hard drive. I'll then put them in storage.
Title: Next step for archive storage
Post by: JessicaLuchesi on January 31, 2008, 08:32:35 am
The main thing I think, is not so much volume, but rather backup safety. If I were going to go HD, I'd go for a digital storage solution ( possibly, an expansible HD rack, with as much data redundancy as possible, electrically shielded from damaging factors ), not just record into postable HDs and shove into an archive. If you want to register stuff and store into an archive type storage, as you did with negatives, I think it's still safer to go towards media that can last 50 years.

I say that, because right now, if I had photos in an old IDE Hard Disk from 1998, I guess I'd have many problems reading it... it could fail, being so long turned off, like old electronics, it can go in the first motion after so many years in storage...  not to mention interface issues...

Going BluRay should mean re-burning all old CDs and DVDs into BluRay for the sake of compatibility.

At least, that's what I think.
Title: Next step for archive storage
Post by: Jack Flesher on January 31, 2008, 12:37:33 pm
Quote
Burning DVDs is a waste of time when the HDD are less than 20 cents per GB.
[a href=\"index.php?act=findpost&pid=170693\"][{POST_SNAPBACK}][/a]

Well said...

Not only that, the current price of single-sided Blue-Ray disks is MORE than that --- $68.95 for a 10-pack at my local retailer (250G total), or roughly 28 cents per Gig.  That same dealer is selling 500G SATA2 drives for $79, or roughly 16 cents a Gig...  

For me, it's a no-brainer to keep triple redundant copies of all my images, mirrored pairs of drives onsite and a duplicate of each drive offsite.   I bought a simple USB2/eSATA to SATA drive adapter ($25), and don't even need cases for the duped offsite drives...
Title: Next step for archive storage
Post by: jjj on January 31, 2008, 01:01:35 pm
For those that advocate HDs. Bear in mind HDs fail and can corrupt data before you realise.
I've lost images in a set like in post above, mirrored disks corrupted the data and even though it was backed up onto other discs, the problem still remained as the corruption was also duplicated. So at least with Optical data, you have increased your safety set. Shame they are too small + slow to bother with at present. I'll invest in a BluRay drive soon and back up onto there as well.
I think you can also get corruption when repeatedly copying to bigger and bigger HDs when each 'enormous' disk fills up.
I 've had CDs fail and I only ever bought good brands and looked after them.
Title: Next step for archive storage
Post by: DarkPenguin on January 31, 2008, 01:22:47 pm
I used to zip (or some other format) my images and create parity files.  This was kind of neat because you could sprinkle the parity files amongst DVDs so if you had a backup that took 10 DVDs you could lose up to like 3 dvds and still restore the backup.

The downside was that it took the Egyptians less time to erect the great pyramid of giza than it took me to make those 10 DVDs.
Title: Next step for archive storage
Post by: Jack Flesher on January 31, 2008, 01:42:31 pm
jjj --- on the corruptions, you need to verify the integrity regularly of of any disks you intend to back-up, whether you are writing to a second HD or optical disk...

As re speed, I now back-up data at SATA2 speeds, which with my newest drives runs at over 100 MB/s SUSTAINED at the front end of the disk, dropping down to maybe 70 MB/s as I near the slow end of the drive. This translates to 500G of images transferring in about 2 hours...    Even FW800 with these new drives runs at around 50 MB/s sustained, or around 3 hours to transfer 500G of images...
Title: Next step for archive storage
Post by: nemophoto on January 31, 2008, 01:59:16 pm
I use pcoket drives for temporaty individual shoot storage. However, when I finish working with those images, I usually archive off to DVD. After investigating prices, etc., I'll probably end up buying 500-750GB drives that I can use with my eSATA connector. I've gradually moved away from USB and Firewire to eSATA because of transfer/write/read speeds. (Even my most recent notebook, an Asus V1s, has a combo USB/eSATA port. It's been great.) I've been procrastinating so long on this, I have about then pocket drives with 80-120GB on each to archive. Too bad I can' use something like Archive Creator to create mini web page indexes to keep track.
Title: Next step for archive storage
Post by: jjj on January 31, 2008, 02:21:58 pm
Quote
jjj --- on the corruptions, you need to verify the integrity regularly of of any disks you intend to back-up, whether you are writing to a second HD or optical disk....
[a href=\"index.php?act=findpost&pid=171281\"][{POST_SNAPBACK}][/a]
I'm not sure they didn't corrupt when drive  filled up or when copying, as the drives are OK.
Title: Next step for archive storage
Post by: Ray on February 01, 2008, 05:31:35 am
The bottom line for me is, I've never lost data that I've recorded to optical disc but I have lost heaps of data recorded on hard drives, but fortunately no permanent loss of RAW images because all my RAW images are on DVD.

On the matter of cost, Blu-ray is in its infancy and very much more expensive per gigabyte than the most economical hard drives, but apparently not more expensive than pocket drives. However, for travelling, pocket drives have the advantage of being lighter and less bulky than current single-sided Blu-ray.

As far as I see from prices on the internet, DVDs which are now a mature technology are the cheapest method of storage, but clearly not the most efficient in terms of bulk, weight and speed of recording and retrieval of data.

It will be interesting to see if the eventuall drop in price of recordable Blu-ray, as the technology matures, results in it becoming the cheapest method of storage.

Bear in mind also that one double-sided Blu-ray disc has the potential to store 200GB of data as more layers are added. I believe the specs allow for 4 layers per side.
Title: Next step for archive storage
Post by: Jack Flesher on February 01, 2008, 11:24:59 am
Quote
The bottom line for me is, I've never lost data that I've recorded to optical disc but I have lost heaps of data recorded on hard drives,
[a href=\"index.php?act=findpost&pid=171463\"][{POST_SNAPBACK}][/a]

I'd be interesting to hear the details on the drives...  Regardless, your back-up plan worked for you and that's what counts

I've had single hard drives fail, but having had all my data redundantly backed up, it was not disastrous.  in my experience the only folks I know who have permanently lost data from a hard drive failure either did not have proper redundant data back-up, or they relied on a single RAID5 array to provide it...

Cheers,
Title: Next step for archive storage
Post by: Farkled on February 01, 2008, 12:19:34 pm
Quote
I'd be interesting to hear the details on the drives...  Regardless, your back-up plan worked for you and that's what counts

I've had single hard drives fail, but having had all my data redundantly backed up, it was not disastrous.  in my experience the only folks I know who have permanently lost data from a hard drive failure either did not have proper redundant data back-up, or they relied on a single RAID5 array to provide it...

Cheers,
[a href=\"index.php?act=findpost&pid=171524\"][{POST_SNAPBACK}][/a]
Exactly.  All media will fail.  Plan for failure; expect failure.  That means redundancy.  There should never be a one and only copy of anything important.
Title: Next step for archive storage
Post by: Ray on February 02, 2008, 03:42:21 am
Quote
I'd be interesting to hear the details on the drives...  Regardless, your back-up plan worked for you and that's what counts

I've had single hard drives fail, but having had all my data redundantly backed up, it was not disastrous.  in my experience the only folks I know who have permanently lost data from a hard drive failure either did not have proper redundant data back-up, or they relied on a single RAID5 array to provide it...

Cheers,
[a href=\"index.php?act=findpost&pid=171524\"][{POST_SNAPBACK}][/a]

Jack,
These are just single drives that have failed for one reason or another during the past 14 years I've been doing digital photography. Because I had all the important stuff bakced up on CD and later DVD, I didn't bother duplicating the data on other hard drives.

The most alarming failure occurred when the power supply in my computer exploded one day. It sounded almost like a gun shot. I replaced the power supply and was able to reboot the computer but wasn't able to open the folders on the D: drive, which was the bigger drive where most of the image files were stored.

I suspect the cause was geckos attracted by the heat of the power supply, crawling in through the wide spacing in the grill to lay their eggs. I taped a piece of mosquito netting to the ventilator grill of the new power supply and haven't had this occur again.

I know it's easier to have all data duplicated on different hard drives, which I'm now doing. I just find it curious that I've never come across an optical disc in my collection that once was readable and then became unreadable, except occasionally when there was a problem with a particular CD/DVD reader or its driver or firmware. Such problems have always been solved by using a different CD/DVD drive.
Title: Next step for archive storage
Post by: Jonathan Wienke on February 02, 2008, 10:39:57 am
Quote
I just find it curious that I've never come across an optical disc in my collection that once was readable and then became unreadable, except occasionally when there was a problem with a particular CD/DVD reader or its driver or firmware.

You're just very lucky.
Title: Next step for archive storage
Post by: marcmccalmont on February 02, 2008, 02:30:29 pm
Cavalry 1TB, (2 500Gb drives backing each other up) $239
http://www.newegg.com/Product/Product.aspx...N82E16822101084 (http://www.newegg.com/Product/Product.aspx?Item=N82E16822101084)

a replacement drive for when the failure occurs $95
http://www.newegg.com/Product/Product.aspx...N82E16822136178 (http://www.newegg.com/Product/Product.aspx?Item=N82E16822136178)

when one drive fails the other automatically rebuilds the replacement drive.
I ordered the replacement drive at the same time as the dual drive so when (not if!) the failure occurs I have a spare drive ready. $335 total for a little peace of mind.
Marc
Title: Next step for archive storage
Post by: Ray on February 02, 2008, 11:39:12 pm
Quote
You're just very lucky.
[a href=\"index.php?act=findpost&pid=171764\"][{POST_SNAPBACK}][/a]

Or perhaps very careful. I've never bought optical discs over the internet, for example. If the price looks too good to be true, it sometimes is.

Before setting off on this trip, I bought 4 spare BP-511A batteries through the internet, not because I needed 4 but because they were so damned cheap. The total cost including postage was less than the internet cost of one genuine Canon battery.

When I first charged them, one simply wouldn't charge. I was too busy with last minute packing and other matters to write letters of complaint or return the battery. 3 were sufficient. 2 months later, a second battery refused to charge and the remaining 2 are now not holding much charge another 2 months later, as though they are close to the end of their life. However, I'm still using the original batteries that came with my 20D and 5D.
Title: Next step for archive storage
Post by: Ray on February 03, 2008, 12:12:08 am
Quote
...when one drive fails the other automatically rebuilds the replacement drive.
I ordered the replacement drive at the same time as the dual drive so when (not if!) the failure occurs I have a spare drive ready. $335 total for a little peace of mind.
Marc
[a href=\"index.php?act=findpost&pid=171809\"][{POST_SNAPBACK}][/a]

What happens in the event of a catastrophe; an unusual power surge, a direct lightning hit or a burglary?

For complete peace of mind, a separate copy of all the important files stored at a separate location is advisable. For me, the important files are the RAW images. They are all on optical media. However, I also keep a second external drive of organised RAW and processed images in a different location, but I do not back up my data regularly to this drive.

About a year ago I had an unfortunate experience with a new LaCie Big Disc (500GB). After many hours of work transferring images from a previous trip (from DVDs ) and after carefully organising the files into readily accessible subject matter, I was about 90% of the way through the task when the external drive malfunctioned and none of the data was able to be accessed. The computer just refused to recognise the drive. I suspect over-heating might have been the cause.

Fortunately, all I'd lost was my time. It could have been worse.
Title: Next step for archive storage
Post by: jjj on February 03, 2008, 03:26:56 pm
Quote
Or perhaps very careful. I've never bought optical discs over the internet, for example. If the price looks too good to be true, it sometimes is.
[a href=\"index.php?act=findpost&pid=171920\"][{POST_SNAPBACK}][/a]
I've only ever bought good discs. I've had some fail.
A journalist on PCPro mag wrote one day he went back through his DVDs and found large No.s had deteriorated only six months later. Again good quality discs - in theory.

I find DVDs too slow and bulky, especially with laptop burners. If I'm away for 4 weeks and produce say 300G+ of data, that's probably 80-90 discs in reality.
My next laptop must have BluRay writing. So I'm hanging on to mine till then.


Quote
I suspect the cause was geckos attracted by the heat of the power supply, crawling in through the wide spacing in the grill to lay their eggs. I taped a piece of mosquito netting to the ventilator grill of the new power supply and haven't had this occur again.[a href=\"index.php?act=findpost&pid=171717\"][{POST_SNAPBACK}][/a]
If this had happened to early computers would we call problems in code, lizards?  
Title: Next step for archive storage
Post by: jjj on February 03, 2008, 03:38:30 pm
Quote
What happens in the event of a catastrophe; an unusual power surge, a direct lightning hit or a burglary?
[a href=\"index.php?act=findpost&pid=171926\"][{POST_SNAPBACK}][/a]
My back up computer is going in locked cellar on a separate mains ring [when builders have finished demolishing house and rebuilt it again, that is].
My office in attic is also locked with serious fire doors and very solid frame.
An RCD board is fitted and surge protectors are between mains and computer.
Backup machine is also very small and easily hidden.
Title: Next step for archive storage
Post by: Ray on February 04, 2008, 12:49:52 am
Quote
A journalist on PCPro mag wrote one day he went back through his DVDs and found large No.s had deteriorated only six months later. Again good quality discs - in theory.
[a href=\"index.php?act=findpost&pid=172032\"][{POST_SNAPBACK}][/a]

One has to wonder about such claims. What is meant by 'deteriorated'? Chemical break-down of the disc coatings? Were such apparently failed discs really recorded properly in the first instance? Did the journalist try the so-called 'deteriorated' discs in a few other drives?

I've written that I've never come across a failed disc in my collection of 2000+ CDs and DVDs, that I know was recorded properly and fully readable after recording.

But I have found that the occasional disc fails to read properly on one particular drive, but reads okay on another. I remember well the occasion when this first happened. It was a Kodak Photo-CD disc that was only 2 years old. I'd just updated my CD reader from a 4x to 20x, fully expecting that the 18MB Photo-CD files that used to take a full 2 minutes to open would then open in 30 seconds or less. Not so. Some files wouldn't open at all. Some did, taking about the same time of 2 minutes. Some looked as though they would open, eventually, but the process would slowly grind to a halt and they never made it.

Because the results were inconsistent from file to file, it was easy to jump to the conclusion that some kind of uneven deterioration of the disc had taken place. The problem was solved by returning to my old 4x CD drive. Those first Kodak Photo-CD discs, which are now about 15 years old, open flawlessly on my current Pioneer DVD burner. In fact, it would be true to say they actually perform better than they did 15 years ago. Each file used to take 2 minutes to be read and to open on the monitor. The process now takes about 5 seconds.
Title: Next step for archive storage
Post by: jjj on February 04, 2008, 05:25:31 am
Quote
One has to wonder about such claims. What is meant by 'deteriorated'? Chemical break-down of the disc coatings? Were such apparently failed discs really recorded properly in the first instance? Did the journalist try the so-called 'deteriorated' discs in a few other drives?
I've thrown the mag out now, but he had a batch that had in fact deteriorated, visibly IIRC.  I know you don't believe anything anyone else says [good up to a point], but sometimes, other people are actually correct.
As for using other drives  to read them, see my last point.

Quote
I've written that I've never come across a failed disc in my collection of 2000+ CDs and DVDs, that I know was recorded properly and fully readable after recording.
I doubt you've ever been murdered either, but I gather it's something that sadly happens to other people far too often.


Quote
Because the results were inconsistent from file to file, it was easy to jump to the conclusion that some kind of uneven deterioration of the disc had taken place. The problem was solved by returning to my old 4x CD drive. [a href=\"index.php?act=findpost&pid=172143\"][{POST_SNAPBACK}][/a]
Assuming you still have the old drive, it still fits in your computer and it also is still functional, that's still not  a good way to archive stuff, relying on old irreplacable, possibly incompatible kit.
Maybe it works in that drive as that is the drive that incorrectly wrote the discs in the first place, as that's another possible  reason for the read failure. The whole point of CDs, DVDs etc is they are usable by anyone with a CD/DVD reader. If that is not true they the archive has failed to be archival on quite a fundamental level.
Title: Next step for archive storage
Post by: Ray on February 04, 2008, 07:52:44 am
Quote
I've thrown the mag out now, but he had a batch that had in fact deteriorated, visibly IIRC.  I know you don't believe anything anyone else says [good up to a point], but sometimes, other people are actually correct.

Whatever the product, there are sometimes lemons that get through the quality control checks. Generally one might expect that the cheaper the product, the poorer the QC, but obviously that's not always the case.

There are also scams. I can imagine that in some disc factory in China, operating on a very narrow profit margin, a batch of CDs or DVDs produced with the wrong mix of chemical dyes, or with out-of-adjustment machinery, is officially consigned for destruction, but due to corruption at the work place the faulty consignment is sold to some shady dealer at a very low price who passes the discs on to another dealer who sells them to unsuspecting Americans or Englishmen on the internet.

Such is life! I think there's less risk of this happening if one buys from a reputable dealer.

Of course I know that other people are sometimes correct. But I also know they are sometimes incorrect. I also know that there's a very strong trait in human nature to blame others for one's own mistakes, to search for a scapegoat, and this is much easier when the scapegoat is an impersonal organisation.

Quote
I doubt you've ever been murdered either, but I gather it's something that sadly happens to other people far too often.

Murders are investigated and there's usually pretty conclusive evidence of the existence of a dead body. An incident of a disc that has chemically deteriorated in just 6 months should be investigated. They are supposed to last about 50 to 100 years, aren't they?

Quote
Assuming you still have the old drive, it still fits in your computer and it also is still functional, that's still not  a good way to archive stuff, relying on old irreplacable, possibly incompatible kit.

I think you missed my point. The solution at the time, 12 years ago, was to revert to using the previous CD drive that I'd had upgraded. The shop recognised that they'd bought a batch of substandard CD drives that had difficulty with the Photo-CD format and reinstalled my old drive free of charge. That was my first computer.

Quote
Maybe it works in that drive as that is the drive that incorrectly wrote the discs in the first place, as that's another possible  reason for the read failure. The whole point of CDs, DVDs etc is they are usable by anyone with a CD/DVD reader. If that is not true they the archive has failed to be archival on quite a fundamental level.

That's true and they are usable by anyone with a CD/DVD reader, except on those occasions when things go wrong. Incompatibility issues are rife in the computer world. You know that   .
Title: Next step for archive storage
Post by: Ray on February 10, 2008, 05:06:56 am
Just to get things in perspective, I thought I would add this piece of arcane information.

Having returned to Australia and whilst doing a bit of shopping I popped into one of my my favourite bricks & mortar stores for good bargains, Crazy Clark's, to buy a few more China made DVD wallets at A$2-3 each.

Whilst there, I notice DVD spindles of 50 discs, 2 for $35. These are recordable DVD-R, 16x with printable surfaces. They have a lifetime warranty.

Just out of curiosity I checked the current price of external hard drives at my favourite computer store in Brisbane. A 500GB Maxtor One Touch external hard drive is priced at $279.

How do these different media compare? On the one hand we have optical storage at 7.5 cents per gigabyte, guaranteed for a lifetime.

On the other hand we have 3.5" hard drive at 56 cents (Australian) per gigabyte guaranteed for 1 year.

Hmmm! I wonder which is the no-brainer?
Title: Next step for archive storage
Post by: Jonathan Wienke on February 10, 2008, 10:15:26 am
Quote
Hmmm! I wonder which is the no-brainer?

Factor in the time you spend burning 100 DVDs and verifying the data, plus indexing and labeling the discs. I'm thinking that's at least 20 hours, probably more, depending on your burner speed, during which you and your computer are pretty much tied up and unable to do other things.  Then there's the hassle of digging through 100 DVDs trying to find one particular image. And copying all of those DVDs to new media before the DVD format becomes obsolete in a few years and DVD drive production ceases and the the drive you have quits working. There's also the factor of whether you are sufficiently dedicated to backing up your data to actually burn all of those DVDs and properly index and label them so that finding one particular image in the stack of discs is not an exercise in frustration. I figure my time is worth at least $15/hour, and spending 20+ hours burning DVDs vs a simple drag-and-drop file copy which can be done unattended once started is a very unattractive proposition. One might even say "no-brainer".
Title: Next step for archive storage
Post by: Ray on February 10, 2008, 11:09:15 am
Quote
Factor in the time you spend burning 100 DVDs and verifying the data, plus indexing and labeling the discs. I'm thinking that's at least 20 hours, probably more, depending on your burner speed, during which you and your computer are pretty much tied up and unable to do other things.
[a href=\"index.php?act=findpost&pid=173713\"][{POST_SNAPBACK}][/a]

Jonathan,
The thread's about using optical media whilst travelling. You don't have to sit patiently waiting for your DVDs to burn. You can take a shower or have a meal. The only extra work really is organising folders to the right size to economically fill a disc. Once the folders are the right size it's just a matter of 'drag and drop' in the burning software and later a 30 second job of labelling each disc with a felt tipped pen. If you're prepared to waste a bit of disc space, you can save time by just recording the contents of each 4GB memory card.
Title: Next step for archive storage
Post by: Jonathan Wienke on February 10, 2008, 02:18:07 pm
Quote
Jonathan,
The thread's about using optical media whilst travelling.

So what? For many types of shoots (weddings, concerts, charity events, school portraits, horse shows, etc) You're going to end up with several DVD's worth of images. I've covered all-day music festivals where I've shot more than 30GB of RAWs in a single day. Spending an hour or two at the end of a 14-hour day just to burn and verify DVD backups when I can simply drag-drop copy the files to a NAS and be done with it would be retarded.
Title: Next step for archive storage
Post by: Ray on February 10, 2008, 06:34:19 pm
Quote
So what? For many types of shoots (weddings, concerts, charity events, school portraits, horse shows, etc) You're going to end up with several DVD's worth of images. I've covered all-day music festivals where I've shot more than 30GB of RAWs in a single day. Spending an hour or two at the end of a 14-hour day just to burn and verify DVD backups when I can simply drag-drop copy the files to a NAS and be done with it would be retarded.
[a href=\"index.php?act=findpost&pid=173774\"][{POST_SNAPBACK}][/a]

Circumstances vary. There's no single hard and fast rule for this sort of thing. Shooting 30GB of RAW in a day is probably unusual. Shooting 30GB of RAW day after day would be even more unusual. But if that's your life-style, then of course, forget about DVDs. If I were shooting weddings and horse shows, I don't think I'd be too concerned about archival issues anyway. Nor would I want the on-going chore of hard drive maintenance and periodic transfer to new drives in case a client wanted a particular image in 10 year's time.

The great thing about taking that initial trouble to record your RAW images on DVD, is that you can store them away and basically forget about them. No maintenance required. For the purpose of fast access to images, you're always going to try to have your images organised and available on hard drives, whether or not you've also got them on DVD.
Title: Next step for archive storage
Post by: Jonathan Wienke on February 10, 2008, 08:36:17 pm
Quote
Shooting 30GB of RAW in a day is probably unusual. Shooting 30GB of RAW day after day would be even more unusual.

Not really; the OP specified that he's shooting 20GB in a typical day with his 1Ds-MkIII.
Title: Next step for archive storage
Post by: Ray on February 10, 2008, 08:56:48 pm
Quote
Not really; the OP specified that he's shooting 20GB in a typical day with his 1Ds-MkIII.
[a href=\"index.php?act=findpost&pid=173864\"][{POST_SNAPBACK}][/a]

Yes, I know. I'm really addressing the archival benefits in conjunction with media costs here. For 20Gb of data a day, DVDs are too troublesome and, for me, Blu-ray discs are presently too expensive. However, the extra time involved in recording a day's work on a single Blu-ray disc whilst one takes a shower or watches CNN news should not be any great impediment if the archival qualities of optical media appeal to one.

However, I can understand that those who have already had a bad experience with the longevity of optical discs would not be interested.
Title: Next step for archive storage
Post by: free1000 on February 22, 2008, 01:33:55 pm
Quote
Yes, I know. I'm really addressing the archival benefits in conjunction with media costs here. For 20Gb of data a day, DVDs are too troublesome and, for me, Blu-ray discs are presently too expensive. However, the extra time involved in recording a day's work on a single Blu-ray disc whilst one takes a shower or watches CNN news should not be any great impediment if the archival qualities of optical media appeal to one.

However, I can understand that those who have already had a bad experience with the longevity of optical discs would not be interested.
[{POST_SNAPBACK}][/a] (http://index.php?act=findpost&pid=173865\")

Optical is both too expensive and slow.

This is where I am headed....

[a href=\"http://www.sharkoon.com/html/produkte/externe_gehaeuse/sata_quickport/index_en.html]http://www.sharkoon.com/html/produkte/exte...t/index_en.html[/url]

I can easily generate 30Gb by the end of a project/shoot with the Aptus 75.  So I'll be adopting the principle of archiving all the RAWs off to an HD via one of these (multiple copies of course)

Then I'll just keep first selects and derived files online.

eSATA is a wonderful thing.
 
Then you just need a way of storing all the SATA drives. No problem.

Currently I'm using the Samsung 500Gb drives as they are a good price and come in plastic transparent boxes which are protective and stackable.
Title: Next step for archive storage
Post by: Ray on February 23, 2008, 06:31:34 pm
Quote
Optical is both too expensive and slow.

I can easily generate 30Gb by the end of a project/shoot with the Aptus 75.  So I'll be adopting the principle of archiving all the RAWs off to an HD via one of these (multiple copies of course)

[a href=\"index.php?act=findpost&pid=176699\"][{POST_SNAPBACK}][/a]

If I were in your situation I'd probably do the same. 30GB is a lot to handle. I rarely shoot more in one day than can fit on a single DVD. Looking into the future, I expect my next camera might be around 24mp, by which time recordable Blu-ray will be more affordable. One single sided, single layered Blu-ray disc would then hold 2 to 3 days of shooting at my current maximum rate doubled.

I just recently had a look at some of my first recordings to CD-R made in early 1999. They are recordings of my first attempts at scanning with my first scanner, the Nikon LS-2000, and because the scans were not too good and the film has all been rescanned later with better scanners, I never bothered transferring the files to DVD.

I was curious as to how these discs would withstand scrutiny with Nero's analysis tools. Have any errors crept in or damage to the disc surface occurred?

None whatsoever, if these tools are credible. No errors found; no glitches and no damage to surfaces. Some of the discs were apparently nondescript bargains and Nero is unable to identify the manufacturer, but they still read perfectly 9 years later.