Luminous Landscape Forum

Equipment & Techniques => Medium Format / Film / Digital Backs – and Large Sensor Photography => Topic started by: Zerui on October 07, 2011, 05:01:30 am

Title: Digital- to-Film for 50-year archive
Post by: Zerui on October 07, 2011, 05:01:30 am
I recently gave high level public lectures illustrated by 100 photographs I had taken fifty years ago in 1961. The lectures are being followed by a book. For the lecture I digitized (and cleaned up) Kodachromes that had been mounted in glass slides and stored for fifty years, without any thought to conservation. The images are still vibrant with the rich colours of Kodachome.  I am now digitizing some of my 70mm (sic) Hasselblad Ektachrome photographs from the 1960s. They will also be used in lectures and a book.
My point is that when stored as film, the photographs have survived fifty years in a box. I did not have to do anything to ensure their survival.  There was no need for active conservation beyond providing a reasonable climate for storage.
My question is how to ensure the fifty-year term survival of images I am taking today with my digital Hasselblad.  I doubt whether they will survive as digital images after I am dead, when nobody is taking care to maintain them in successive new formats and media that will emerge over the decades.  But the interest shown in my 1961 photographs suggests it is worth preserving those taken in 2011. My recent experience suggest that the photographer cannot predict which images will prove most interesting in fifity years time.  So it is necessary to preserve a much larger collection of original images than one can reasonably store as prints for fifty years. Those I choose to print today are for the interests of today, not those of 2071, which I cannot predict.
One solution might be to convert the digital images to film. That is done in the motion film world with systems like the Arrilaser and Celco's Film Fury.But I doubt that the operators of these sophisticated devices would quote an affordable price for converting a few thousand digital still images to film.  Do you know of any equivalent service for still photographs? Are museums doing this?
Do you have any experience of archiving still images on film? What is your strategy for photographic posterity in the digital age, other than archiving prints?
John   
Title: Re: Digital- to-Film for 50-year archive
Post by: Sareesh Sudhakaran on October 07, 2011, 05:45:09 am
No experience with archiving, but here's a guess: Why not use the internet and save it under different servers for free? Someone will have to wipe out the internet to destroy all copies of your data. This way, it stays digital, in the exact specifications you stored it in. Most common archival format is TIFF (at least that's what it says on wikipedia!). I don't know if this will help, but it's a start.
Title: Re: Digital- to-Film for 50-year archive
Post by: Graham Mitchell on October 07, 2011, 05:58:14 am
There are commercial services for producing film images from digital files but I haven't used one in years and I'm not sure how common they are any more. But if you want to do a few thousand images, then this is probably going to cost in the tens of thousands of dollars. That could buy you a LOT of hard drives, and it will only buy you one film backup per image. One copy doesn't help in the case of a fire, flood, theft, etc.

I agree this is an issue, but film is not necessarily the answer.

Title: Re: Digital- to-Film for 50-year archive
Post by: TH_Alpa on October 07, 2011, 07:27:47 am
I would get in touch with a Data Online Storage company. There are a lot of companies and solutions with differents warranties, most probably much cheaper than outputting it on film.

Thierry
Title: Re: Digital- to-Film for 50-year archive
Post by: Zerui on October 07, 2011, 10:07:26 am
Thank you to those who have responded so far, much appreciated.  I have my doubts about relying on ANY digital solution, whether hard disk or cloud, on the grounds that 50 years is much longer than the half-life of (1) computer technology and (2) commercial enterprises. TIFF in 2071?  Maybe?  I still think film is probably the best solution. It has worked for fifty years in my case. Perhaps some enterprising person will design an affordable desktop "reverse scanner": digital in, film out. There might be a market for it. Meanwhile I look forward to further suggestions.  John
Title: Re: Digital- to-Film for 50-year archive
Post by: Ajoy Roy on October 07, 2011, 10:20:41 am
Here is a service provider http://www.prolab.com.au/services/Lab-Services/Film-Writer-LVT.php

I think that the problem with digital is more in the data format than the storage or the medium. If you have data from early eighties, and if you wants to preserve it, it will have to go through a format conversion now and may be every ten years. This is in contrast to paper (or film in our case) which needs no conversion through out its life. As newer and more efficient formats appear, the older formats fall into disuse and finally there is no way to read them. So if there is a need for long term archiving, a long term data format standard needs to evolve. Another problem with digital data is the media. I have yet to come across a media which is guaranteed to last a reasonably long time, without being prohibitively expensive. The best I have come across is 50 years, and that is an estimate, not proven life. In contrast film and paper, if properly preserved will last till eternity.

At one time I was also thinking of converting the digital images to a film. In my case I had thought of rigging up a solution using 4x5 or 8x10 colour sheet film (though expensive it has the advantage of easier review compared to 35mm or 6x6. If the demand is high enough, some one will come up with a cost effective service. After all what is needed is a high resolution film writer, similar to printer but with a finer density (2000DPI vs 600DPI).

The only source for film that I could find is http://www.fujifilm.com/products/professional_films/pdf/velvia_100f_datasheet.pdf
As there is quite a decent volume of film based processing still, I am investigating the feasibility of digital to film conversion, and if the volumes are high enough to justify the investment, a unit can be set up in India.

Title: Re: Digital- to-Film for 50-year archive
Post by: cng on October 07, 2011, 10:41:17 am
My 2cents: if you're really trying to avoid digital archiving, then I would consider outputting to print (not film). Document scanners are more prevalent than film scanners, and could be argued to continue to be so in the future. Also, is there any archival rating or guarantee for the emulsions used in the digital-film transfer?

Print multiple copies (say A4 or 8x10) using archival materials and store appropriately in multiple locations for redundancy. Repro from prints can actually be more desirable in some situations.
Title: Re: Digital- to-Film for 50-year archive
Post by: Sareesh Sudhakaran on October 07, 2011, 12:08:40 pm
Fuli Velvia 100 is rated for 300 years without fading and major changes in color stability. Kodak, too, agrees that film is the best way to archive images. Is there a control process to manage color and the full dynamic range (and color properties) of different digital color spaces?

There is a system for motion pictures, though. You could try contacting a film lab (for movies) that does DI and film outs - they might handle your files for you at a reasonable cost, but they only output to 35mm film (mostly) - unless you can afford IMAX. The higher end labs are also used to dealing with the CIE XYZ color space (DCI spec). All the best.
Title: Re: Digital- to-Film for 50-year archive
Post by: Graham Mitchell on October 07, 2011, 01:26:18 pm
Thank you to those who have responded so far, much appreciated.  I have my doubts about relying on ANY digital solution, whether hard disk or cloud, on the grounds that 50 years is much longer than the half-life of (1) computer technology and (2) commercial enterprises. TIFF in 2071?  Maybe?  I still think film is probably the best solution. It has worked for fifty years in my case. Perhaps some enterprising person will design an affordable desktop "reverse scanner": digital in, film out. There might be a market for it. Meanwhile I look forward to further suggestions.  John

If TIFF is ever replaced, it will be a gradual process. Photoshop 2071 could be used to batch convert the files from TIFF to the new standard. That part of the problem is insignificant.

Every home should have a digital vault, and probably will do some day. The concept exists already but will become more mainstream as internet speeds improve and data storage costs drop.
Title: Re: Digital- to-Film for 50-year archive
Post by: Fine_Art on October 07, 2011, 02:21:05 pm
My 2cents: if you're really trying to avoid digital archiving, then I would consider outputting to print (not film). Document scanners are more prevalent than film scanners, and could be argued to continue to be so in the future. Also, is there any archival rating or guarantee for the emulsions used in the digital-film transfer?

Print multiple copies (say A4 or 8x10) using archival materials and store appropriately in multiple locations for redundancy. Repro from prints can actually be more desirable in some situations.

Hows about printing on transparency with archival inks? The celluloid is much more stable than paper. The biggest problem short of a catastrophe is humidity cycles with the seasons.
Title: Re: Digital- to-Film for 50-year archive
Post by: Wayne Fox on October 07, 2011, 08:56:40 pm
You are comparing the preservation by yourself of images you placed some value on while you are alive to what will happen once you are not.  

If the images are valued by future generations there will be efforts to preserve them regardless of how they are archived.  It they are not valued, I don't think using film will help, because someone will just toss them some day (or they'll get ruined in a flood or fire).  so despite the ability of the film to perhaps maintain  quality, 50 years from now will anyone care enough about that box of pictures?  Digitally there can be redundant copies and any transition to newer formats will certainly be slow with tools to handle that so your odds might be better with digital.

Another challenge, if they are valued and held onto will there be any technology then which can utilize the transparencies?  How do you know there will be anyway to convert the visual information to a useful format ... who knows if there will even be a way to scan them anymore?

MJ over at TOP did a nice article (http://theonlinephotographer.typepad.com/the_online_photographer/2011/06/eight-ways-to-preserve-your-pictures.html) about this, which puts a little different spin on the whole permanence/archival thing.

(also wondering why this is in this particular forum, not that I care, just sort of curious as it seems there are some other forums that may get more responses).
Title: Re: Digital- to-Film for 50-year archive
Post by: ced on October 08, 2011, 01:49:00 pm
As all the major government archival agencies, institutes, cities etc. are digitising images, documents etc; etc. I think there is no need to worry about archiving digital data.  There would be such an outcry if these bodies were unable to access this archived material.
They could also initiate research to recover such data if there ever was some way technology would leave this method behind.
I think that you have been fortunate that your films kept so long, I know & also have personal experience that this is not always so.
Save 2 copies one Tiff & one Jpeg and put them on separate raid sets of disks and sleep peacefully for another 50 yrs.
Some Links to extra info:
http://digitalcommons.uconn.edu/cgi/viewcontent.cgi?article=1027&context=libr_pubs&sei-redir=1&referer=http%3A%2F%2Fwww.google.be%2Furl%3Fsa%3Dt%26source%3Dweb%26cd%3D23%26ved%3D0CCgQFjACOBQ%26url%3Dhttp%253A%252F%252Fdigitalcommons.
uconn.edu%252Fcgi%252Fviewcontent.cgi%253Farticle%253D1027%2526context%253Dlibr
_pubs%26rct%3Dj%26q%3Dbest%2520format%2520for%2520image%2520archival%26ei%3D7YuQTpv1F6rS4QSOmdTCAQ%26usg%
3DAFQjCNHGEAD2XCRXQGLcYxquFInUfDuDGg%26sig2%3DBQe4Fj-2ODH9kDP9Autk_w#search=%22best%20format%20image%20archival%22

www.historicalvoices.org/papers/image_digitization2.pdf
Title: Re: Digital- to-Film for 50-year archive
Post by: hjulenissen on October 08, 2011, 04:27:28 pm
There was recently some media articles about a new DVD format designed to last for 100s of years, while still being playable in regular DVD players.

If you can do some intervention every 10 years, then I think the problem is easy. If you want to do 50 years with no intervention, it gets harder. Perhaps put a good-quality PC with a hard-drive in a bank deposit box. If the PC runs or can be fixed (a few caps perhaps), it may be easier exporting the data when you have the software and electronics to read it.

But using tried-and-true media is a good point. Analog film and analog music media have survived for quite some time (with some degradation and some total loss). The track-record for digital content remains to be seen.

-h
Title: Re: Digital- to-Film for 50-year archive
Post by: design_freak on October 08, 2011, 04:41:14 pm
There was recently some media articles about a new DVD format designed to last for 100s of years, while still being playable in regular DVD players.

If you can do some intervention every 10 years, then I think the problem is easy. If you want to do 50 years with no intervention, it gets harder. Perhaps put a good-quality PC with a hard-drive in a bank deposit box. If the PC runs or can be fixed (a few caps perhaps), it may be easier exporting the data when you have the software and electronics to read it.

But using tried-and-true media is a good point. Analog film and analog music media have survived for quite some time (with some degradation and some total loss). The track-record for digital content remains to be seen.

-h

I can say only that 100 years for DVD - it is only marketing. After 10 years you could lost everything.
Even goog quality HD is not good idea. Till today the best possible medium to archive is tape :-) If you have 1% of nominal signal on the tape it is possible to read it without problems. It's my 2 cents


Title: Re: Digital- to-Film for 50-year archive
Post by: hjulenissen on October 08, 2011, 04:53:52 pm
I can say only that 100 years for DVD - it is only marketing. After 10 years you could lost everything.
This is my reference:
http://www.computerworld.com/s/article/9140771/Start_up_claims_its_DVDs_last_1_000_years

Of course, no matter if it lasts 12 years or 1000 years, this thread will be dead by then anyways :-D

-h
Title: Re: Digital- to-Film for 50-year archive
Post by: Sareesh Sudhakaran on October 09, 2011, 12:28:45 am
Kodak sells archival DVDs too, which will supposedly last 100+ years. But can you find players for 5.25 or 3.5 floppies today? It might be a good idea if these companies also guaranteed players to last that long.

I do not recommend backups on tape or HDD - even the companies making the most expensive and robust archival systems can go kaput. Even film needs periodical maintenance - as filmmakers know only too well.

Or one could seal their work in vacuum and store them at sub zero temperatures in an air controlled vault - like Mr. Gates is doing. The Egyptians tried something like that a few thousand years ago...it might just work.
Title: Re: Digital- to-Film for 50-year archive
Post by: hjulenissen on October 09, 2011, 05:08:45 am
Kodak sells archival DVDs too, which will supposedly last 100+ years. But can you find players for 5.25 or 3.5 floppies today? It might be a good idea if these companies also guaranteed players to last that long.
3.5" and 5.25" players: no problem. Older storage formats a little harder but usually doable if you spend some time. Eventually, it is going to be hard to find a computer with the necessary interface to use that player (serial port is still fine, but what about "printer port" and scsi port drives? parallell ATA is also going the way of the dodo). Putting a self-contained PC with display and everything in a vacum container is safe in that everything is there (you just need to make it work and figure a way to get the data out of it).

The problem more often (I believe) is that even if you can read the raw data off the medium, interpreting the data can be hard. Especially if the data is complex, database-like, proprietary (like my database of Canon raw files stored in a Microsoft-type filesystem neatly organized and tagged Adobe proprietary lightroom database, really).

So perhaps the optimal solution is to spread your images as film, on the internet, as hard-drives, etc. If you have enough independent "channels", the probability that at least one survives for some finite period of time should increase.

-h
Title: Re: Digital- to-Film for 50-year archive
Post by: amsp on October 09, 2011, 01:02:03 pm
Honestly, I don't see how anything could be safer than saving them in TIFF with an online hosting company. It's not like the format is going to die and be replaced overnight and all of a sudden all programs that supported it will decide to no longer be backwards compatible. Same thing goes for a company offering the archival service, they don't close down business and erase your data out of the blue. I've used the same hosting service for 14 years now for example. The cloud is your safest bet IMO, with copies stored locally on your HD.
Title: Re: Digital- to-Film for 50-year archive
Post by: John.Williams on October 09, 2011, 01:51:42 pm
Taking a step back, I see the significance of your premise, John.

In other words, fast forward to 2020 and will you be able to with the same degree of ease, reference your digital files as though retrieving the shoebox, opening, and holding up to the light, the instant visibility of the image.

A retiring Fuji film representative told me a decade ago that he would argue that film is:

Of course, we had a good chuckle about that, but the implications of his statement are solid points to ponder. This does not mean we are to abandon our digital technology, but to make the effort to meet the need for retrieving our images in 2020 with the same degree of ease and reliability of "the shoebox."

P.S. as a completely shameless plug, we have a Flextight scanner in our rental pool for this exact scenario: shoebox + film = digital file.

Enjoy exploring!

John
Title: Re: Digital- to-Film for 50-year archive
Post by: Ajoy Roy on October 09, 2011, 11:17:10 pm
A visual image directly decoded by our eyes, beats digital archiving for instant retrieval. Think of all the problems involved in decoding lost scripts, and you can visualise the problems that can creep up with digital archives. I always wonder how in those end-of-the world scenarios, the hero has always the means to access high tech gadgets, while the villains are have lost it all. In reality it may not be so, and I will not want the future generation to wonder how the gadgets left by the posterity function. Give me film which is recognised immediately, without bothering about readers, electric voltage and accompanying infrastructure. You do not even have to know how to read to appreciate the photo on film.
Title: Re: Digital- to-Film for 50-year archive
Post by: Ben Rubinstein on October 10, 2011, 04:30:05 am
Although in 100 years you may not be able to interface or even read today's digital media without significant expense, the same will be as true but a lot sooner with film. How many companies make decent multi format scanners even here and now in 2011? How many companies will make enlargers and chemistry to print those films in 2050? I'd be willing to hazard a guess that it will be the same number as the companies who will read your Kodak/Delkin 100 year DVD's or provide a service to read older standards of tape or SATA II drives. Difference being that it will be far cheaper to retrieve 2000 images from a tape (and far cheaper to have made multiple copies in the first place) than to scan and work 2000 negs or trannies in 2050. Just as it is today.

As an aside, with practically every museum and library worldwide having spent huge resources on the digitalisation of their collections to TIFF I very much doubt that you will have difficulty whatsoever in reading the TIFF file format a century from now.
Title: Re: Digital- to-Film for 50-year archive
Post by: Zerui on October 10, 2011, 05:21:26 am
Thank you to all who have contributed to addressing my question about photoposterity (50 years plus).
As I suspected, this forum reaches parts of the knowledge base that others miss.
And I suspect that many of its subscribers are producing work that might be appreciated in the far future
Think of Eugène Atget's photos documenting the fast disappearing old Paris.

Let me summarize my reactions so far:
1.  50 years is a really long time.  Longer than the half life of (1) data media and (2) businesses.
2.  I liked the suggestion that archiving in prints is probably better than film, on the grounds that film scanners may be rare in 2071.
     The prints will have to be at 100%, which makes them quite large for Medium Format photographs.
     So one has to consider the cost per print, and the storage problem.
3.  I was also impressed by the point that museums are scanning documents into TIFF files.
    National Museums will surely have a longevity that far exceeds the fifity years I posed.
    Museum staff can reformat, or change digital medium as needed to keep up with evolving technology.
    A Museum with its digital archive is likely to be more permanent than a commercial enterprise.
    If the British Museum or British Library (or the equivalent in your country) set up a service to maintain my digital files I would trust them for posterity.

So, the best bet seems to be either
(1) prints made and stored by me, or
(2) digital files maintained by a national museum/library that is likely to survive for ever.

Further thoughts on the general issue, or on these preliminary conclusions would be welcome.
John
Title: Re: Digital- to-Film for 50-year archive
Post by: TH_Alpa on October 10, 2011, 06:03:37 am
Honestly, I don't see how anything could be safer than saving them in TIFF with an online hosting company. It's not like the format is going to die and be replaced overnight and all of a sudden all programs that supported it will decide to no longer be backwards compatible. Same thing goes for a company offering the archival service, they don't close down business and erase your data out of the blue. I've used the same hosting service for 14 years now for example. The cloud is your safest bet IMO, with copies stored locally on your HD.
+1

Thierry
Title: Re: Digital- to-Film for 50-year archive
Post by: theguywitha645d on October 10, 2011, 10:30:42 am
CDs and DVDs are not archival. These media are not for long-term storage.

For private use, hard drives are the best bet, but you will have to have a plan to copy forward files--drive disks will corrupt as well. I use a RAID array. The cloud is another good option as they will be using more sophisticated drives and systems. Certainly have the images in more than one place.

TIFF, JPEG, and PDF are considered archival formats that can carry forward. RAW images should be converted to TIFF, but I would archive the RAW as well--If you have the RAW profile, you should always be able to process them.

The surest way to preserve your images is to become really famous.
Title: Re: Digital- to-Film for 50-year archive
Post by: feppe on October 10, 2011, 12:06:14 pm
Honestly, I don't see how anything could be safer than saving them in TIFF with an online hosting company. It's not like the format is going to die and be replaced overnight and all of a sudden all programs that supported it will decide to no longer be backwards compatible. Same thing goes for a company offering the archival service, they don't close down business and erase your data out of the blue. I've used the same hosting service for 14 years now for example. The cloud is your safest bet IMO, with copies stored locally on your HD.

I agree mostly. The key is to have multiple backups in multiple locations. I've chosen always-on HDD, and offline backup to external HDDs (weekly on site, monthly kept at the office to cover for theft/fire/water damage) and online cloud (Crashplan) for redundancy.

But I would never ever trust the cloud as only backup. If a hosting service or cloud provider goes insolvent it is unlikely to cause immediate loss of service, but it is possible - imagine how much the bandwidth costs are for the company when they announce and uneducated or unprepared people start pulling their data back for whatever reason.

More likely scenario is service disruptions, and they can last for days or weeks: severe ones have happened to Amazon cloud quite recently. If you run a business, can you afford to not have access to your backups for weeks on end, and/or not be able to backup your new images without signing up for another costly service? Note it takes time to move those terabytes of data to the new provider, so you need a backup of backup plan.

Also, your backup might be shared on a server or storage unit along with other content, and the police can confiscate an entire rack along with your data even if it had nothing to do with the case. It has also happened. Absolutely no warning in these cases, and it can again take weeks or months for the data to be recovered, if ever.

I'm not saying you recommended a cloud-only solution, just wanted to re-iterate that online-only is not a suitable backup.
Title: Re: Digital- to-Film for 50-year archive
Post by: Doug Peterson on October 10, 2011, 03:47:44 pm
TIFF in 2071?

It's very easy to bundle file-compatibility and hardware-compatibility together but they are vastly different things.

Yes, TIFF will be supported in 2071. If for no other reason than it's an incredibly simple format to read/write. 

Doug Peterson (e-mail Me) (doug@captureintegration.com)
__________________

Head of Technical Services, Capture Integration
Phase One Partner of the Year
Leaf, Leica, Cambo, Arca Swiss, Canon, Apple, Profoto, Broncolor, Eizo & More

National: 877.217.9870  |  Cell: 740.707.2183
Newsletter (http://"http://www.captureintegration.com/our-company/newsletters/") | RSS Feed (http://"http://www.captureintegration.com/2008/08/11/rss-feeds/")
Buy Capture One 6 at 10% off (http://"http://www.captureintegration.com/phase-one/buy-capture-one/")

Masters Series Workshop:
New England Landscape - Fall Color (Oct 5-8) (http://"http://www.captureintegration.com/2011/06/30/nelandscape/")
Title: Re: Digital- to-Film for 50-year archive
Post by: Wayne Fox on October 10, 2011, 09:40:13 pm
For private use, hard drives are the best bet,
for long term storage hard drives are iffy.  The OP was talking about finding something comparable to sticking a box in a closet for 50 years and have it be preserved.  A hard drive that has not been spun up in fifty years may not have much of a chance of even spinning up.  So hard drive storage implies maintenance and continual recopying over the decades.

I think SSD drives are currently the most physically durable storage device out there.  A SSD drive that has been written to and verified may see no degradation at all ... there is really nothing to degrade.  (I've done a little research on this and am basing my conclusions on what I've read, but I certainly may be wrong)

Of course, the interface may no longer be usable.  Think about trying to get data off of a SCSI drive now ... challenging and getting harder.

Personally I use SSD drives as my main backup system ... they are written to with the raw files then stored in my safety deposit box. But they aren't meant to be something for someone to find in 50 years, more just my backup in case I have a meltdown at my house.

 Cloud computing looks like a nice answer except the upload bandwidth limitations by most providers.  Hopefully this will change.

Title: Re: Digital- to-Film for 50-year archive
Post by: elf on October 11, 2011, 12:54:59 am
I'd say the "cloud" is the least likely storage strategy to succeed.  My ISP was purchased by another company that decided to change all of the urls.  My pages disappeared with no trace...

Another point to ponder: Will an online storage service continue to store data if there is no income for doing so?
Title: Re: Digital- to-Film for 50-year archive
Post by: Wayne Fox on October 11, 2011, 01:29:53 am
I'd say the "cloud" is the least likely storage strategy to succeed.  My ISP was purchased by another company that decided to change all of the urls.  My pages disappeared with no trace...

Another point to ponder: Will an online storage service continue to store data if there is no income for doing so?
I think cloud computing is in its infancy.  It's different than just storing things on an online storage facility or on an ISP's server.  I think it has a better chance of succeeding than those.

As far as your second point, I agree.  As I mentioned in my original post, unless there is value in what you are trying to preserve it doesn't have much of a chance.  What remains in 50 years will be things that people care about, or perhaps the OP premise is right, something that can be lost in an attic and "discovered" 50 years from now which might then be regarded as a treasure.
Title: Re: Digital- to-Film for 50-year archive
Post by: Ajoy Roy on October 11, 2011, 02:32:32 am
Come what may, if the film is still whole and picture visible, there will always be a method to transform it to a digital format. The same cannot be said of the various digital archive formats. In our life time we have seen formats and media come and go, with the attendant problems in recovering the data. To preserve digital data you have to be actively involved in renewing (and preserving the readability) of the data, where as film and paper can be (figuratively) left in the attic to be discovered at a later date (and interpreted visually without taking any recourse to technology).
Title: Re: Digital- to-Film for 50-year archive
Post by: Zerui on October 11, 2011, 04:50:54 am
This topic is becoming better every day.
Here are some further reactions to recent posts.

1.  SSD.   
     
     I am taken by Wayne's suggestion to keep one's personal archive on a RAID SSD system.
     I propose to do that as the first step in my photo-posterity programme.

2.  Cloud data services (1) provided by a Commercial enterprise. 
   
    Great for the here-and-now, but there is the probability they will let one down on a 50 year timescale.
    Even big companies like Google may not be around in 2071, let alone in 2111.
    I can image a one time up-front payment, rather than monthly subscription.
    But I cannot be sure that the service provider will honour the commitment for fifty years.
__________________________________________________________________________________
    The following horrifying anecdote may help explain my concern:
    When he was a student my son was employed during the summer vacation by a major highstreet bank to perform triage on customers' boxes in their vault.
    The triage went as follows:
     - If the customer still had an active account his box was kept in the local highstreet branch.
     - If the customer no longer had an active account, the box was sent to a central warehouse, miles away from the customers' highstreet branch.
     - If the label had been lost the box was destroyed.  No attempt was made to trace the owner or his descendants by examining the contents.
     The triage decision was made by three young students hired during their summer vacation.
     The bank is one of the biggest in UK, and has been in business for well over 100 years and looks likely to stay in business for another century.
     Its bosses seem to have abandoned their duty of caring for boxes customers lodged for safety in their vault.
__________________________________________________________________________________

3.  Cloud data services (2) provided by a National Institution (e.g. British Library).

     I can see a deal by which I pay them up front to maintain my digital image files for ever.
     In a sense I would have donated my photographs to the Institution with a legacy to fund their preservation.
     I assume my files would be listed in the Catalogue of the National Institution.
     And that anyone who wants to see them can do so over the web, and download copies for a fee (costs plus a fee - if any - to the IPR owners).
     The issue of copyright and licence-to-reproduce can be negotiated when the original contract is signed.
     I would trust such a system if it existed.
     It could bcome a nice earner for the National Institution.
     "Give us your files plus a legacy and we'll keep them for posterity and make them available to all-comers, while protecting your copyright."

Does anyone work with National Institutions that have implemented a programme digitizing their images for posterity.
How does it work in practice? 
Is there scope for a public service riding on their in-house operation?
i.e. a service for preserving digital images that the Institution does not itself choose to acquire.

John
   
Title: Re: Digital- to-Film for 50-year archive
Post by: theguywitha645d on October 11, 2011, 10:21:43 am
for long term storage hard drives are iffy.  The OP was talking about finding something comparable to sticking a box in a closet for 50 years and have it be preserved.  A hard drive that has not been spun up in fifty years may not have much of a chance of even spinning up.  So hard drive storage implies maintenance and continual recopying over the decades.

I think SSD drives are currently the most physically durable storage device out there.  A SSD drive that has been written to and verified may see no degradation at all ... there is really nothing to degrade.  (I've done a little research on this and am basing my conclusions on what I've read, but I certainly may be wrong)

Of course, the interface may no longer be usable.  Think about trying to get data off of a SCSI drive now ... challenging and getting harder.

Personally I use SSD drives as my main backup system ... they are written to with the raw files then stored in my safety deposit box. But they aren't meant to be something for someone to find in 50 years, more just my backup in case I have a meltdown at my house.

 Cloud computing looks like a nice answer except the upload bandwidth limitations by most providers.  Hopefully this will change.



I may not have been clear about copying forward. No, I would not trust a hard drive to sit for 50 years. But rather you need to have a migration plan and keep moving the data to new media/drives.

The SSD look really good and I have been doing some research. From what I understand, it is easier to pull data of a failed HD than a SSD--you can even take the disk out of the drive and read it. I certainly have had faulty flash memory, but I am sure the SSD are more stable. I have not been able to find a good answer to which is better.

I think photographers are just going to have to wade through this stuff and figure it out. The problem is that you will never know a system is good until it fails. I work with an archive and this has always been an interesting topic to talk with them about. They are also having to solve the same problems except they can have many more file types to deal with. With proprietary file types the archives understand that the technology to read them may simply disappear and so the files will be essentially worthless and the archives know that someday that data might simply be dumped, but things like TIFF, JPEG, and PDF should always be readable. I hope you don't have Word Perfect files.
Title: Re: Digital- to-Film for 50-year archive
Post by: feppe on October 11, 2011, 01:17:03 pm
If the idea is to store the SSDs for 50+ years in a box with the hope of recovering the data from them, I'd be more worried about compatibility and interfacing with future computers than data loss and readability. To put things into perspective, good luck trying to read punch cards these days. Even when 50+ year data recovery from a dead system can be done, it can be a huge undertaking (http://www.nasa.gov/topics/moonmars/features/LOIRP/). As Wayne pointed out, SCSI HDDs are already getting iffy, and they were common just 20 years ago.

Recovering data from dead HDDs is possible, but I'd urge you to check just how much it costs before putting any faith in that - it costs a fortune.

Fortunately current standards, are, well, standard, and wide-spread. But many of them are proprietary which means that the specifications needed to reverse engineer them could be locked up in a company vault somewhere, or lost altogether due to bankruptcies.

Proper rolling forward of all data mediums is necessary for peace of mind, ie. floppies to CD-ROMs to tapes to HDDs to SSDs to holographic memory etc.

For the store and forget, solution for the paranoid is to store the SSDs with a complete computer in a climate-controlled environment, but that introduces many more points of potential fault and costs a lot.
Title: Re: Digital- to-Film for 50-year archive
Post by: theguywitha645d on October 11, 2011, 02:05:20 pm
Recovering data from dead HDDs is possible, but I'd urge you to check just how much it costs before putting any faith in that - it costs a fortune.

But much less than taking them all over again.  ;D

I am not sure the right way. Like I said, you are only going to know when you have done it wrong. SSD are great, but also expensive, so the size of the archive is important. But in 5 years a 23 petabyte organically composted supercolor RAID 27 array will cost $37.50. But then you will need it for your 180MB compact camera.

I think planning storage now for the 50 years is going to be tough. Predicting is hard to do, especially about the future.
Title: Re: Digital- to-Film for 50-year archive
Post by: theguywitha645d on October 11, 2011, 02:07:40 pm
But much less than taking them all over again.  ;D

I am not sure the right way. Like I said, you are only going to know when you have done it wrong. SSD are great, but also expensive, so the size of the archive is important. But in 5 years a 23 petabyte organically composted supercolor RAID 27 psychically-connected array will cost $37.50. But then you will need it for your 180MB compact camera.

I think planning storage now for the 50 years is going to be tough. Predicting is hard to do, especially about the future.
Title: Re: Digital- to-Film for 50-year archive
Post by: Ajoy Roy on October 11, 2011, 11:46:21 pm
For all its faults, magnetic media is more stable and data recovery friendly compared to other digital storage media. I have songs recorded on 1/4" reel tapes which are still retrievable after fifty years, of course on an ancient tape recorder. SSD and Flash memories on the other hand require precise technology to read out, and are more prone to failure and data corruption.

I would rule out RAID configurations, as any problem with the controller and all your data is gone. I have faced this when proprietary RIAD controller failed, and no newer generation RAID controller could retrieve the data. It is safer to keep data on individual disks; two or three copies if you are paranoid; as failure of a single disk/controller will not take down the whole archive. Further magnetic media retains information deep in its substrate which is recoverable, albeit at a cost (that is how security agencies recover data from wiped and formatted disks). For long time archiving, the less technology required to preserve and retrieve the better.
Title: Re: Digital- to-Film for 50-year archive
Post by: John R Smith on October 12, 2011, 06:07:14 am
I have thought about this problem a great deal, both on a personal level and also as part of my professional duties. The simple fact is that as of today, there is no credible digital archiving strategy which does not require periodic intervention by the curator. None.

And from a museum or even a personal archive perspective, it is not at all unreasonable to expect and demand an archiving method which allows for (controlled environment) storage for a period of fifty years without intervention. At present, the only way in which a photographer can be sure of this is to archive their work as finished prints or as large-format film negatives or positives (with quite a lot of caveats for colour transparencies).

I am at present re-working some transcripts of interviews I made in the early 1980s. It's just as well that I typed them all up on plain paper with my old Remington - if they had been on some sort of digital media I would not have had anything to read them up on  ;)

John
Title: Re: Digital- to-Film for 50-year archive
Post by: amsp on October 12, 2011, 06:52:09 am
Look, I think it boils down to one thing really, namely is there anyone to care for your photos when you die? If nobody cares to preserve your photos after you die it's highly unlikely they will survive 50 years no matter what format they're in. So, if you have kids make sure they're interested in preserving them and know how to, if you don't try to donate them to some museum or foundation. Either way I think digital is going to be the easiest format to preserve going forward, there's a reason why museums and national archives are digitizing their collections.
Title: Re: Digital- to-Film for 50-year archive
Post by: ondebanks on October 12, 2011, 07:20:21 am
I wouldn't rely on the "put the digital medium in a vault" approach - especially if the medium is based on electrical or magnetic storage - HDD, SSD, tape, entire PC.
Because of http://en.wikipedia.org/wiki/Bit_rot (http://en.wikipedia.org/wiki/Bit_rot)
It doesn't take many bit corruptions to make entire files unreadable. The longer the medium sits passively in the vault, the more bit corruptions can occur, uncorrected.

I think the safest bet is active, switched-on HDDs or SSDs. These can have frequent (scheduled) periodic disk-checks, which self-correct bit corruptions (before they get too numerous to correct) and flag any files which appear to be uncorrectable. Those files can be recopied (synched) from secondary HDDs/SSDs, and the corruption clock is reset again. This is the approach that I use. And of course, maintain a geographical separation between the primary and secondary (and tertiary...) disks, to cover the "fire and flood" risk.

I don't know a lot about "cloud" storage solutions, but I imagine that this is also how they work? I.e. always-on, always-online, self-checking redundant farms of HDDs?

For all its faults, magnetic media is more stable and data recovery friendly compared to other digital storage media. I have songs recorded on 1/4" reel tapes which are still retrievable after fifty years, of course on an ancient tape recorder.

That's because they are analog recordings - not digital files. Localised tape damage or degradation does not destroy the entire song.
OTOH I have lost entire backups on a magnetic DAT tape, because corruption occured in a key place near the start of the tape.

Ray
Title: Re: Digital- to-Film for 50-year archive
Post by: Ajoy Roy on October 12, 2011, 08:32:12 am
That's because they are analog recordings - not digital files. Localised tape damage or degradation does not destroy the entire song.
OTOH I have lost entire backups on a magnetic DAT tape, because corruption occured in a key place near the start of the tape.

Ray

That is how the topic started. Film is an analog medium, and inspite of all its faults, analog media fails gracefully and not catastrophically ;D You can always recover some data from analog.

I still think that film is a better archive material, digital is yet not mature enough.
Title: Re: Digital- to-Film for 50-year archive
Post by: feppe on October 12, 2011, 01:08:59 pm
That is how the topic started. Film is an analog medium, and inspite of all its faults, analog media fails gracefully and not catastrophically ;D You can always recover some data from analog.

I still think that film is a better archive material, digital is yet not mature enough.

Digital is much better with proper data management. You only have 1 copy of a film unless you do in-camera dupes, and when that's gone, the image is gone. With digital you can have multiple literally identical copies in multiple locations.

But yes, for 50+ year store-and-forget archival it's clear that film or prints is the only way.
Title: Re: Digital- to-Film for 50-year archive
Post by: hjulenissen on October 13, 2011, 03:08:40 pm
That is how the topic started. Film is an analog medium, and inspite of all its faults, analog media fails gracefully and not catastrophically ;D You can always recover some data from analog.

I still think that film is a better archive material, digital is yet not mature enough.
You could of course save your digital image as punch cards. Choose an image format that is really easy to decode (i.e. divide the image into equal sized blocks, and encode the rgb values of each pixel directly as numerical values on the punch cards). By labelling each card with file name and block index, it should be fairly easy for anyone to decode it sometime in the future. (although the sRGB definition may be lost, let us hope that mankind still know how the l/m/s cones of our vision works).

There is a possibility that some cards are lost, eaten by rats or damaged by moisture. A simple solution is to have duplication. Another possibility is to encode the image at progressively lower resolutions (20MP, 10MP, 5MP,...). That way, loss off high-resolution data can be partially concealed by using low-resolution data.


All physical media are prone to damage. What we are talking about now is (in my view) not analog vs digital, but media technology that is so evident that any civilization at out present level of sophistication would be expected to be able to decode it easily. Much like the Voyager Golden Record:
 http://en.wikipedia.org/wiki/Voyager_Golden_Record

-h
Title: Re: Digital- to-Film for 50-year archive
Post by: Ajoy Roy on October 14, 2011, 08:30:29 am
The problem is that our five senses perceive and interpret Analog and not Digital data. Digital storage requires conversion to analog for us to interpret the information - visual, audio, pressure or smell. I assume that 100 or even 10,000 years hence our senses will still be in analog mode.

So along with digital archive, we will always require the decoding hardware/software combo, where as with analog all that we require is the media. At present the film longevity is similar to digital recording media. The same cannot be said of the digital-to-analog conversion of data. Monotonically as newer hardware and software become popular, older ones fall into disuse and finally disappear.

Till a standard for archives is established and accepted by all, digital archive is wholly dependent on the archivist maintaining the archive, an iffy situation at the best.
Title: Re: Digital- to-Film for 50-year archive
Post by: keithrsmith on October 14, 2011, 08:55:50 am
I believe that the film industry actually archive onto 3 colour separation films - i.e B&W- which gets over the achive properties of the colour dyes.
- and then put these into a temperature controller eviroment.

keith
Title: Re: Digital- to-Film for 50-year archive
Post by: shelby_lewis on October 16, 2011, 10:43:33 am
I'm with those that say to print the significant images (hey, you can actually enjoy them in your present life). If printed with a high-end inkjet with archival inks on non-OBA-utilizing paper that is rated for permanence, you'll have a stable archive that can be reproduced easily (more prints) for storage in multiple locations. Just choose a printer/paper combination that offers the best combo of resolution, color accuracy, and permanence and you'll be set. On the plus side, the printer can be repurposed via print sales to offset its own costs if your work is good enough to sell at a profit.

In the end, the whole idea of archiving has IMO become a bit of an overblown stressor on us photographers. Commercial work? Sure (for a while)... personal work? Sure (in moderation). But, I think most of us (me for sure) have somehow gotten into the mindset that everything we shoot is precious. I've got TBs of images in archive, and I bet very few will even be worth saving for later generations.

Good luck!
Title: Re: Digital- to-Film for 50-year archive
Post by: hjulenissen on October 21, 2011, 03:18:41 pm
The problem is that our five senses perceive and interpret Analog and not Digital data.
If you want to be pedantic about it, all of our senses are based on sending discrete (digital) information to our brain, which is itself based on a discrete number of components signalling in discrete ways. Further, the dot-pattern used in some inkjet printer could be seen as digital (either a dot, or no dot).

I maintain that it is not a question about analog vs digital, but about formats that are easy to reconstruct vs those that are less easy to reconstruct. For humans in 10 years, or an alien civilization in 1000 years, depending on how difficult you want to do it.

-h
Title: Re: Digital- to-Film for 50-year archive
Post by: Fine_Art on October 22, 2011, 05:07:57 pm
I pulled an old backup hard-drive from a safe to look at some old pictures. There are entire date folders that are empty. I'm not too worried at this point as they were also on the laptop, DVD and  another backup hard-drive. Anyone putting all their faith in an electronic device with a 5 yr warranty is deluding themselves with the convenience. You will lose everything, it is a matter of a few years.
Title: Re: Digital- to-Film for 50-year archive
Post by: cng on October 23, 2011, 05:38:53 am
I pulled an old backup hard-drive from a safe to look at some old pictures. There are entire date folders that are empty. I'm not too worried at this point as they were also on the laptop, DVD and  another backup hard-drive. Anyone putting all their faith in an electronic device with a 5 yr warranty is deluding themselves with the convenience. You will lose everything, it is a matter of a few years.

Bit rot (aka data evaporation) is a danger with all digital media, whether magnetic, optical or flash-based.  The reasons differ with each form of storage media but the consequences are still the same.  A quick primer:
http://en.wikipedia.org/wiki/Bit_rot

Having said that, physical media such as film and prints also need their own set of optimal conditions to be suitably archived.  As an example, merely storing these in dark conditions may not avoid fading since some dyes are prone to dark-fading.

Then there are the constant dangers of humidity, heat, vermin, theft, fire etc., which pose a danger to all forms of media.

I still stand by my prior post recommending prints as the preferred long-term archival medium (suitably printed and stored, of course, and ideally in combination with other archival strategies/media).  Edit ruthlessly and print only your strongest images, and you not only have your archive but also a coherent summary of your life's work ready to be viewed without the need for intermediary technology such as a lightbox, enlarger, slide projector, monitor, computer, scanner, printer etc.
Title: Re: Digital- to-Film for 50-year archive
Post by: ondebanks on October 28, 2011, 07:09:27 am
Anyone putting all their faith in an electronic device with a 5 yr warranty is deluding themselves with the convenience. You will lose everything, it is a matter of a few years.

That's why I said earlier that the hard disk needs to be switched on, active, and running periodic error checks.

If your reference to "a 5 year warranty" means that the mfgr warrantied that the disk would still be operational in 5 years, well it is.  You could probably reformat it and it would still work fine. If the warranty was for data integrity 5 years on from being written to the disk, you'd have to check the fine print - there might be an exception clause for when the file system is not actively on/maintained for a long period.

Ray