Pages: 1 [2] 3   Go Down

Author Topic: Digital- to-Film for 50-year archive  (Read 10709 times)

Ben Rubinstein

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 1822
Re: Digital- to-Film for 50-year archive
« Reply #20 on: October 10, 2011, 04:30:05 am »

Although in 100 years you may not be able to interface or even read today's digital media without significant expense, the same will be as true but a lot sooner with film. How many companies make decent multi format scanners even here and now in 2011? How many companies will make enlargers and chemistry to print those films in 2050? I'd be willing to hazard a guess that it will be the same number as the companies who will read your Kodak/Delkin 100 year DVD's or provide a service to read older standards of tape or SATA II drives. Difference being that it will be far cheaper to retrieve 2000 images from a tape (and far cheaper to have made multiple copies in the first place) than to scan and work 2000 negs or trannies in 2050. Just as it is today.

As an aside, with practically every museum and library worldwide having spent huge resources on the digitalisation of their collections to TIFF I very much doubt that you will have difficulty whatsoever in reading the TIFF file format a century from now.
Logged

Zerui

  • Jr. Member
  • **
  • Offline Offline
  • Posts: 59
    • Foto Zerui
Re: Digital- to-Film for 50-year archive
« Reply #21 on: October 10, 2011, 05:21:26 am »

Thank you to all who have contributed to addressing my question about photoposterity (50 years plus).
As I suspected, this forum reaches parts of the knowledge base that others miss.
And I suspect that many of its subscribers are producing work that might be appreciated in the far future
Think of Eugène Atget's photos documenting the fast disappearing old Paris.

Let me summarize my reactions so far:
1.  50 years is a really long time.  Longer than the half life of (1) data media and (2) businesses.
2.  I liked the suggestion that archiving in prints is probably better than film, on the grounds that film scanners may be rare in 2071.
     The prints will have to be at 100%, which makes them quite large for Medium Format photographs.
     So one has to consider the cost per print, and the storage problem.
3.  I was also impressed by the point that museums are scanning documents into TIFF files.
    National Museums will surely have a longevity that far exceeds the fifity years I posed.
    Museum staff can reformat, or change digital medium as needed to keep up with evolving technology.
    A Museum with its digital archive is likely to be more permanent than a commercial enterprise.
    If the British Museum or British Library (or the equivalent in your country) set up a service to maintain my digital files I would trust them for posterity.

So, the best bet seems to be either
(1) prints made and stored by me, or
(2) digital files maintained by a national museum/library that is likely to survive for ever.

Further thoughts on the general issue, or on these preliminary conclusions would be welcome.
John
Logged

TH_Alpa

  • Full Member
  • ***
  • Offline Offline
  • Posts: 214
Re: Digital- to-Film for 50-year archive
« Reply #22 on: October 10, 2011, 06:03:37 am »

Honestly, I don't see how anything could be safer than saving them in TIFF with an online hosting company. It's not like the format is going to die and be replaced overnight and all of a sudden all programs that supported it will decide to no longer be backwards compatible. Same thing goes for a company offering the archival service, they don't close down business and erase your data out of the blue. I've used the same hosting service for 14 years now for example. The cloud is your safest bet IMO, with copies stored locally on your HD.
+1

Thierry
Logged

theguywitha645d

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 970
Re: Digital- to-Film for 50-year archive
« Reply #23 on: October 10, 2011, 10:30:42 am »

CDs and DVDs are not archival. These media are not for long-term storage.

For private use, hard drives are the best bet, but you will have to have a plan to copy forward files--drive disks will corrupt as well. I use a RAID array. The cloud is another good option as they will be using more sophisticated drives and systems. Certainly have the images in more than one place.

TIFF, JPEG, and PDF are considered archival formats that can carry forward. RAW images should be converted to TIFF, but I would archive the RAW as well--If you have the RAW profile, you should always be able to process them.

The surest way to preserve your images is to become really famous.
« Last Edit: October 10, 2011, 10:32:26 am by theguywitha645d »
Logged

feppe

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 2906
  • Oh this shows up in here!
    • Harri Jahkola Photography
Re: Digital- to-Film for 50-year archive
« Reply #24 on: October 10, 2011, 12:06:14 pm »

Honestly, I don't see how anything could be safer than saving them in TIFF with an online hosting company. It's not like the format is going to die and be replaced overnight and all of a sudden all programs that supported it will decide to no longer be backwards compatible. Same thing goes for a company offering the archival service, they don't close down business and erase your data out of the blue. I've used the same hosting service for 14 years now for example. The cloud is your safest bet IMO, with copies stored locally on your HD.

I agree mostly. The key is to have multiple backups in multiple locations. I've chosen always-on HDD, and offline backup to external HDDs (weekly on site, monthly kept at the office to cover for theft/fire/water damage) and online cloud (Crashplan) for redundancy.

But I would never ever trust the cloud as only backup. If a hosting service or cloud provider goes insolvent it is unlikely to cause immediate loss of service, but it is possible - imagine how much the bandwidth costs are for the company when they announce and uneducated or unprepared people start pulling their data back for whatever reason.

More likely scenario is service disruptions, and they can last for days or weeks: severe ones have happened to Amazon cloud quite recently. If you run a business, can you afford to not have access to your backups for weeks on end, and/or not be able to backup your new images without signing up for another costly service? Note it takes time to move those terabytes of data to the new provider, so you need a backup of backup plan.

Also, your backup might be shared on a server or storage unit along with other content, and the police can confiscate an entire rack along with your data even if it had nothing to do with the case. It has also happened. Absolutely no warning in these cases, and it can again take weeks or months for the data to be recovered, if ever.

I'm not saying you recommended a cloud-only solution, just wanted to re-iterate that online-only is not a suitable backup.

Doug Peterson

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 4210
    • http://www.doug-peterson.com
Re: Digital- to-Film for 50-year archive
« Reply #25 on: October 10, 2011, 03:47:44 pm »

TIFF in 2071?

It's very easy to bundle file-compatibility and hardware-compatibility together but they are vastly different things.

Yes, TIFF will be supported in 2071. If for no other reason than it's an incredibly simple format to read/write. 

Doug Peterson (e-mail Me)
__________________

Head of Technical Services, Capture Integration
Phase One Partner of the Year
Leaf, Leica, Cambo, Arca Swiss, Canon, Apple, Profoto, Broncolor, Eizo & More

National: 877.217.9870  |  Cell: 740.707.2183
Newsletter | RSS Feed
Buy Capture One 6 at 10% off

Masters Series Workshop:
New England Landscape - Fall Color (Oct 5-8)

Wayne Fox

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 4237
    • waynefox.com
Re: Digital- to-Film for 50-year archive
« Reply #26 on: October 10, 2011, 09:40:13 pm »

For private use, hard drives are the best bet,
for long term storage hard drives are iffy.  The OP was talking about finding something comparable to sticking a box in a closet for 50 years and have it be preserved.  A hard drive that has not been spun up in fifty years may not have much of a chance of even spinning up.  So hard drive storage implies maintenance and continual recopying over the decades.

I think SSD drives are currently the most physically durable storage device out there.  A SSD drive that has been written to and verified may see no degradation at all ... there is really nothing to degrade.  (I've done a little research on this and am basing my conclusions on what I've read, but I certainly may be wrong)

Of course, the interface may no longer be usable.  Think about trying to get data off of a SCSI drive now ... challenging and getting harder.

Personally I use SSD drives as my main backup system ... they are written to with the raw files then stored in my safety deposit box. But they aren't meant to be something for someone to find in 50 years, more just my backup in case I have a meltdown at my house.

 Cloud computing looks like a nice answer except the upload bandwidth limitations by most providers.  Hopefully this will change.

Logged

elf

  • Full Member
  • ***
  • Offline Offline
  • Posts: 244
Re: Digital- to-Film for 50-year archive
« Reply #27 on: October 11, 2011, 12:54:59 am »

I'd say the "cloud" is the least likely storage strategy to succeed.  My ISP was purchased by another company that decided to change all of the urls.  My pages disappeared with no trace...

Another point to ponder: Will an online storage service continue to store data if there is no income for doing so?
Logged

Wayne Fox

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 4237
    • waynefox.com
Re: Digital- to-Film for 50-year archive
« Reply #28 on: October 11, 2011, 01:29:53 am »

I'd say the "cloud" is the least likely storage strategy to succeed.  My ISP was purchased by another company that decided to change all of the urls.  My pages disappeared with no trace...

Another point to ponder: Will an online storage service continue to store data if there is no income for doing so?
I think cloud computing is in its infancy.  It's different than just storing things on an online storage facility or on an ISP's server.  I think it has a better chance of succeeding than those.

As far as your second point, I agree.  As I mentioned in my original post, unless there is value in what you are trying to preserve it doesn't have much of a chance.  What remains in 50 years will be things that people care about, or perhaps the OP premise is right, something that can be lost in an attic and "discovered" 50 years from now which might then be regarded as a treasure.
Logged

Ajoy Roy

  • Full Member
  • ***
  • Offline Offline
  • Posts: 117
Re: Digital- to-Film for 50-year archive
« Reply #29 on: October 11, 2011, 02:32:32 am »

Come what may, if the film is still whole and picture visible, there will always be a method to transform it to a digital format. The same cannot be said of the various digital archive formats. In our life time we have seen formats and media come and go, with the attendant problems in recovering the data. To preserve digital data you have to be actively involved in renewing (and preserving the readability) of the data, where as film and paper can be (figuratively) left in the attic to be discovered at a later date (and interpreted visually without taking any recourse to technology).
Logged
Ajoy Roy, image processing

Zerui

  • Jr. Member
  • **
  • Offline Offline
  • Posts: 59
    • Foto Zerui
Re: Digital- to-Film for 50-year archive
« Reply #30 on: October 11, 2011, 04:50:54 am »

This topic is becoming better every day.
Here are some further reactions to recent posts.

1.  SSD.   
     
     I am taken by Wayne's suggestion to keep one's personal archive on a RAID SSD system.
     I propose to do that as the first step in my photo-posterity programme.

2.  Cloud data services (1) provided by a Commercial enterprise. 
   
    Great for the here-and-now, but there is the probability they will let one down on a 50 year timescale.
    Even big companies like Google may not be around in 2071, let alone in 2111.
    I can image a one time up-front payment, rather than monthly subscription.
    But I cannot be sure that the service provider will honour the commitment for fifty years.
__________________________________________________________________________________
    The following horrifying anecdote may help explain my concern:
    When he was a student my son was employed during the summer vacation by a major highstreet bank to perform triage on customers' boxes in their vault.
    The triage went as follows:
     - If the customer still had an active account his box was kept in the local highstreet branch.
     - If the customer no longer had an active account, the box was sent to a central warehouse, miles away from the customers' highstreet branch.
     - If the label had been lost the box was destroyed.  No attempt was made to trace the owner or his descendants by examining the contents.
     The triage decision was made by three young students hired during their summer vacation.
     The bank is one of the biggest in UK, and has been in business for well over 100 years and looks likely to stay in business for another century.
     Its bosses seem to have abandoned their duty of caring for boxes customers lodged for safety in their vault.
__________________________________________________________________________________

3.  Cloud data services (2) provided by a National Institution (e.g. British Library).

     I can see a deal by which I pay them up front to maintain my digital image files for ever.
     In a sense I would have donated my photographs to the Institution with a legacy to fund their preservation.
     I assume my files would be listed in the Catalogue of the National Institution.
     And that anyone who wants to see them can do so over the web, and download copies for a fee (costs plus a fee - if any - to the IPR owners).
     The issue of copyright and licence-to-reproduce can be negotiated when the original contract is signed.
     I would trust such a system if it existed.
     It could bcome a nice earner for the National Institution.
     "Give us your files plus a legacy and we'll keep them for posterity and make them available to all-comers, while protecting your copyright."

Does anyone work with National Institutions that have implemented a programme digitizing their images for posterity.
How does it work in practice? 
Is there scope for a public service riding on their in-house operation?
i.e. a service for preserving digital images that the Institution does not itself choose to acquire.

John
   
Logged

theguywitha645d

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 970
Re: Digital- to-Film for 50-year archive
« Reply #31 on: October 11, 2011, 10:21:43 am »

for long term storage hard drives are iffy.  The OP was talking about finding something comparable to sticking a box in a closet for 50 years and have it be preserved.  A hard drive that has not been spun up in fifty years may not have much of a chance of even spinning up.  So hard drive storage implies maintenance and continual recopying over the decades.

I think SSD drives are currently the most physically durable storage device out there.  A SSD drive that has been written to and verified may see no degradation at all ... there is really nothing to degrade.  (I've done a little research on this and am basing my conclusions on what I've read, but I certainly may be wrong)

Of course, the interface may no longer be usable.  Think about trying to get data off of a SCSI drive now ... challenging and getting harder.

Personally I use SSD drives as my main backup system ... they are written to with the raw files then stored in my safety deposit box. But they aren't meant to be something for someone to find in 50 years, more just my backup in case I have a meltdown at my house.

 Cloud computing looks like a nice answer except the upload bandwidth limitations by most providers.  Hopefully this will change.



I may not have been clear about copying forward. No, I would not trust a hard drive to sit for 50 years. But rather you need to have a migration plan and keep moving the data to new media/drives.

The SSD look really good and I have been doing some research. From what I understand, it is easier to pull data of a failed HD than a SSD--you can even take the disk out of the drive and read it. I certainly have had faulty flash memory, but I am sure the SSD are more stable. I have not been able to find a good answer to which is better.

I think photographers are just going to have to wade through this stuff and figure it out. The problem is that you will never know a system is good until it fails. I work with an archive and this has always been an interesting topic to talk with them about. They are also having to solve the same problems except they can have many more file types to deal with. With proprietary file types the archives understand that the technology to read them may simply disappear and so the files will be essentially worthless and the archives know that someday that data might simply be dumped, but things like TIFF, JPEG, and PDF should always be readable. I hope you don't have Word Perfect files.
« Last Edit: October 11, 2011, 10:27:33 am by theguywitha645d »
Logged

feppe

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 2906
  • Oh this shows up in here!
    • Harri Jahkola Photography
Re: Digital- to-Film for 50-year archive
« Reply #32 on: October 11, 2011, 01:17:03 pm »

If the idea is to store the SSDs for 50+ years in a box with the hope of recovering the data from them, I'd be more worried about compatibility and interfacing with future computers than data loss and readability. To put things into perspective, good luck trying to read punch cards these days. Even when 50+ year data recovery from a dead system can be done, it can be a huge undertaking. As Wayne pointed out, SCSI HDDs are already getting iffy, and they were common just 20 years ago.

Recovering data from dead HDDs is possible, but I'd urge you to check just how much it costs before putting any faith in that - it costs a fortune.

Fortunately current standards, are, well, standard, and wide-spread. But many of them are proprietary which means that the specifications needed to reverse engineer them could be locked up in a company vault somewhere, or lost altogether due to bankruptcies.

Proper rolling forward of all data mediums is necessary for peace of mind, ie. floppies to CD-ROMs to tapes to HDDs to SSDs to holographic memory etc.

For the store and forget, solution for the paranoid is to store the SSDs with a complete computer in a climate-controlled environment, but that introduces many more points of potential fault and costs a lot.

theguywitha645d

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 970
Re: Digital- to-Film for 50-year archive
« Reply #33 on: October 11, 2011, 02:05:20 pm »

Recovering data from dead HDDs is possible, but I'd urge you to check just how much it costs before putting any faith in that - it costs a fortune.

But much less than taking them all over again.  ;D

I am not sure the right way. Like I said, you are only going to know when you have done it wrong. SSD are great, but also expensive, so the size of the archive is important. But in 5 years a 23 petabyte organically composted supercolor RAID 27 array will cost $37.50. But then you will need it for your 180MB compact camera.

I think planning storage now for the 50 years is going to be tough. Predicting is hard to do, especially about the future.
Logged

theguywitha645d

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 970
Re: Digital- to-Film for 50-year archive
« Reply #34 on: October 11, 2011, 02:07:40 pm »

But much less than taking them all over again.  ;D

I am not sure the right way. Like I said, you are only going to know when you have done it wrong. SSD are great, but also expensive, so the size of the archive is important. But in 5 years a 23 petabyte organically composted supercolor RAID 27 psychically-connected array will cost $37.50. But then you will need it for your 180MB compact camera.

I think planning storage now for the 50 years is going to be tough. Predicting is hard to do, especially about the future.
Logged

Ajoy Roy

  • Full Member
  • ***
  • Offline Offline
  • Posts: 117
Re: Digital- to-Film for 50-year archive
« Reply #35 on: October 11, 2011, 11:46:21 pm »

For all its faults, magnetic media is more stable and data recovery friendly compared to other digital storage media. I have songs recorded on 1/4" reel tapes which are still retrievable after fifty years, of course on an ancient tape recorder. SSD and Flash memories on the other hand require precise technology to read out, and are more prone to failure and data corruption.

I would rule out RAID configurations, as any problem with the controller and all your data is gone. I have faced this when proprietary RIAD controller failed, and no newer generation RAID controller could retrieve the data. It is safer to keep data on individual disks; two or three copies if you are paranoid; as failure of a single disk/controller will not take down the whole archive. Further magnetic media retains information deep in its substrate which is recoverable, albeit at a cost (that is how security agencies recover data from wiped and formatted disks). For long time archiving, the less technology required to preserve and retrieve the better.
Logged
Ajoy Roy, image processing

John R Smith

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 1357
  • Still crazy, after all these years
Re: Digital- to-Film for 50-year archive
« Reply #36 on: October 12, 2011, 06:07:14 am »

I have thought about this problem a great deal, both on a personal level and also as part of my professional duties. The simple fact is that as of today, there is no credible digital archiving strategy which does not require periodic intervention by the curator. None.

And from a museum or even a personal archive perspective, it is not at all unreasonable to expect and demand an archiving method which allows for (controlled environment) storage for a period of fifty years without intervention. At present, the only way in which a photographer can be sure of this is to archive their work as finished prints or as large-format film negatives or positives (with quite a lot of caveats for colour transparencies).

I am at present re-working some transcripts of interviews I made in the early 1980s. It's just as well that I typed them all up on plain paper with my old Remington - if they had been on some sort of digital media I would not have had anything to read them up on  ;)

John
Logged
Hasselblad 500 C/M, SWC and CFV-39 DB
an

amsp

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 810
Re: Digital- to-Film for 50-year archive
« Reply #37 on: October 12, 2011, 06:52:09 am »

Look, I think it boils down to one thing really, namely is there anyone to care for your photos when you die? If nobody cares to preserve your photos after you die it's highly unlikely they will survive 50 years no matter what format they're in. So, if you have kids make sure they're interested in preserving them and know how to, if you don't try to donate them to some museum or foundation. Either way I think digital is going to be the easiest format to preserve going forward, there's a reason why museums and national archives are digitizing their collections.
Logged

ondebanks

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 858
Re: Digital- to-Film for 50-year archive
« Reply #38 on: October 12, 2011, 07:20:21 am »

I wouldn't rely on the "put the digital medium in a vault" approach - especially if the medium is based on electrical or magnetic storage - HDD, SSD, tape, entire PC.
Because of http://en.wikipedia.org/wiki/Bit_rot
It doesn't take many bit corruptions to make entire files unreadable. The longer the medium sits passively in the vault, the more bit corruptions can occur, uncorrected.

I think the safest bet is active, switched-on HDDs or SSDs. These can have frequent (scheduled) periodic disk-checks, which self-correct bit corruptions (before they get too numerous to correct) and flag any files which appear to be uncorrectable. Those files can be recopied (synched) from secondary HDDs/SSDs, and the corruption clock is reset again. This is the approach that I use. And of course, maintain a geographical separation between the primary and secondary (and tertiary...) disks, to cover the "fire and flood" risk.

I don't know a lot about "cloud" storage solutions, but I imagine that this is also how they work? I.e. always-on, always-online, self-checking redundant farms of HDDs?

For all its faults, magnetic media is more stable and data recovery friendly compared to other digital storage media. I have songs recorded on 1/4" reel tapes which are still retrievable after fifty years, of course on an ancient tape recorder.

That's because they are analog recordings - not digital files. Localised tape damage or degradation does not destroy the entire song.
OTOH I have lost entire backups on a magnetic DAT tape, because corruption occured in a key place near the start of the tape.

Ray
Logged

Ajoy Roy

  • Full Member
  • ***
  • Offline Offline
  • Posts: 117
Re: Digital- to-Film for 50-year archive
« Reply #39 on: October 12, 2011, 08:32:12 am »

That's because they are analog recordings - not digital files. Localised tape damage or degradation does not destroy the entire song.
OTOH I have lost entire backups on a magnetic DAT tape, because corruption occured in a key place near the start of the tape.

Ray

That is how the topic started. Film is an analog medium, and inspite of all its faults, analog media fails gracefully and not catastrophically ;D You can always recover some data from analog.

I still think that film is a better archive material, digital is yet not mature enough.
Logged
Ajoy Roy, image processing
Pages: 1 [2] 3   Go Up