Pages: [1]   Go Down

Author Topic: Archiving digital images  (Read 567 times)

bobtrlin

  • Newbie
  • *
  • Offline Offline
  • Posts: 32
Archiving digital images
« on: November 23, 2017, 11:56:47 PM »

With Adobe's move to a subscription model and emphasis on cloud computing, the question of archiving and future proofing (Adobe proofing) images comes to the fore.  Archiving literature suggest that TIFF is the way to go.  File sizes are much larger but the longevity of support for this format seems better than for any other.  The question then arises does one convert to TIFF the original raw or the raw plus enhancements?  It seems to me that an essential part of the photographer's art is the PP so it makes sense to include that into the archival TIFF.  After all, to archive a painting, one doesn't scrape away the paint in order to preserve the original charcoal sketch underneath.  So, my intension is to convert all my enhanced images to TIFF and discard everything else.  Despite constant improvements in PP technology, I think it would be rare that I'd ever want to go back to an original raw to try something else.  I'd be interested to know what others think.

On the related matter of transportability, when Adobe was the golden child of PP, converting raws to DNG seemed a no brainer, at least to me.  There is now some serious competition to Lightroom and I'm wondering how wise it was to convert to DNG.  OK, in the interim, it seems unlikely that any other platform would not support DNG.  Nevertheless, it may have been wiser to stay with the native raw format.  Any thoughts?   
Logged

langier

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 872
    • Celebrating Rural America, the Balkans and beyond
Re: Archiving digital images
« Reply #1 on: November 24, 2017, 01:24:30 AM »

Now with about 20 years of digital behind me and still access to most of my early work, I say save it all.

Though DNG sounded good, I've hardly used it over the years except more recently to stitch panos and create HDR files, otherwise, I simply leave the raw files as is and most of my archives are this. Master files are usually layered TIFF files and derivative files are what ever they need to be for the final use, many JPEG from the TIFFs.

Because hard drives are large, fast and relatively cheap, there's little reason to toss files, other than to winnow out the chaff...

Countless times during the deadline of editing, I've missed both gems and found later that my choice of image was not the best. Having the deep and organized archives, nearly all with adequate metadata is a lifesaver.

An image I created this week required finding images I didn't remember I had. A simple keyword search yielded many more photos than I realized, thus making the final product richer. Had I dumped these photos as "obsolete", to save space or because the format may one day become orphaned, this little project would never have materialized.

However, another factor for "future proofing" one's work is to not only create a master file but to print it on paper since in centuries of use, paper seldom becomes a victim of digital rot and the march of technology.
Logged
Larry Angier
ASMP, ACT, and many more!

Webmaster, RANGE magazine
Editor emeritus, NorCal Quarterly

web--http://www.angier-fox.photoshelter.com facebook--larry.angier twitter--#larryangier [email protected]_Icons

rdonson

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 2661
Re: Archiving digital images
« Reply #2 on: November 24, 2017, 02:50:01 PM »

TIFF offers compression (ZIP and LZW). 

I've NEVER gotten rid of my RAW files. 

I've heard all the arguments for DNG and I haven't been moved to use it very much.  I do use DNG when I use Iridient Developer or X-Transformer as I round trip from Lightroom for better demosaicing. 

I'll consider DNG again when Canon, Nikon, Sony, Fuji, Panasonic, Olympus, et al take the file format seriously and offer it in their cameras. 

As stated by Langier, disk space, redundant drive enclosures and cloud storage are quite reasonable these days.  Take advantage of these.
Logged
Regards,
Ron

Farmer

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 2332
Re: Archiving digital images
« Reply #3 on: November 24, 2017, 03:29:16 PM »

I also did conversion to DNG years ago, but I also kept the raw files.  Actually, the idea was to never touch the raw files and work from the DNGs, but not to replace the raw files with DNG from an archival perspective.  I stopped doing it after a few years because it was a just another step and was just costing more disk space and time for no practical gain. I still have all the DNGs I created, but also every single raw (and JPEG for that matter, when I've used a non-raw capable camera or film scans - although I mostly scanned to TIFF).

At the end of the day - keep it all, keep multiple copies, keep copies offsite (cloud is one way), and follow a consistent strategy (even if it changes over time, be consistent with whatever is your current), and remember that RAID is not backup/archiving.
Logged
Phil Brown

bobtrlin

  • Newbie
  • *
  • Offline Offline
  • Posts: 32
Re: Archiving digital images
« Reply #4 on: November 24, 2017, 04:00:05 PM »

and remember that RAID is not backup/archiving.
Totally agree.  If a disc fails, replace it with no data loss.  However, if there is a glitch in the RAID software, you have a bank of discs with nothing but irrecoverable garbage.
Logged

bobtrlin

  • Newbie
  • *
  • Offline Offline
  • Posts: 32
Re: Archiving digital images
« Reply #5 on: November 24, 2017, 04:38:01 PM »

I guess I've confused the issue by combining two related issues here.  The first is transportability.  That is, future proofing images so that they and the metadata is more transportable between PP platforms.  The second is archiving for posterity.

Regarding transportability, if I chose to move to another platforms, I risk loosing hundreds of man-hours of enhancements and metadata particularly keywording hierarchy.  What is the best strategy?

By archiving, I'm talking about leaving data for posterity.  I have safe storage by double backups on separate HDD's.  However, leaving raws of various proprietary formats of companies that may no longer exist seems pointless.  Prints appear to have the longest longevity and I'll certainly do that for some but it's impractical to print all my images.  For this reason, I'm thinking for all images more than a few years old, convert them to TIFF and junk the rest.  As an amateur, it's highly unlikely I'll ever want to revisit and rework those images but I'll still have the TIFFs in any case.
Logged

Farmer

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 2332
Re: Archiving digital images
« Reply #6 on: November 24, 2017, 04:45:59 PM »

Convert to TIFF by all means, but why junk the raws?  All you are doing is saving a little space, which is extremely cheap now (put them on Amazon Glacier) and then if you ever do want to revisit and rework, you can.
Logged
Phil Brown

Moritz

  • Newbie
  • *
  • Offline Offline
  • Posts: 8
    • Moritz Schmittat Wedding Photographer
Re: Archiving digital images
« Reply #7 on: November 29, 2017, 05:44:34 AM »

Quick idea on how to save money on secure storage:

I also always keep all raw files. I produce about 10tb per year which I store on 4tb USB3 drives (the 2.5 size, they are powered by USB). I keep copies of everything on two drives always which means I'm ordering two identical drives every couple of months. I sync them with a clever program (on a Mac) called SyncFolder Pro - just copy everything to one drive and the software will make sure it's on both. So by default I have about 4 or 6 drives connected to a USB 3 hub (which I find fast enough). Basically this works like a raid system but MUCH cheaper plus I can easily take 'a bit' of data with me on a single drive if I need to get work done while travelling. The unconnected drives are safely stacked on a shelf (about 20 of them).

If you also want to safe data on your final images I can really recommend JPEGmini. It reduces the files by about half the size and I can never see any difference...
Logged
Learning with every shot I take.
Surrey Wedding Photographer

bobtrlin

  • Newbie
  • *
  • Offline Offline
  • Posts: 32
Re: Archiving digital images
« Reply #8 on: November 29, 2017, 04:31:34 PM »

I use SyncBackPro on Windows.  I guess it's similar to your SyncFolderPro.  It makes a straight copy of one disk to another.  That way one avoids the problem of proprietary backup software.  So often I've tried to recover data just to find that my backup software is no longer supported by the system.
Logged

Moritz

  • Newbie
  • *
  • Offline Offline
  • Posts: 8
    • Moritz Schmittat Wedding Photographer
Re: Archiving digital images
« Reply #9 on: November 30, 2017, 06:58:25 AM »

I also have two Google G Suite accounts. One via a school were I teach and one for my own business. If an organisation has more than 5 members (which the college definitely has) all users get UNLIMITED google drive storage. At the moment I have 9.2tb online on my teaching account without any problems. However, I won't work there forever and I'm consider moving everything to my G Suite account.

So I need more users.... would anybody be interested in sharing my G Suite account? It's 3.30 per month. So far I have three users (me, my wife and my assistant) so two more and ALL users will have unlimited storage.

Logged
Learning with every shot I take.
Surrey Wedding Photographer

BobShaw

  • Sr. Member
  • ****
  • Online Online
  • Posts: 1039
    • Aspiration Images
Re: Archiving digital images
« Reply #10 on: December 02, 2017, 06:42:52 PM »

Convert to TIFF by all means, but why junk the raws?  All you are doing is saving a little space, which is extremely cheap now (put them on Amazon Glacier) and then if you ever do want to revisit and rework, you can.
Completely agree. I use TIFF for print files and everything else is raw except work in progress which is psd.
If you don't have enough space for your raw files then you probably need to go back to film.
I have no interest in DNG. Hasselblad added it to their cameras about 10 years ago and then deleted it as limiting innovation. There used to be a very good explanation on their website which unfortunately has disappeared from it.

I don't believe in storing in the cloud. If you ever had to download the lot it would take months (at least where I live). My entire library can be restored overnight from a server.
Logged
Website - http://AspirationImages.com
Blog - http://AspirationImages.com/blog
Landscape, Portrait, Product Photography - Sydney, Australia

bobtrlin

  • Newbie
  • *
  • Offline Offline
  • Posts: 32
Re: Archiving digital images
« Reply #11 on: December 02, 2017, 09:13:21 PM »

Point taken.  You make a lot of sense.  On a point of clarification, I'm looking at TIFF as a format that is likely to survive all other.  However, you say you use TIFF to print from.  Is there a reason for this?  Surely you can print direct from your enhanced raw.
Logged

BobShaw

  • Sr. Member
  • ****
  • Online Online
  • Posts: 1039
    • Aspiration Images
Re: Archiving digital images
« Reply #12 on: December 03, 2017, 03:48:20 PM »

However, you say you use TIFF to print from.  Is there a reason for this?  Surely you can print direct from your enhanced raw.
Yes you can and I did for a couple of years. Then someone showed me Mirage Print and I have never printed from an editing programme since.
No more masses of presets that need to be redone if you upgrade or change printers or even change the connection method, or if you have a different size of paper or a different paper.
Set the paper once, set the printer once and it all just works like magic. Any size you specify printed exactly as it looks.

So I have single 16bit Prophoto TIFF which is my print file for each image.
Logged
Website - http://AspirationImages.com
Blog - http://AspirationImages.com/blog
Landscape, Portrait, Product Photography - Sydney, Australia

bobtrlin

  • Newbie
  • *
  • Offline Offline
  • Posts: 32
Re: Archiving digital images
« Reply #13 on: December 03, 2017, 04:14:22 PM »

Yes you can and I did for a couple of years. Then someone showed me Mirage Print and I have never printed from an editing programme since.
No more masses of presets that need to be redone if you upgrade or change printers or even change the connection method, or if you have a different size of paper or a different paper.
Set the paper once, set the printer once and it all just works like magic. Any size you specify printed exactly as it looks.

So I have single 16bit Prophoto TIFF which is my print file for each image.
Sound great!  I'll check it out.
Logged
Pages: [1]   Go Up