Revisiting an Old Debate

[fike]

I have periodically debated converting my large archive of Canon CR2 files to DNG format.  To me, the debate boils down the the question of what format will be supported longer Canon CR2 or Adobe DNG.  I can't really come up with any reason that either company is superior in its long-term outlook, so by default, I have stuck with CR2 files.  

Another reason I was reluctant to make DNG conversion part of my workflow was the loss of Canon proprietary CR2 properties that can only be accessed through Canon DPP.  Though my workflow doesn't use DPP, I always figured that if I had a compelling reason to go back to it, it would be there if I used CR2 but not with DNG.  

Has the landscape in this area changed since I made this conclusion.  Today, is there any better reason to convert to DNG than two years ago?

[John R Smith]

The only thing which would swing the balance in favour of the DNG format would be if a significant number of camera makers themselves supported it. Thus far, only Leica (as far as I am aware) have chosen to do so.

The Tower of Babel situation we have now is obviously insane. It cannot survive into the future with any hope of backwards compatibility. But unless the camera makers are willing to forego their proprietary RAW formats, and work with Adobe to develop DNG into a format which can acommodate their individual image enhancing data, nothing will change.

John

[Mike Arst]

Have you considered converting the files to DNG and at the same time storing the original .CR2 image data inside the DNG file? That of course increases your drive-space requirements, per DNG file, by quite a lot. Later the converter could be used to extract the original RAW file from the DNG. You might have the best of both worlds that way ... assuming the Adobe program really is doing nothing other than storing the original RAW data -- and keeping it intact. (All that said, I don't know if I could work up the nerve to place all of my RAW files inside DNGs and then delete them. The idea of permanently removing them makes me feel a bit queasy.)

[feppe]

Good question, Marc.

Adobe lists some benefits of DNG for photographers, but none of them have materialized.

Archivability has always been a "selling" point of DNGs, but DNG is not an open, free format although its proponents try to make it look like it is. CR2 is more locked down, but as you point out DNG also relies on Adobe's blessing and continued support. I have no reason to suggest Adobe would stop supporting DNG, and they might even open source it in the future to help make it de facto standard. In the meanwhile I'd rather not waste the time and/or space listed below.

CR2 is just as widespread as DNG - I would be very surprised if there was a RAW converter which didn't read CR2, but wouldn't be surprised if there was one which didn't support DNG. If the sky falls and future versions of Lightzone stop supporting CR2, for example, one can convert her entire stock of RAWs to DNG at that time.

The only difference of using DNGs for all intents and purposes is that you either lose proprietary information (might or might not be an issue for you), or you have to essentially double your RAW storage space. Neither is an attractive option and I certainly can't see a compelling, or even a minor reason to go DNG.

Even if one insists on using DNG, I would argue it's actually better to wait, as Adobe might be able to reverse engineeer some of CR2's proprietary features in the future - or Canon might open up their format. Therefore throwing away the original RAWs would be destructive. Another option is of course to embed the original RAWs, but doubling terabytes of storage is not an attractive proposition for most.

[Per Ofverbeck]

Have you considered converting the files to DNG and at the same time storing the original .CR2 image data inside the DNG file? That of course increases your drive-space requirements, per DNG file, by quite a lot. Later the converter could be used to extract the original RAW file from the DNG. You might have the best of both worlds that way ... assuming the Adobe program really is doing nothing other than storing the original RAW data -- and keeping it intact. (All that said, I don't know if I could work up the nerve to place all of my RAW files inside DNGs and then delete them. The idea of permanently removing them makes me feel a big queasy.)

Me too...  Since I´ve switched to a M9, all recent files are DNG anyhow, but those old ones (even a few from a Canon D60 and an Olympus E10...) are a concern.

How much less space is a DNG *with* embedded original raw vs. the sum of a DNG *without* embedding plus the original as a separate file?  After all, storage space is quite cheap these days.

[Mike Arst]

I would be very surprised if there was a RAW converter which didn't read CR2, but wouldn't be surprised if there was one which didn't support DNG.

I've tried several that didn't support DNG at all or that did a poor job at best of supporting it -- most recently DxO Optics Pro, which is superb in many respects. But it cannot read DNG format (that I know of).

The threat seems to be that Canon, Nikon, or others would stop supporting their own early RAW formats. But nobody has ever suggested why they would do such a thing. The code needed to convert a given "flavor" of RAW, having been written and released once (in the RAW converter), doesn't have to be rewritten or updated, does it? Surely the portion of the code that decodes, say, what's in the Maker Note doesn't take up a lot of space within the converter. What would be the motive for discontinuing support? I got a bit freaked out when there was an "authoritative" rumor that Canon would be dropping support for the D3's .CRW format. But the threat never materialized...

The DNG concept is appealing. But the lack of DNG-related tools -- at least, I haven't found any -- that improve upon Adobe's own converter has been a bit disappointing. After giving up finding a command-line-only tool for doing batch DNG conversions, I finally had to roll my own script for this purpose. (Adobe's grasp of command-line interfaces seems very crude at best.)

[Mike Arst]

How much less space is a DNG *with* embedded original raw vs. the sum of a DNG *without* embedding plus the original as a separate file?  After all, storage space is quite cheap these days.

I've never made the comparison, but my (rough) impression is that for a given file the two approaches use about the same amount of drive space.

[alain]

I have periodically debated converting my large archive of Canon CR2 files to DNG format.  To me, the debate boils down the the question of what format will be supported longer Canon CR2 or Adobe DNG.  I can't really come up with any reason that either company is superior in its long-term outlook, so by default, I have stuck with CR2 files.  

Another reason I was reluctant to make DNG conversion part of my workflow was the loss of Canon proprietary CR2 properties that can only be accessed through Canon DPP.  Though my workflow doesn't use DPP, I always figured that if I had a compelling reason to go back to it, it would be there if I used CR2 but not with DNG.  

Has the landscape in this area changed since I made this conclusion.  Today, is there any better reason to convert to DNG than two years ago?

If you have any doubts:

 test the free dng converter.  If it works keep it stored with you're files without converting them.  Only if it's ever needed do the conversion.

DNG is a adobe specific format and most other raw converters will not read it.  (Unless the camera specific dng's in most cases)

An easier solution is to keep you're current raw converter(s) available with a good backup.  

[john beardsworth]

Limiting the discussion to "I can sidestep the issue until there's a problem" is a bit like abolishing Glass-Steagall and placing your trust in bankers' sense of responsibility to society. After all, who would have thought Lehmans etc would disappear, and who would have thought Kodak would become so irrelevant - or withdraw support of their old formats?

As well as the long term archival question, also consider the portability of transferring metadata between the different applications you'll use over time (metadata embedded in a DNG is much more portable than in a sidecar file), the safety of letting any app write metadata directly into an image (safe with DNG, a guess with raw), and the value of the adjusted previews and thumbnails which can again be leveraged in other apps which aren't raw converters, as well as some that are.

A DNG workflow has such advantages. And it doesn't mean trashing your raws (putting them in the DNG itself is unwieldy and just risks locking the raw inside the format the DNG doubters fear). It's best to try a DNG-based workflow, and archiving your raw files separately.

John

[alain]

Limiting the discussion to "I can sidestep the issue until there's a problem" is a bit like abolishing Glass-Steagall and placing your trust in bankers' sense of responsibility to society. After all, who would have thought Lehmans etc would disappear, and who would have thought Kodak would become so irrelevant - or withdraw support of their old formats?

As well as the long term archival question, also consider the portability of transferring metadata between the different applications you'll use over time (metadata embedded in a DNG is much more portable than in a sidecar file), the safety of letting any app write metadata directly into an image (safe with DNG, a guess with raw), and the value of the adjusted previews and thumbnails which can again be leveraged in other apps which aren't raw converters, as well as some that are.

A DNG workflow has such advantages. And it doesn't mean trashing your raws (putting them in the DNG itself is unwieldy and just risks locking the raw inside the format the DNG doubters fear). It's best to try a DNG-based workflow, and archiving your raw files separately.

John

As far as I know only adobe software is able to use dng (except the Leica dng's).  So the dng metadata is not portable.  xmp is a standard format that's even human readable.

[Geoff Wittig]

Limiting the discussion to "I can sidestep the issue until there's a problem" is a bit like abolishing Glass-Steagall and placing your trust in bankers' sense of responsibility to society. After all, who would have thought Lehmans etc would disappear, and who would have thought Kodak would become so irrelevant - or withdraw support of their old formats?

As well as the long term archival question, also consider the portability of transferring metadata between the different applications you'll use over time (metadata embedded in a DNG is much more portable than in a sidecar file), the safety of letting any app write metadata directly into an image (safe with DNG, a guess with raw), and the value of the adjusted previews and thumbnails which can again be leveraged in other apps which aren't raw converters, as well as some that are.

A DNG workflow has such advantages. And it doesn't mean trashing your raws (putting them in the DNG itself is unwieldy and just risks locking the raw inside the format the DNG doubters fear). It's best to try a DNG-based workflow, and archiving your raw files separately.

John

Nah, Glass-Steagall is a poor analogy. The banksters had a powerful incentive (one might say, billions of reasons) to be dishonest about their intentions when pushing for deregulation. But Canon & Nikon have no incentive at all to suck customers in and then withdraw support for their RAW file formats in the future. It would badly damage their brands and lead customers to flee to competitors. My sense is that powerhouse companies in digital imaging like CaNikon will continue to use proprietary RAW formats to eeke out a little bit more image quality, while smaller players like Leica or Pentax will see a marketing advantage in using the DNG standard. It's by no means clear that DNG will outlast the proprietary formats, no matter what Adobe says, so it still seems to be a crapshoot when it comes to archiving. So far I'm saving native RAW files together with the edited image files, but it would be a simple matter to batch-convert the RAW files to DNG if and when it made more sense. And if Adobe stopped releasing its DNG converter software, what would that say about their committment to DNG?

[john beardsworth]

Only if you misrepresent the analogy. For 2 decades we were thinking we could rely on them, but big names do disappear for whatever motive. Look at BP or the US (or British) motor industry. Again though, that's too narrow a focus.

[john beardsworth]

As far as I know only adobe software is able to use dng (except the Leica dng's).  So the dng metadata is not portable.  xmp is a standard format that's even human readable.

Aperture can, CaptureOne seems to.... as well as dozens of cataloguing and browsing programs which can use the adjusted previews. And xmp in sidecar files isn't as portable as the same xmp in embedded metadata (xmp is not the same as a human readable text file) - far more programs read embedded metadata than read sidecars.

[john beardsworth]

My test: (substitute your preferred RAW extension)

1. Are there applications *I want to use* that:
    A. read .NEF? (yes)
    B. read .DNG? (yes)
    C. don't read .DNG? (yes)
    D. don't read .NEF? (no)

2. Can I:
    A. convert from .NEF to .DNG? (yes)
    B. convert from .DNG to .NEF (no - must keep the .NEF either inside the .DNG or separately)

I don't need to go any farther than that.

Too simplistic.

[alain]

My test: (substitute your preferred RAW extension)

1. Are there applications *I want to use* that:
    A. read .NEF? (yes)
    B. read .DNG? (yes)
    C. don't read .DNG? (yes)
    D. don't read .NEF? (no)

2. Can I:
    A. convert from .NEF to .DNG? (yes)
    B. convert from .DNG to .NEF (no - must keep the .NEF either inside the .DNG or separately)

I don't need to go any farther than that.

Indeed sometimes live is very easy

[AFairley]

My sense is that powerhouse companies in digital imaging like CaNikon will continue to use proprietary RAW formats to eeke out a little bit more image quality.

But for purposes of this discussion, I have never seen proof that a CR2 or NEF or ORF converted to DNG has any less IQ than the original RAW.  And as to the raw formats themselves, aren't they just containers anyway?

[fike]

Okay...so the answer here is that nothing has changed.  There is no more compelling reason today than two years ago to switch to DNG.  To me that means DNG is probably doomed to be a niche format for smaller manufacturers and a few enthusiastic supporters.  

I see no compelling reason to convert my CR2 files to DNG.

[BobFisher]

The threat seems to be that Canon, Nikon, or others would stop supporting their own early RAW formats. But nobody has ever suggested why they would do such a thing.

I think a lot of Canon FD shooters wondered the same thing when they were abandoned at the time Canon went with the EF mount. ;-)

[john beardsworth]

Okay...so the answer here is that nothing has changed.  There is no more compelling reason today than two years ago to switch to DNG.  To me that means DNG is probably doomed to be a niche format for smaller manufacturers and a few enthusiastic supporters.  

I see no compelling reason to convert my CR2 files to DNG.

It was never "compelling", more "on balance", Marc, so I'd quibble at "enthusiasts" too. It's probably fair to say that not much has changed in two years - those who could see the advantages two years ago can still do so, and those who couldn't, still can't. If your focus was only on raw conversion, you'll reach the same conclusion as before, but if you had a wider and/or DAM perspective your views remain in favour of DNG. Over those two years, more software programs have been able to exploit DNGs safely. Often it's along the lines of a marginal or niche change (eg what would you prefer something like HoudahGeo to do - write directly inside a documented format DNG or inside a proprietary raw file) and so it's a matter of how many niche changes tip the balance.

John

[madmanchan]

Hi fike, a couple of comments ...

DNG is not an either-or proposition. Sometimes folks choose to use DNG as an interchange format (e.g., when sending files to a client or colleague, with adjustments embedded), for example, while keeping original non-DNG raw files archived separately. This may require additional space, but usually not a burden from an expenses point of view (i.e., the cost of hard drives is usually not the limiting factor).

Leica does use DNG as its in-camera raw file format, but is not the only one. Others include Casio, Pentax, and Ricoh. Others are close to DNG but have not adopted all of the relevant tags yet.