DNG puzzle

[digitaldog]

When I started this thread with a simple technical question, I could not have imagined how it would evolve. It's both horrifying and awe-inspiring.

What about post #2, specifically asking you about what settings you used and why, the DNG being smaller than the original proprietary raw in many cases is to be expected?
The rest is to be expected when DNG or something Adobe controls and provides comes up. Might as well ask, what wights more, a Canon or a Nikon, a Mac or a WIN PC; then all hell breaks loose. 

[Eric Myrvaagnes]

When I started this thread with a simple technical question, I could not have imagined how it would evolve. It's both horrifying and awe-inspiring.

From DNG to Playboy...   

[Rob C]

From DNG to Playboy...   

So true Eric: DNG etc. are all terrifying to consider instead of merely accepting; Playboy used to be inspiring, and then became awe-inspiring which wasn't quite as pleasant a vibe.

;-(

[Denis de Gannes]

When I started this thread with a simple technical question, I could not have imagined how it would evolve. It's both horrifying and awe-inspiring.

To me its a simple answer, if you you are committed to use only Adobe software then using the DNG option has some benefits, however if you are also utilising other software applications then you need to seriously consider if any pros will negate the cons you have to overcome.

[digitaldog]

To me its a simple answer, if you you are committed to use only Adobe software then using the DNG option has some benefits, however if you are also utilising other software applications then you need to seriously consider if any pros will negate the cons you have to overcome.

A simple answer to this exact question?
It's my understanding that the DNG format preserves all the info in the RAW, so how can I get a 50% file size reduction?
Hum..... Again, I believe post#2 had the simple answer prior to the political “answers” (to be kind).

[Damon Lynch]

A good RAW format should (among plenty of other criteria not listed below):

• Be predictable, obvious and logical in its layout.
• Be predictable and logical in what metadata it includes.
• Put all its metadata at the very front. Goodness grief don't scatter metadata all over the file. There should be no reason to need the entire file just to read image metadata including the thumbnail / image previews.

Why? If you know exactly what you (i.e. your software program) are looking for, and where to look for it, it's much quicker and much less prone to error. For example: if you have a RAW on a medium like a camera that is comparatively super-slow to read from, then being able to grab the image orientation, thumbnail, date/time and other key metadata quickly is extremely helpful. Just read a small chunk of the start of the file, and there is much you can do with it. If instead you have to fish around all throughout the file for this information, looking for field X in one file and field Y in another, then it's a total dog's breakfast.

The best mainstream format I know of with respect to these criteria is Canon's CR2 format (CRW was awful). The worst by far is Nikon's NEF format. (I'm ignoring the many marginal RAW formats, some of which are equally if not more awful). DNG is somewhat in the middle -- perhaps because it is created in post-processing as much as it is on the camera, there is more latitude for creativity to solve problems of the kinds that software developers must confront. Therefore it's more inconsistent than a format like CR2. That's not blaming anyone or casting aspersion on anyone's reputation -- I suspect it's an inevitable side-effect of the big-tent approach inherent to DNG.

Sadly, my preliminary impression of CR3 is that with respect to the criteria above, it regresses rather than improves on CR2. Bummer. It's highly annoying. Why did Canon go and screw it up?

For a discussion among software developers who are reverse engineering CR3 (DNG is mentioned), see https://github.com/Exiv2/exiv2/issues/236

As an aside, I'm writing this from the perspective of a software developer who confronts the issues mentioned above. I understand there are many other important attributes of a RAW format, for reasons already discussed. But these are the kinds of things I am confronted with, hence my two cents.

And as a final aside, good on Mr Knoll / Adobe for creating the DNG format and also the XMP standard. I wish more would stick to the XMP standard. I hate having to deal with variations on it because of smart-ass individual software developers who think their way is better.

[Doug Gray]

For what it's worth most RAW file use "tiff" containers. This makes DCRAW.C's job simpler. OEMs use a lot of proprietary tags and formats but the tiff spec defines how all this stuff is contained in the tiff container. Matlab can read the RAW tiff container and it will yield all sorts of fascinating stuff. Walking through dcraw.c as it processes a RAW file while looking at the contents in Matlab is an interesting exercise and will give an idea of what at least part of the RAW file contains and what it does but much remains unspecified by the OEM.