DNG/RAW file size differences...

[tingyat]

Hi,

I'm currently "experimenting" with Adobe's DNG converter 6.7 RC1 beta and this on WIN 7 64-bit platform. In there, the settings are set to their default values - as in: Compatibility: Camera RAW 6.6 later, JPEG Preview: Medium Size, Don't embed fast load data, Don't use lossy compression, Preserve Pixel Count, Don't embed Original.

As an observation, there's a noticeable size reduction from my RAW CR2 files to DNG files. Typically I seem to be "losing" around 5 Mb per file - as in a 25.25 Mb CR2 file and a 20.64 Mb DNG file of the same image. In another example, 28.45 Mb CR2 file and a 23.11 Mb DNG file for the same image.

Two questions, why this size reduction and, if any, what image data is possibly being "lost" as a result?

[digitaldog]

Two questions, why this size reduction and, if any, what image data is possibly being "lost" as a result?

Because the DNG conversion is better at reducing the size using a lossless compression (nothing new in terms of DNG). Nothing is lost with the exception of maybe some proprietary metadata and your possible inability to use the DNG in other converters that don’t understand DNG. IOW, it is still raw data.

And you probably want the Fast Load on, it will really help speed things up in the Develop module or in ACR when working with the data.

[tingyat]

... possible inability to use the DNG in other converters that don’t understand DNG...

Hi Andrew,

Appreciate the follow up and and your comments - so, thank you for that.

I'm now beginning to appreciate some of the "subtlety" of the DNG paradigm - this for want of a better term. The last time I looked at DNG was some years back and then, I was using a Canon EOS 1D MkII - this with appreciably smaller RAW files. With a 5D MkII, these files are a lot bigger.

That said and referring to the quote above, I presume you mean applications such as Capture One not being able "read" DNG ?

I'll give the "Fast Load" feature a try. Sort of getting a little confused between this feature and the "lossy compression" feature and some of the issues that seem to be have been created and mentioned elsewhere - hence the "experimenting" part.

Thanks again and take care.

[Redcrown]

Three reasons for size difference between CR2 and DNG.

1. The DNG compression is different. Still lossless, but different and slightly more "efficient". So a small difference is due to DNG compression.

2. Embedded jpeg. Depending on your camera model, the size of the embedded jpeg in the CR2 might be full, medium, or small. And the compression of that jpeg is different than the compression by DNG. My Canon 5D2 embeds a full size, low compression jpeg in the CR2.

I tell the DNG converter to embed no jpeg. In spite of that, the DNG converter still embeds a very small jpeg, but that can account for about 10mb of file size difference. If your CR2 has a full size jpeg and you told DNG to embed a medium size jpeg, I'd bet that accounts for most of the 5mb difference your are seeing.

3. The DNG converter passes the EXIF pretty much as is. However, you can also do the DNG conversion with Photoshop. If you do that, the images pass through ACR and in passing the the default Camera Raw settings get added to EXIF. That only adds about 5k.

That last difference was a mystery I discovered in a recent debate in the Adobe forums. I and others discovered that doing the DNG conversion via Photoshop/ACR was considerably faster that the DNG converter. The reason, we learned, was that Photoshop/ACR operates in 64bit mode while the DNG converter is still a 32 bit app. The 5K size difference was a mystery within that mystery, but you can see it by looking at the EXIF in Bridge for the two different methods.

[tingyat]

Hi Redcrown,

This is really comprehensive - just the kind of information I've been looking for.

Items 1 and 2 I can handle.

In item 3, you mention that the "default Camera RAW settings get added to the EXIF". What do you specifically mean by the "default Camera RAW settings"?

I only ask because, in the recent past, I've been adding metadata to my RAW images via the metadata tool in Bridge. While I see this metadata in the Adobe environment - Bridge, C5, LR3.x - invariably I don't see it in other applications. As I understand it, this metadata information may reside in the Bridge cache. In revamping my image archive, trying to "find" this metadata information has now become something of a hit and miss affair.

On the last item where you mention PS/ACR is 64 bit and DNG converter is 32 bit. This might be significant in terms of accessing system memory. Learnt from this forum about hitting Control O to fire this up. As an extension to all this, does 8 bit and 16 bit come into this at all? When I recently opened up a DNG image in CS5, this was an 8 bit image file. I was sort of expecting it to be a 16 bit file. Not sure if I have something wrong here? 

[digitaldog]

The embedded JPEG in the DNG (a size you can control) shows the current rendering so this is quite different from a JPEG in a proprietary raw file. The idea is, as you apply edits in say LR or ACR, you have the option to update that current rendering in the DNG via the JPEG (which can be extracted). Kind of a belt and suspender approach but should the need to get to the rendering be necessary, you have a pretty high quality JPEG as a back up. Other applications could access (preview, print) that JPEG, there isn’t anything in the spec that prevents this access.

As for the question about Capture 1 and DNG, you’ll have to ask them. There isn’t anything too costly in any company supporting the use of DNG (like TIFF and JPEG, there is no fee to use).

[Redcrown]

Tingyat,

Convert a CR2 to DNG using the standalone DNG converter. Then look at the "metadata" of that DNG in Bridge. Remember that what you see in the Bridge Metadata panel is controlled (filtered) by your Bridge Preferences. So make sure Bridge metadata preferences are set to display all "Camera RAW" fields.

Notice that the "Camera RAW" fields do not display. That's because they are empty. ACR has never touched this file.

Now open that same CR2 file in Photoshop/ACR. Don't make any changes, just do a "Save Image" from within ACR and save as a DNG. Look at the metadata of that version in Bridge and see that the "Camera Raw" fields are all present and their values are whatever your ACR defaults are.

Those 2 DNG files will be slightly different in size, due to the extra metadata.

"Metadata" is a confusing beast. It can be stored in many places, depending on the program used. It might be stored in the EXIF header of a file, it might be in a database somewhere. In the EXIF headers, different programs interpret the fields differently. Canon or Adobe may stick data in some EXIF field that other programs simply ignore and don't display.

And if you opened a DNG image in CS5 and got an 8bit file, that's probably because you have your ACR set to output 8bit. You can change that within ACR.

[Robert55]

Whether a DNG [or a RAW] is opened with a bit depth of 8 or 16 is determined by your settings in ACR. Below the image you should see something that looks like a hyperlink. When you click on that a menu will open, by which you can set a number of workflow option.
Note that these settings are sticky: when change them, all images will be opened using these settings [untill you change them again of course.