Pages: [1]   Go Down

Author Topic: DNG - What Value?  (Read 11152 times)

Farkled

  • Jr. Member
  • **
  • Offline Offline
  • Posts: 97
DNG - What Value?
« on: December 17, 2007, 02:42:59 am »

Is there significant value to standardizing on DNG?

From what I've read thus far, I am confused.  

It seems that the Canon CR2 files are a variation of TIF as is DNG and nothing I've read asserts an image quality or processing advantage to either format.

DNG files seem to be somewhat smaller - at least before processing  

CR2s need an XMP file, DNGs hold all - a much safer solution.

One can return to "as shot" with either format.

Canon seems likely to be around as long as Adobe so I don't see a format longevity advantage.  Both formats will start to die as soon as 32 bit formats show up.  I suspect that, in 30 years, it will be up to 3rd party software to deal with either format.  Maybe not.

I have only a single Canon body (40D) and don't see that changing in the next couple or so years.

I believe the above is true for Nikon NEF files too.

If someone would straighten me out, I would be very grateful.
Logged

Jonathan Wienke

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 5829
    • http://visual-vacations.com/
DNG - What Value?
« Reply #1 on: December 17, 2007, 06:56:06 am »

Every new camera model that does not output DNG requires your RAW converter to be rewritten before it can process the files. So if you have a favorite RAW converter and buy a new camera and the developer doesn't update the converter, you're screwed. OTOH, if DNG becomes widely accepted, RAW converter developers can focus in improving image quality instead of learning hundreds of different undocumented file formats. Losslessly compressed DNG files are also smaller than the original RAW files; Canon RAWs lose 5-10% of size with no quality loss and Olympus RAWs lose about 50% of their bulk. So for archiving, DNG has some distinct advantages. I'd be more inclined to trust DNG's long-term viability than any particular camera manufacturer's proprietary format; Canon has changed their RAW format several times, and so has Nikon and Sony (who actually tried encrypting their RAWs so only Sony software could be used on some cameras until somebody figured out the encryption), and several other companies have started supporting DNG as a standard.

32-bit RAW format is pretty unlikely, as cameras that output 16-bit data are only using maybe 12 of those bits at most for real data, and that isn't likely to change any time soon unless some radical breakthroughs are made in both lens and sensor technology so that one can capture more than 12 stops of DR in a single frame.
Logged

digitaldog

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 20646
  • Andrew Rodney
    • http://www.digitaldog.net/
DNG - What Value?
« Reply #2 on: December 17, 2007, 09:43:05 am »

Logged
http://www.digitaldog.net/
Author "Color Management for Photographers".

sniper

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 670
DNG - What Value?
« Reply #3 on: December 17, 2007, 10:35:08 am »

I'm afraid I'm a little more wary of .DNG, Adobes all to quick to drop support for it's earlier versions, who's to say they do the same for .DNG?  Theres already talk of a new version of .DNG (which may or may not be compatable) and theres precious little other support for the format, it's not exactly taken the world by storm has it.
Now theres also a "new" .jpeg format care of MS (heaven help us) to compete.
Personally I'm covering my back and saving my RAW as well as .DNG for important stuff, just in case...     Wayne
Logged

Jonathan Wienke

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 5829
    • http://visual-vacations.com/
DNG - What Value?
« Reply #4 on: December 17, 2007, 12:10:57 pm »

Quote
Theres already talk of a new version of .DNG (which may or may not be compatable)

Care to share your sources on that?
Logged

Panopeeper

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 1805
DNG - What Value?
« Reply #5 on: December 17, 2007, 12:22:48 pm »

This is a huge and complex subject; it has been discussed extensively and keeps being discussed. I mention a few aspects:

1. The idea of standardizing the format was good. Adobe defined an "initial standard". Unfortunately, instead of widely discussing it, Adobe threw it on the market. There are some problems with details, and they could be mended, but in the meantime people converted millions of images into this form.

2. Adobe now tries to pass the buck to ISO. There are talks about accepting it as a "true standard".

3. Adobe believes (? I don't believe, that they believe it), that the DNG specification is enough to cover everything. However, the camera development did not stop and won't stop due to a fixed standard, i.e. the standard will have to be updated time and again, which takes time.

4. Camera manufacturers are not prepared to disclose their new developments (not even to partners like Adobe under a nondisclosure-agreements). Consequently, a camera may appear on the market with some new feature, which is not covered by the current standard.

Examples:

-  the DNG specification still does not cover the "dust delete data" of newer Canon cameras.,

- the Highlight Tone Priority feature is unknown in DNG (the automatic adjustment of the exposure is covered by the specification, but the fact that the indicated ISO is not the actual ISO is not covered).

Consequently many raw processors need the original camera specific information, the structure of which is unpublished. Therefor DNG more of a common framework than a complete standard.

Still, the idea is good and I hope it will gain momentum, if ISO takes over the standard (and mends it).

However, the advantages are irrelevant for those, whose camera does not produce DNG. Plus, an unfortunate circumstance, even some of those DNG files created in camera are messy. As a general rule, camera manufacturers ought to refrain from touching software issues; they are suckers, particularly the Japanese.

Now some practical considerations:

1. Most raw processors support now DNG, but it does not help anything if Adobe updates the DNG converter only with new releases of ACR. Even if a sensor is identical to a former one (like Canon 30D after the 20D), the converter program code needs to be upgraded. It is plainly bad design. (ACR too needs to be updated, but that's the pain of ACR users only).

2. Some people are keeping (archiving) *only* DNG. There are some supporters of DNG, who are blind and deaf (but not mute). This is not the question of the standard, but of the actual converter program. Adobe recalled ACR 4.3 a few weeks ago, together with the associated DNG converter, because of errors in the conversion.

It is possible to keep the original raw inside the DNG. IMO this is a ridiculous way to preserve the data.

3. The fact, that the DNG compressed images are smaller than some others is conceptually irrelevant. The matter of fact is, that Adobe adopted the very same compression method, which is used in the Canon CR2 files (namely the original JPEG lossless encoding). The file size advantage comes from a rather lazy approach on Canon's side (did I mention, that they suck in software?).

4. Recording the adjustments made on the raw data is an important consideration.

Canon's DPP appends all adjustments to the raw file, or they can be saved in separate files, though the latter costs extra steps.

ACR can save the raw adjustments in sidecar files, or in the database; however, in case of DNG, the adjustments will be inserted in the file.

There is no generic rule, which way is better, for it depends on one's workflow. However, ACR's DNG handling makes it practically impossible to keep several versions of adjustments. Note, that this is not the disadvantage of DNG but of ACR.

5. Adobe's DNG converter is a stand-alone program, hopefully without a time bomb. I keep all previous versions in my archive (even though I don't archive DNG files :-). However, the program source is not open entirely.

On the other hand, should Adobe stop supporting DNG, someone would certainly jump into the niche, so there is no reason to fear for the support of old stuff.
Logged
Gabor

Panopeeper

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 1805
DNG - What Value?
« Reply #6 on: December 17, 2007, 12:29:06 pm »

Quote
32-bit RAW format is pretty unlikely, as cameras that output 16-bit data are only using maybe 12 of those bits at most for real data, and that isn't likely to change any time soon unless some radical breakthroughs are made in both lens and sensor technology so that one can capture more than 12 stops of DR in a single frame.

You still don't want to accept, that the bit depth and DR are two separate issues. Here is the news: the Phase One P45+ records FULL 16 BIT of raw data. Yes, 65536 levels.
Logged
Gabor

Jonathan Wienke

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 5829
    • http://visual-vacations.com/
DNG - What Value?
« Reply #7 on: December 17, 2007, 12:45:38 pm »

Quote
Here is the news: the Phase One P45+ records FULL 16 BIT of raw data. Yes, 65536 levels.

I know that. That doesn't mean you get any more stops of usable DR though. The sensor readout amplifiers and the ADC all have to be additionally precise for those additional bits to constitute actual signal, though, and that hasn't happened yet. The noise floor is far higher than one bit in the RAW data, so several of the least significant bits are doing nothing more than more precisely defining sensor, readout, and ADC noise. Until it is customary for cameras to have a noise level < 4 out of 65535 levels, bit depth >16 will be a pointless waste of RAW file space.

Going from 12 to 14 bits hasn't improved Canon's DR significantly, at best doing so has reduced quantization noise in the lower levels, but did not increase DR by two stops. Mostly the extra recorded bits are noise.
« Last Edit: December 17, 2007, 01:20:59 pm by Jonathan Wienke »
Logged

Panopeeper

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 1805
DNG - What Value?
« Reply #8 on: December 17, 2007, 01:24:19 pm »

Quote
I know that. That doesn't mean you get any more stops of usable DR though

Exactly, that was my point.

Quote
Going from 12 to 14 bits hasn't improved Canon's DR significantly

Canon has never claimed that. Even more, it was explicitely stated, that the DR was not improved significantly. Their claim is, that more levels contribute to the image quality. I guess PhaseOne claims the same.

Quote
Mostly the extra recorded bits are noise

This claim is unproven, just like the opposing claim.

I find it questionable that so many more levels contribute to the perceivable image quality, but one can say they are useful in the de-mosaicing.
« Last Edit: December 17, 2007, 01:38:23 pm by Panopeeper »
Logged
Gabor

Jonathan Wienke

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 5829
    • http://visual-vacations.com/
DNG - What Value?
« Reply #9 on: December 17, 2007, 01:35:53 pm »

Quote
Exactly, that was my point.

I never disputed that, in fact I was quite clear to distinguish between the theoretical maximum DR of linear-gamma RAW files of a given bit depth and what DR was achievable in real-world conditions.
Logged

Farkled

  • Jr. Member
  • **
  • Offline Offline
  • Posts: 97
DNG - What Value?
« Reply #10 on: December 21, 2007, 04:43:27 am »

OK, what I've gathered from this is that for a low volume hobbyist, standardizing on DNG and ACR is neither particularly helpful nor does it hurt.

I think that if I were pushing volume through Lightroom then DNG/ACR would be a no brainer, but I'm not.  As an ex-software developer, I love the idea of an open standard.  Right now, however, it seems that the execution is sufficiently flawed that staying with CR2 files is not a bad choice.

I thank you all for your time and lucid discussions.
Logged

Nick Rains

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 705
    • http://www.nickrains.com
DNG - What Value?
« Reply #11 on: December 21, 2007, 06:02:05 am »

Quote
it seems that the execution is sufficiently flawed that staying with CR2 files is not a bad choice.
[a href=\"index.php?act=findpost&pid=162243\"][{POST_SNAPBACK}][/a]

What do you see as flaws? DNG files are a very convenient way to archive files and metadata with the only downside being that they have to be converted from RAW which takes time.
Logged
Nick Rains
Australian Photographer Leica

DarkPenguin

  • Guest
DNG - What Value?
« Reply #12 on: December 21, 2007, 09:37:43 am »

The only flaw I know of is that canon's software will not open a dng.

Lose your sidecar files often enough and dng starts looking really good.
Logged

Panopeeper

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 1805
DNG - What Value?
« Reply #13 on: December 21, 2007, 12:31:57 pm »

Quote
Lose your sidecar files often enough and dng starts looking really good.

This aspect may be an advantage or a disadvantage, depending on the workflow and post-processing requirements.

I did not have any problem with Raw Shooter's sidecar files, they were kept nicely in the same folder as the raw files, it would have been difficult to lose them. No effort was necessary to create or use them (unlike with DPP).

On the other hand,  the disadvantage is a show stopper: if one needs several sets of adjustments for the same image, one can not achieve that with DNG at all. I do not regard copying and archiving the very same raw data with different adjustments a practical way.
Logged
Gabor

Farkled

  • Jr. Member
  • **
  • Offline Offline
  • Posts: 97
DNG - What Value?
« Reply #14 on: December 22, 2007, 12:10:04 pm »

Quote
What do you see as flaws? DNG files are a very convenient way to archive files and metadata with the only downside being that they have to be converted from RAW which takes time.
[a href=\"index.php?act=findpost&pid=162254\"][{POST_SNAPBACK}][/a]

Please understand that I asked this question in the beginners forum because I am a newbie when it comes to RAW workflow and processing.  I'm taking the words presented here as more or less gospel.  I would like to lay the correct foundation before I have many thousands of images in RAW.

Panopeeper's reply is what caused me to use the word flaw - which may have been a poor choice.  DNG is a file format that is not all that different from CR2.  What is really different is the software that uses those files.

Does DNG offer an archival advantage?  I suppose only time will tell, but for the moment I don't see it.  Does DNG/ACR offer a workflow advantage?  I don't know.  What little I do know suggests that DPP offers some advantages for my situation.  I do not yet have Lightroom.  I will keep experimenting as time marches on.
Logged

digitaldog

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 20646
  • Andrew Rodney
    • http://www.digitaldog.net/
DNG - What Value?
« Reply #15 on: December 22, 2007, 02:33:41 pm »

Quote
Does DNG offer an archival advantage?


Yes of course.  That's one of the really big deals about it.

Anyone got an old Kodak DCS Raw?

Quote
Does DNG/ACR offer a workflow advantage? 

That you can send the Raw plus one (and hopefully soon multiple) possible renderings is super useful. The embedded JPEG when it reflects one or more renderings is super useful (draft printing, web page generation etc).
Logged
http://www.digitaldog.net/
Author "Color Management for Photographers".

Panopeeper

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 1805
DNG - What Value?
« Reply #16 on: December 22, 2007, 03:25:58 pm »

Quote
Yes of course.  That's one of the really big deals about it.
The only advantage (for those who find this disadvantage advantageous) is, that the adjustments are stored in the file.

Well, is this truely an advantage over native raw files? No, it is none, for it is not the characteristic of the file format but of the raw processor. For example DPP stores the adjustment metadata in the CR2 files if one chooses that way.

Quote
The embedded JPEG when it reflects one or more renderings is super useful (draft printing, web page generation etc).

This again has nothing to do with DNG but with the raw processor.

Plus, embedding a JPEG with many hundreds of kilobytes defeats the small advantage of the DNG compression over for example CR2 (which is anyway not the advantage of DNG but of the converter).

It's a disturbance in the evaluation if someone does not make distinction between the chacateristics of the file format and of the programs supporting the formats.
Logged
Gabor

sniper

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 670
DNG - What Value?
« Reply #17 on: December 24, 2007, 10:52:34 am »

Quote
Care to share your sources on that?
[a href=\"index.php?act=findpost&pid=161254\"][{POST_SNAPBACK}][/a]

I have come across quite a few mentions of a "new version" coming out soon, I don't think theres any real secret in it when it will appear is anybody guess (outside adobe anyway)  Wayne
Logged
Pages: [1]   Go Up