Pages: 1 [2]   Go Down

Author Topic: Understanding reduction in file size from Camera Raw to DNG  (Read 18740 times)

sandymc

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 350
Re: Understanding reduction in file size from Camera Raw to DNG
« Reply #20 on: April 21, 2014, 03:06:12 pm »

I fully agree! But you're not helping me in understanding when and where that happens. I've got about 27K DNG's. AFAIK, none are linear. I've never asked for Linear. I think you're saying there is a case where *some* camear raws are converted to Linear without the end user being told? If so what is the trigger?

Yes. Like  I said above, if the version of DNG doesn't support something in the raw (which was the case for lens corrections on I think Panasonic cameras pre the versions of DNG with op codes), or if ACR/LR would throw up (which was the case for XTrans - before Adobe put good support in). Under those circumstances, DNG converter silently went to linear raw.

See, e.g., this thread on the Adobe forums: http://forums.adobe.com/message/5144148#5144148
Logged

noavscinc

  • Newbie
  • *
  • Offline Offline
  • Posts: 8
Re: Understanding reduction in file size from Camera Raw to DNG
« Reply #21 on: April 21, 2014, 03:10:57 pm »

sandymc and dgitaldog:

I'm a newbie to all of this but based on the link I posted earlier it seems like the data loss and potential linearization of a file pertains to the ways in which a file is made backwards compatible by ACR. Even if you set up your preferences to preform a lossless conversion the compression process to make a file backwards compatible is lossy:

"The DNG format makes backwards compatibility possible in several different ways. Sometimes this is done without any loss of quality, and sometimes it requires “baking in” some of the changes to the raw file, which does remove some flexibility to re-edit your files. Let’s look at the issues.....Sometimes the backwards compatibility must be done by linearizing the DNG because some new features are not supported by older software. For instance, the Panasonic LX5 camera requires some image de-warping in the raw conversion in order to remove lens distortion. Older Camera Raw versions did not support this function. In order to open these new files with old software, the DNG must be created with the de-warping baked in to the DNG, which is done by selecting a Compatibility prior to the latest version, as shown in Figure 5. This is a “lossy” operation, since you’ll be making changes that can’t be undone." http://dpbestflow.org/DNG#backwards-compatibility


Logged

digitaldog

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 20630
  • Andrew Rodney
    • http://www.digitaldog.net/
Re: Understanding reduction in file size from Camera Raw to DNG
« Reply #22 on: April 21, 2014, 03:11:54 pm »

Yes. Like  I said above, if the version of DNG doesn't support something in the raw (which was the case for lens corrections on I think Panasonic cameras pre the versions of DNG with op codes), or if ACR/LR would throw up (which was the case for XTrans - before Adobe put good support in). Under those circumstances, DNG converter silently went to linear raw.
Did you write there:
Quote
3. "Normal" aka CFA DNG. The DNG contains CFA (aka Bayer mosaiced) data, and whatever reads the DNG does the demosaicing. You're not locked in.
I believe I'm also using option 3. Further I've got no Fuji cameras. So outside of that model, what data loss could I find?
Adobe SHOULD pop a warning dialog, no question.
Logged
http://www.digitaldog.net/
Author "Color Management for Photographers".

digitaldog

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 20630
  • Andrew Rodney
    • http://www.digitaldog.net/
Re: Understanding reduction in file size from Camera Raw to DNG
« Reply #23 on: April 21, 2014, 03:16:01 pm »

In order to open these new files with old software, the DNG must be created with the de-warping baked in to the DNG, which is done by selecting a Compatibility prior to the latest version, as shown in Figure 5. This is a “lossy” operation, since you’ll be making changes that can’t be undone." http://dpbestflow.org/DNG#backwards-compatibility
So it appears there is a rare condition in which without warning, conversion to DNG can be Linear. Again Adobe should pop a working. But unless you care about an old version of software support, the latest version for DNG (my default) isn't going to cause any issues? 
Logged
http://www.digitaldog.net/
Author "Color Management for Photographers".

noavscinc

  • Newbie
  • *
  • Offline Offline
  • Posts: 8
Re: Understanding reduction in file size from Camera Raw to DNG
« Reply #24 on: April 21, 2014, 03:24:37 pm »

I'm not sure how rare it is. It sounds like any time the ACR encounters a feature that is not supported by earlier versions of Camera Raw, it will linearize the file...Is this what you are suggesting sandymc?
Logged

sandymc

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 350
Re: Understanding reduction in file size from Camera Raw to DNG
« Reply #25 on: April 21, 2014, 03:32:50 pm »

The point is, there are circumstances under which there may be data loss in ways that would not be obvious to the casual user. In many, or even most, situations there will not be a problem, but I'm very wary of blanket assurances that DNG conversion is always lossless.
Logged

digitaldog

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 20630
  • Andrew Rodney
    • http://www.digitaldog.net/
Re: Understanding reduction in file size from Camera Raw to DNG
« Reply #26 on: April 21, 2014, 04:00:02 pm »

The point is, there are circumstances under which there may be data loss in ways that would not be obvious to the casual user. In many, or even most, situations there will not be a problem, but I'm very wary of blanket assurances that DNG conversion is always lossless.
Yes, there are ways to hose any data during conversion. You had me worried but now I see this is a very isolated case with one or two cameras with a setting that I wouldn't recommend using for DNG conversion. Unless I'm missing something more. So in the three cases you cited, it's really one. If the end user really knew what we now know, they wouldn't have an issue, they would use a more appropriate conversion setting. With these older cameras set for older software. Adobe should pop an dialog for these rare cases. No excuse for that. I wasn't aware of this so this was useful.
Logged
http://www.digitaldog.net/
Author "Color Management for Photographers".

Schewe

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 6229
    • http:www.schewephoto.com
Re: Understanding reduction in file size from Camera Raw to DNG
« Reply #27 on: April 21, 2014, 06:27:33 pm »

  • What Adobe doesn't know, Adobe doesn't transfer to DNG; what you're getting is Adobe's understanding of the file format, which may or may not be complete

While that is true for "some" raw file formats, I'm pretty sure (post opcodes) that the DNG conversion will find a place for private maker notes and other data into the DNG file even if DNG doesn't understand that data. You'll have to ask Eric to confirm this...but what DNG (or ACR/LR) doesn't know about can't be used in DNGs anyway. So, if ACR/LR can't use the metadata, not sure what good it is.

Now, there are some issues with Bridge and Lightroom failing to show or disclose certain metadata such as focus points in a capture, but I don't think converting to DNG strips them, they just are not used in Bridge and LR.
Logged

Bob Rockefeller

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 548
  • macOS, iOS, OM Systems, Epson P800
    • Bob Rockefeller
Re: Understanding reduction in file size from Camera Raw to DNG
« Reply #28 on: August 09, 2014, 11:25:01 am »

Now, there are some issues with Bridge and Lightroom failing to show or disclose certain metadata such as focus points in a capture, but I don't think converting to DNG strips them, they just are not used in Bridge and LR.

I believe that a NEF converted to DNG changes or strips the focus point metadata in such a way that Aperture can no longer use it. At least is seems to in the testing I've done.

This had made me leery of the DNG format, not knowing what else might be striped out that I could want days or years later. If Lightroom were to one day display Nikon focus points, would it still be able to in a NEF converted to DNG?
Logged
Bob Rockefeller
Midway, GA   www.bobrockefeller.com

madmanchan

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 2115
    • Web
Re: Understanding reduction in file size from Camera Raw to DNG
« Reply #29 on: August 09, 2014, 02:09:43 pm »

Yes, the focus point metadata (and other Nikon-related metadata) is preserved in the DNG.  At the technical level, vendor private data such as MakerNotes are basically TIFF-like tags that software can simply copy over from one container/file to another, even if the software doesn't understand the meaning of those tags (*).  Just because other software does not show the metadata after conversion to DNG, it does not mean that the metadata does not exist.

Eric

(*)  It would be like going to a warehouse filled with boxes and being asked to save or move those boxes.  You can move those boxes from one warehouse to another even if you don't know or understand what's in those boxes.
Logged
Eric Chan

Bob Rockefeller

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 548
  • macOS, iOS, OM Systems, Epson P800
    • Bob Rockefeller
Re: Understanding reduction in file size from Camera Raw to DNG
« Reply #30 on: August 09, 2014, 02:14:05 pm »

Yes, the focus point metadata (and other Nikon-related metadata) is preserved in the DNG.  At the technical level, vendor private data such as MakerNotes are basically TIFF-like tags that software can simply copy over from one container/file to another, even if the software doesn't understand the meaning of those tags (*).  Just because other software does not show the metadata after conversion to DNG, it does not mean that the metadata does not exist.

Eric, thanks for the authoritative answer; it's good to hear from the source.

I guess what is happening is that the DNG Converter moves the Nikon focus point metadata and then Aperture doesn't know where to look to find it.
Logged
Bob Rockefeller
Midway, GA   www.bobrockefeller.com
Pages: 1 [2]   Go Up