Pages: 1 [2] 3 4   Go Down

Author Topic: Converting to DNG - do you lose quality?  (Read 71293 times)

Panopeeper

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 1805
Converting to DNG - do you lose quality?
« Reply #20 on: July 13, 2008, 03:59:04 pm »

Quote
Which one do you use?
Adobe's stand-alone DNG converter is part of the ACR package. When downloading, you can chose this package and unpack the converter. "Stand alone" means in this context, that it is running w/o PS or LR. It created a DNG file, which can be processed by ACR in LR or PS.

This conversion is not influenced by any other setting, except the few options like "compression", "JPEG preview" (do *not* convert in linear).

After the conversion open both the CR2 and the DNG *together* in Photoshop, set all adjustments equal in ACR and see if there is any difference.
Logged
Gabor

Jonathan Wienke

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 5829
    • http://visual-vacations.com/
Converting to DNG - do you lose quality?
« Reply #21 on: July 13, 2008, 09:23:00 pm »

Quote
Not exactly a terminus technicus, at least not in photography.

It's a perfectly accurate description of the ignorance and misinformation contained in your original post.

DNG uses a lossless compression method that reduces file size without altering the RAW data contained in the file. Think of a ZIP file, but with a different compression algorithm optimized for image data. Therefore, the DNG file size is almost always smaller than the original, but that does not mean the RAW data inside has been altered in any way.
Logged

The View

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 1284
Converting to DNG - do you lose quality?
« Reply #22 on: July 13, 2008, 09:34:47 pm »

Quote
perfectly accurate
[a href=\"index.php?act=findpost&pid=207969\"][{POST_SNAPBACK}][/a]

You could have handled it with courtesy and knowledge, and simply stated facts, like others did.

But I see you're not good with words and can't handle your temper. So be it.
Logged
The View of deserts, forests, mountains. Not the TV show that I have never watched.

ErikKaffehr

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 11311
    • Echophoto
Converting to DNG - do you lose quality?
« Reply #23 on: July 14, 2008, 01:20:12 am »

Just a comment.

I think that we can see from the conversation that the issue you have is probably not with DNG but with LR. Sorry for that! It seems that LR works better for some folks than for others.

Now, there is an idea behind LR and that idea is that you keep the RAW file untouched and render it on the fly for different needs. LR gives you a very large gamut RGB which is internally linear.

If you just want to use LR as raw-converter it is probably not the best tool, it may be better to use some of the top class raw-converters like Phase One, Iridient Developer or what tool you want to use and than have an industrial strength asset management system for your "tiffs".

If you wan't to have a "raw" based workflow I think that LR is one of the better solutions, the other one is Aperture.

Regarding the issue with DNG I think that there may be a couple of issues. We have seen that the "pixels themselves" are not affected by conversion to DNG. As I told you there are a lot of additional information called "tags". Some of these are standard and some are vendor specific.

Now, one of the features of DNG is that it can contain XMP information. Lightroom actually stores it's development settings in the DNG file, it can also put it into a "sidecar" file. So this may have caused some of your issues, not the DNG file on its own, but the instructions in it. Sorry for being diplomatic, but I just try to guess what is happening.

The development settings go in three places in Lightroom

1) The internal database
2) I think also the DNG files
3) Optionally the XMP-sidecar files

Best regards
Erik

Quote
I see it can't be DNG's fault. Whatever it was, the difference was so striking...

And it can't be some kind of color space switch without my knowledge.

For the moment, I'll keep working with the original Canon RAW format until I know what happened.
[a href=\"index.php?act=findpost&pid=207734\"][{POST_SNAPBACK}][/a]
Logged
Erik Kaffehr
 

Panopeeper

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 1805
Converting to DNG - do you lose quality?
« Reply #24 on: July 14, 2008, 11:56:26 am »

Quote
LR gives you a very large gamut RGB which is internally linear
Do you mind explaining what this means? I don't know of any RGB, which is linear "internally" or "externally", nor do I see any CR option for generating a linear output.
Logged
Gabor

ErikKaffehr

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 11311
    • Echophoto
Converting to DNG - do you lose quality?
« Reply #25 on: July 14, 2008, 12:22:13 pm »

Hi!

As far as I understand LR is internally using a version of ProPhoto RGB but without a gamma curve. It's called "Melissa RGB".

Best regards
Erik

Quote
Do you mind explaining what this means? I don't know of any RGB, which is linear "internally" or "externally", nor do I see any CR option for generating a linear output.
[a href=\"index.php?act=findpost&pid=208095\"][{POST_SNAPBACK}][/a]
Logged
Erik Kaffehr
 

Panopeeper

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 1805
Converting to DNG - do you lose quality?
« Reply #26 on: July 14, 2008, 12:34:08 pm »

Quote
As far as I understand LR is internally using a version of ProPhoto RGB but without a gamma curve. It's called "Melissa RGB"
Ok, now I understand. ProPhoto is not linear, but LR converts an RGB image (which is never linear) into linear form and makes calculations on that; that is nothing special. The gamma-converted format is not suitable even for the simplest adjustments, like brightness or contrast. When writing the file, it gets gamma-converted again.
Logged
Gabor

Jonathan Wienke

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 5829
    • http://visual-vacations.com/
Converting to DNG - do you lose quality?
« Reply #27 on: July 17, 2008, 07:47:47 pm »

Quote
You could have handled it with courtesy and knowledge, and simply stated facts, like others did.

Others tried that approach, and you continued to insist on your point of view that converting to DNG caused quality loss and negatively impacted the conversion of images from the DNG compared to the original RAW. I restated the point in more forceful terms, and somewhere along the line you finally realized (I think) the error of your original position. If you don't know what you're talking about, asking questions is fine (it's how we all learn), but passing off your own ignorance as fact is likely to confuse and misinform others who read your posts. If you'd done even the most basic research on DNG and how it works you'd have realized that the issues you were encountering were the result of user error, and not a loss of quality due to converting the file to DNG.
Logged

Farkled

  • Jr. Member
  • **
  • Offline Offline
  • Posts: 97
Converting to DNG - do you lose quality?
« Reply #28 on: July 17, 2008, 09:05:44 pm »

Nick Rains is correct.  What you are seeing is the different interpretation (initial or otherwise) presented by each program.  DPP tends to preprocess the RAW preview into the same form the JPG would have taken in camera.  DNG is not so aggressive.

The primary reason to not convert to DNG is that you then lose DPP as a processor.  It can be very valuable.
Logged

Panopeeper

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 1805
Converting to DNG - do you lose quality?
« Reply #29 on: July 17, 2008, 09:35:55 pm »

Quote
DPP tends to preprocess the RAW preview into the same form the JPG would have taken in camera.  DNG is not so aggressive.
(LOL, another one making definitive statements instead of asking about the subject)

DPP is a program, while DNG is an image data format.
Logged
Gabor

Farkled

  • Jr. Member
  • **
  • Offline Offline
  • Posts: 97
Converting to DNG - do you lose quality?
« Reply #30 on: July 19, 2008, 02:17:52 pm »

Quote
(LOL, another one making definitive statements instead of asking about the subject)

DPP is a program, while DNG is an image data format.
[a href=\"index.php?act=findpost&pid=209032\"][{POST_SNAPBACK}][/a]
I apologize.  I wrote DNG when I meant the viewer in Bridge - which does not produce the same initial results as DPP when displaying CR2 files.
Logged

madmanchan

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 2115
    • Web
Converting to DNG - do you lose quality?
« Reply #31 on: July 19, 2008, 10:03:11 pm »

Quote
Ok, now I understand. ProPhoto is not linear, but LR converts an RGB image (which is never linear) into linear form and makes calculations on that; that is nothing special. The gamma-converted format is not suitable even for the simplest adjustments, like brightness or contrast. When writing the file, it gets gamma-converted again.
[a href=\"index.php?act=findpost&pid=208109\"][{POST_SNAPBACK}][/a]

Just to clarify: CR/LR converts the linear raw data to a RGB color space which has ProPhoto primaries, but linear gamma (i.e., gamma = 1.0). We sometimes refer to this informally has "ProPhoto linear." Subsequent image processing is done using this color space, the idea being that (1) the color space is very large, so it's unlikely that users will clip captured colors as a result of image processing, and (2) the pixel values are still linear, which is needed (or at least beneficial) for certain types of image processing.

Once the raw image processing is done, the ProPhoto linear data is then converted to the output space chosen by the user (e.g., in CR's workflow options or LR's export dialog). For display to the screen, of course the image data is converted via the monitor profile.

Some more details are in the DNG 1.2 spec for those who are interested.
« Last Edit: July 19, 2008, 10:04:52 pm by madmanchan »
Logged
Eric Chan

budjames

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 956
    • http://www.budjamesphotography.com
Converting to DNG - do you lose quality?
« Reply #32 on: July 20, 2008, 05:43:53 am »

For about 6 months, I've been archiving my Canon RAW files (40D, 1Ds MkII, 1Ds MkIII) and converting to DNG upon ingestion into Lightroom v1.4.

I cannot tell any difference between ACR and LR conversions using the original RAW file or DNG created by LR. I've made a number of prints in varying sizes from the DNG files and they look great! Any of my images requiring pixel mashing is done in PS CS3 with final sharpening done using PhotoKit Sharpener. The convenience of not having sidecar files lose is a nice plus to using DNG.

The only reason that I see to use RAW as the working format is if you use DXO Optics RAW conversion as this program does not read DNG. I've found that LensFixCL plug-in for PS CS3 provides the lens correction capability to PS which is all I need to fix the distortion of Canon wide angle zooms.

When you look at the great image masters like Jeff Schewe and Michael Reichmann using DNG, I think that they have enough experience to better judge the benefits of DNG vs original RAW conversions.

Cheers.
Bud James
North Wales, PA
Logged
Bud James
North Wales, PA [url=http://ww

Schewe

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 6229
    • http:www.schewephoto.com
Converting to DNG - do you lose quality?
« Reply #33 on: July 22, 2008, 03:12:13 pm »

Quote
When you look at the great image masters like Jeff Schewe and Michael Reichmann using DNG, I think that they have enough experience to better judge the benefits of DNG vs original RAW conversions.
[a href=\"index.php?act=findpost&pid=209510\"][{POST_SNAPBACK}][/a]


If you are talking about Camera Raw or Lightroom's ability to render either the original raws of DNGs, there is ZERO differences...in point of fact, Camera Raw's first steps when opening a raw file is to convert that data into DNG data (not a DNG file, just DNG data) in order to process it. I'm pretty sure Lightroom does the same thing.

So, from the standpoint of CR/LR, there CAN'T be any difference in rendering because the rendering is being based on the exact same data.

Look, there's a lot of misconception and myth regarding raw image data. Some people think the only Nikon and Canon can possible know what the raw files' secret data means...and to an extent it's true that the undocumented data does produce problems for 3rd parties. But if you look at the experience and knowledge Adobe has obtained regarding raw files, I think that with over 170 raw file formats under their belt, Thomas Knoll and the other Camera Raw engineers pretty much know exactly what's in those raw files.

There are some real advantages regarding DNG...the raw data & xmp metadata is contained in a single file. DNG has completely lossless compression. DNG is fully documented and for long term conservation and preservation, that's critical. DNG, as a raw file format, has enormous 3rd party support which can not be said for all the 170+ raw file formats which can be turned into DNG. Adobe is also in talks with the ISO to allow the ISO to take over stewardship of DNG. Adobe has already allowed the ISO to take over portions of TIFF for the development of TIFF-EP (Tiff for electronic photography) and they have a lot of experience in standardized formats such as PDF and initiatives such as XMP.

On the contrary, the camera companies (particularly Nikon and Canon) continue to keep their formats proprietary and undocumented and "secret" even if their formats are based on TIFF-EP and NEF & CR2 file formats are so close to DNG that a camera firmware update could prolly allow current cameras to write DNG in addition to their own formats.

There is nothing in Nikon and Canon's desire to keep their formats proprietary that is in the user's best interest...so, as a photographer I can't see any reason to support Nikon and Canon refusing to accept a standard...and any photographer who does support Nikon and Canon either don't understand the issues or have their own agenda in play.

There is one area where DNG is not a benefit with Lightroom and that is writing XMP metadata into the DNG files and then backing up. If you do choose to write the XMP metadata into the DNG files, their modification dates will be changes (because of the XMP metadata update) and any backup scheme that uses modification date to do incremental backups will see those files as updates cause all of the DNG file to be copied on backup. If you do import the raw files not as DNGs, the only way to save out the metadata is in XMP sidecar files–which compared to the image data is tiny.

But that's the only downside of using DNG if you are using Camera Raw/Lightroom...
Logged

budjames

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 956
    • http://www.budjamesphotography.com
Converting to DNG - do you lose quality?
« Reply #34 on: July 22, 2008, 06:02:13 pm »

Quote
If you are talking about Camera Raw or Lightroom's ability to render either the original raws of DNGs, there is ZERO differences...in point of fact, Camera Raw's first steps when opening a raw file is to convert that data into DNG data (not a DNG file, just DNG data) in order to process it. I'm pretty sure Lightroom does the same thing.

So, from the standpoint of CR/LR, there CAN'T be any difference in rendering because the rendering is being based on the exact same data.

Look, there's a lot of misconception and myth regarding raw image data. Some people think the only Nikon and Canon can possible know what the raw files' secret data means...and to an extent it's true that the undocumented data does produce problems for 3rd parties. But if you look at the experience and knowledge Adobe has obtained regarding raw files, I think that with over 170 raw file formats under their belt, Thomas Knoll and the other Camera Raw engineers pretty much know exactly what's in those raw files.

There are some real advantages regarding DNG...the raw data & xmp metadata is contained in a single file. DNG has completely lossless compression. DNG is fully documented and for long term conservation and preservation, that's critical. DNG, as a raw file format, has enormous 3rd party support which can not be said for all the 170+ raw file formats which can be turned into DNG. Adobe is also in talks with the ISO to allow the ISO to take over stewardship of DNG. Adobe has already allowed the ISO to take over portions of TIFF for the development of TIFF-EP (Tiff for electronic photography) and they have a lot of experience in standardized formats such as PDF and initiatives such as XMP.

On the contrary, the camera companies (particularly Nikon and Canon) continue to keep their formats proprietary and undocumented and "secret" even if their formats are based on TIFF-EP and NEF & CR2 file formats are so close to DNG that a camera firmware update could prolly allow current cameras to write DNG in addition to their own formats.

There is nothing in Nikon and Canon's desire to keep their formats proprietary that is in the user's best interest...so, as a photographer I can't see any reason to support Nikon and Canon refusing to accept a standard...and any photographer who does support Nikon and Canon either don't understand the issues or have their own agenda in play.

There is one area where DNG is not a benefit with Lightroom and that is writing XMP metadata into the DNG files and then backing up. If you do choose to write the XMP metadata into the DNG files, their modification dates will be changes (because of the XMP metadata update) and any backup scheme that uses modification date to do incremental backups will see those files as updates cause all of the DNG file to be copied on backup. If you do import the raw files not as DNGs, the only way to save out the metadata is in XMP sidecar files–which compared to the image data is tiny.

But that's the only downside of using DNG if you are using Camera Raw/Lightroom...
[a href=\"index.php?act=findpost&pid=209965\"][{POST_SNAPBACK}][/a]

Jeff, as always, thanks for your detailed explanation. I agree with you since I'm a follower (and purchaser of your products and tutorials). I learned a lot from the tutorials that you and Michael have produced, so keep them coming.

As a photo hobbyist, although a serious one, the workflow offered by using LR and PS CS3 instead of Canon's DPP serves me well as to keep me from having to deal with unfriendly software that does not have such a great community of sharing pros like yourself to support and encourage the learning process like Adobe products.

My prints and images look great so I quite happy with the results and the virtual network of similarly minded individuals on forums like this.

Cheers.
Bud James
North Wales, PA
Logged
Bud James
North Wales, PA [url=http://ww

Panopeeper

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 1805
Converting to DNG - do you lose quality?
« Reply #35 on: July 22, 2008, 08:29:52 pm »

Before we burst into tears, because not all files have been converted in DNG, a few things have to be explained.

Note, that the issue is NOT if converting native raw files in DNG causes loss of raw data. Such claims came only from amateurs.

To the evaluation of "advantages" coming with DNG:

1. Embedded XMP tags.

Well, you may clasp for this, or you may whine about it. It may seem to be an advantage for some, it is a show stopper for others. I deem it as an idiotic idea to store the adjustment parameters (secondary, changable information) inside the original, primary information, which ought to be preserved absolutely unchanged.

2. Lossless compression.

I don't know if I am supposed to laugh or to whine when reading this rubbish. The raw data is lossless, if is has been created losslessly. However, the lossy raw data does not become lossless by virtue of converting it in DNG.

3. "Enormous support".

It is totally irrelevant for any user, how many cameras are supported by the software (s)he is using.

4. Some (in fact, all important) manufacturers are sticking to their proprietory format.

I have all understanding for them. The DNG format does not allow for any new features, it does not allow for honouring in-camera settings (contrast, saturation, style, color space selection, etc.), and most importantly, it does not allow for the best possible color representation.

This is a very difficult problem. On one hand, the DNG specification can not account for future features. On the other hand, manufacturers don't want to reveal their plans, nor do they want to delay their release, until the DNG specification gets updated.

For example the proprietory raw converters honour picture styles and dust delete data. If someone goes DNG, one can forget about that.

5. DNG is a standard.

Well, it is a standard of Adobe. I hope, that before it will be declared an official standard, some software architecs will be asked about it, because this did not happen until now.

Let's keep an important fact in eyes: camera owners are using DNG due to Lightroom, Bridge and mainly for the features of ACR, not for DNG. If Canon came out with a raw converter equipped comparable to ACR, photogs would make a run for it and forget about ACR.

(Please note: I am a proponent of a raw standard, something like DNG, but I am not blind.)
Logged
Gabor

budjames

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 956
    • http://www.budjamesphotography.com
Converting to DNG - do you lose quality?
« Reply #36 on: July 22, 2008, 09:50:56 pm »

Quote
Before we burst into tears, because not all files have been converted in DNG, a few things have to be explained.

Note, that the issue is NOT if converting native raw files in DNG causes loss of raw data. Such claims came only from amateurs.

To the evaluation of "advantages" coming with DNG:

1. Embedded XMP tags.

Well, you may clasp for this, or you may whine about it. It may seem to be an advantage for some, it is a show stopper for others. I deem it as an idiotic idea to store the adjustment parameters (secondary, changable information) inside the original, primary information, which ought to be preserved absolutely unchanged.

2. Lossless compression.

I don't know if I am supposed to laugh or to whine when reading this rubbish. The raw data is lossless, if is has been created losslessly. However, the lossy raw data does not become lossless by virtue of converting it in DNG.

3. "Enormous support".

It is totally irrelevant for any user, how many cameras are supported by the software (s)he is using.

4. Some (in fact, all important) manufacturers are sticking to their proprietory format.

I have all understanding for them. The DNG format does not allow for any new features, it does not allow for honouring in-camera settings (contrast, saturation, style, color space selection, etc.), and most importantly, it does not allow for the best possible color representation.

This is a very difficult problem. On one hand, the DNG specification can not account for future features. On the other hand, manufacturers don't want to reveal their plans, nor do they want to delay their release, until the DNG specification gets updated.

For example the proprietory raw converters honour picture styles and dust delete data. If someone goes DNG, one can forget about that.

5. DNG is a standard.

Well, it is a standard of Adobe. I hope, that before it will be declared an official standard, some software architecs will be asked about it, because this did not happen until now.

Let's keep an important fact in eyes: camera owners are using DNG due to Lightroom, Bridge and mainly for the features of ACR, not for DNG. If Canon came out with a raw converter equipped comparable to ACR, photogs would make a run for it and forget about ACR.

(Please note: I am a proponent of a raw standard, something like DNG, but I am not blind.)
[a href=\"index.php?act=findpost&pid=210047\"][{POST_SNAPBACK}][/a]

This is not a political debate. Do what you want for your purposes. By making it easy, consistent and repeatable, Adobe and DNG will yield better results in the long-term. I'll stick to DNG, you do as you please.

Bud
Logged
Bud James
North Wales, PA [url=http://ww

ErikKaffehr

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 11311
    • Echophoto
Converting to DNG - do you lose quality?
« Reply #37 on: July 23, 2008, 12:21:53 am »

Hi,

Can you explain "It does not allow for the best possible color representation.", please.

Best regards
Erik
 

Quote
I have all understanding for them. The DNG format does not allow for any new features, it does not allow for honouring in-camera settings (contrast, saturation, style, color space selection, etc.), and most importantly, .


[a href=\"index.php?act=findpost&pid=210047\"][{POST_SNAPBACK}][/a]
Logged
Erik Kaffehr
 

Panopeeper

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 1805
Converting to DNG - do you lose quality?
« Reply #38 on: July 23, 2008, 01:08:20 pm »

Quote
Can you explain "It does not allow for the best possible color representation.", please
The DNG specification prescribes a certain method of converting the colors from the camera's color space in RGB (you can see often references to "color conversion matrixes"). The result of this conversion using the default matrixes is far from being optimal. The calibration process shows the weakness: only three or four colors can be included.

Other programs adopt different approaches of color conversion. The manufacturers' own converters can reproduce the colors better than ACR (there is no real discussion about it, only a few fanatics would dispute this). The difference may not be relevant for some (like myself), but it is the show stopper for others.

DNG version 1.2 has been published a few weeks ago; many of the changes are the implicite acknowledgment of this problem. It has to be shown yet, if the problems are solvable with the extentions.
Logged
Gabor

ErikKaffehr

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 11311
    • Echophoto
Converting to DNG - do you lose quality?
« Reply #39 on: July 23, 2008, 03:48:00 pm »

Thanks!

Quote
The DNG specification prescribes a certain method of converting the colors from the camera's color space in RGB (you can see often references to "color conversion matrixes"). The result of this conversion using the default matrixes is far from being optimal. The calibration process shows the weakness: only three or four colors can be included.

Other programs adopt different approaches of color conversion. The manufacturers' own converters can reproduce the colors better than ACR (there is no real discussion about it, only a few fanatics would dispute this). The difference may not be relevant for some (like myself), but it is the show stopper for others.

DNG version 1.2 has been published a few weeks ago; many of the changes are the implicite acknowledgment of this problem. It has to be shown yet, if the problems are solvable with the extentions.
[a href=\"index.php?act=findpost&pid=210196\"][{POST_SNAPBACK}][/a]
Logged
Erik Kaffehr
 
Pages: 1 [2] 3 4   Go Up