Pages: 1 2 [3] 4   Go Down

Author Topic: Converting to DNG - do you lose quality?  (Read 71290 times)

The View

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 1284
Converting to DNG - do you lose quality?
« Reply #40 on: July 24, 2008, 01:46:19 am »

Quote
The DNG specification prescribes a certain method of converting the colors from the camera's color space in RGB (you can see often references to "color conversion matrixes"). The result of this conversion using the default matrixes is far from being optimal. The calibration process shows the weakness: only three or four colors can be included.

Other programs adopt different approaches of color conversion. The manufacturers' own converters can reproduce the colors better than ACR (there is no real discussion about it, only a few fanatics would dispute this). The difference may not be relevant for some (like myself), but it is the show stopper for others.

DNG version 1.2 has been published a few weeks ago; many of the changes are the implicite acknowledgment of this problem. It has to be shown yet, if the problems are solvable with the extentions.
[a href=\"index.php?act=findpost&pid=210196\"][{POST_SNAPBACK}][/a]

Sounds very interesting. So, conversion to DNG is lossless, but... you may lose colors?
Logged
The View of deserts, forests, mountains. Not the TV show that I have never watched.

The View

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 1284
Converting to DNG - do you lose quality?
« Reply #41 on: July 24, 2008, 01:49:43 am »

I actually don't need DNG.

I'm perfectly satisfied with Canon's proprietary CR2 format.

There aren't actually THAT many RAW formats out there.

Don't we have a different lens mount for every camera system, too?

On the other hand, it seems to be critical to try to invent an artificial standard.

Remember the M42 lens mount?

I feel DNG limits me, as many software applications don't accept it. Lossless, or not, this seems a loss to me.
Logged
The View of deserts, forests, mountains. Not the TV show that I have never watched.

Panopeeper

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 1805
Converting to DNG - do you lose quality?
« Reply #42 on: July 24, 2008, 11:46:21 am »

Quote
So, conversion to DNG is lossless, but... you may lose colors?
Well, this term is correct if you make a direct comparison and find, that DPP creates more fidel colors, than ACR.

However, one has to consider, that the camera can be profiled in ACR. Some users have been doing that and reported better results, but allegedly still not as good as DPP. Apologists of the ACR approach counter, that the profiling needs to be done better, but that is ridiculous, as no-one came up with a perfect profile.

AFAIK it is not proven, that it is possible at all to adjust the ACR conversion with the result equalling the rendering of a camera-specific conversion (you can read sometimes about LUT or look-up table, meaning a very practical, sensor specific conversion instead of the generic approach).

The new DNG spec provides for additional profiles (these profiles have nothing to do with the profiling available on the user interface of ACR). However, these are still based on the basic approach of primary conversion by the matrixes; I don't know, how much this will help (I am not saying, that this is not an adequate solution; I don't know).

Adobe resisted to calls for allowing ICC profiles in the raw conversion; I don't know their reason.
Logged
Gabor

madmanchan

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 2115
    • Web
Converting to DNG - do you lose quality?
« Reply #43 on: July 24, 2008, 01:06:33 pm »

My understanding is that DNG support in non-Adobe apps is on its way, but will take some time to get to market.

There are some development concerns that we (i.e., members of the CR team at Adobe) have that end up trickling down to hurting users, in our opinion. Cameras with new formats take longer to support, which means a longer wait between the time a photographer gets a new camera and when the software will support it. Sometimes the new format is easy to add support for and can be done within a few days of work, thereby meaning the software is ready by the time the new model hits the streets and ends up in users' hands. Other times it takes weeks, or even months. A couple of new models that Michael covered recently on this site fall have new formats, unfortunately. They'll take extra time to support. With a bit homework you can probably figure which ones they are.

I also believe this indirectly hurts users because we get a lot of feature requests from users. Time spent deciphering new formats just to support a camera means less time spent working on improvements.

In general camera formats aren't very interesting. Photography, on the other hand, is very interesting. We'd rather spend time building better tools for developing photographs rather than figuring out what the 1's and 0's in the file mean. Ick.

BTW, Gabor, the DNG 1.2 spec allows a camera color profile to use 1 or more lookup tables, in addition to matrices. It is certainly true that none of the current profiles (built prior to DNG 1.2 development) contain any such lookup tables; the current profiles only use matrices to map camera 3-color or 4-color data to XYZ.
Logged
Eric Chan

Schewe

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 6229
    • http:www.schewephoto.com
Converting to DNG - do you lose quality?
« Reply #44 on: July 24, 2008, 02:28:06 pm »

Quote
There aren't actually THAT many RAW formats out there.

Don't we have a different lens mount for every camera system, too?

On the other hand, it seems to be critical to try to invent an artificial standard.
[{POST_SNAPBACK}][/a]


So, 170+ file formats isn't a lot of different file formats? That's what DNG/Camera Raw supports. And 170+ would be a problem if you were a software engineer that had to keep track of each and every new camera model's file format. Yes, there's NEF and CR2 (Nikon and Canon) but each and EVERY camera model produces a DIFFERENT file format, even if the file extension is NEF or CR2.

Take Canon for example...they've produces digital cameras whose file format went from CRW to TIF to CR2...that's three changes in base level file formats from one company in less that 8 years. Take Nikon...in their marketing stupidity they named digital files from their cameras NEF. What do they name files from their scanners? NEF...are they the same? Nope...not even close.

Until and unless the digital camera companies do finally arrive at a standardized file format, the long term preservation and conservation of today's digital capture is at severe risk. The fact that YOU don't see it is actually part of the problem, not the solution. And DNG is not an "artificial" solution...it's a well designed and implemented solution by a company with a track record for producing standards...something neither Nikon nor Canon have much experience with.

So, until you education yourself a bit more about the issues involved, it would be useful to quit making broad and sweeping and technically incorrect statements. Try doing some reading...[a href=\"http://photoshopnews.com/2005/05/11/digital-preservation/]Digital Preservation[/url]. In fact Congress has spent nearly 100 million dollars looking into the issue, see: The Library of Congress. You'll find that undocumented and proprietary file formats are the WORST thing for the industry.

And to have somebody like you pass off the efforts as a "artificial standard" is simply not useful (and ignorant).
« Last Edit: July 24, 2008, 02:29:37 pm by Schewe »
Logged

Panopeeper

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 1805
Converting to DNG - do you lose quality?
« Reply #45 on: July 24, 2008, 02:35:09 pm »

Quote
the DNG 1.2 spec allows a camera color profile to use 1 or more lookup tables, in addition to matrices
I did only a cursory reading of this subject yesterday, but my understanding is, that the hue-saturation mapping is an addition to the primary color space conversion, which remains unchanged (based on the conversion matrixes).

I can not judge, if this addition is enough to achieve an adequate conversion. At first sight, one could say this allows for any transformation; however, that is not so. The first, matrix based transformation leads to invalid results for many source co-ordinates, i.e. many colors can not be converted in XYZ, thus the following hue-saturation conversion is helpless. While this may not be totally avoidable by any method, I don't see it as given, that a sensor-specific conversion instead of the conversion by matrix would not yield a result preferrable to some customers.
« Last Edit: July 24, 2008, 02:45:46 pm by Panopeeper »
Logged
Gabor

The View

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 1284
Converting to DNG - do you lose quality?
« Reply #46 on: July 24, 2008, 08:07:03 pm »

Quote
I did only a cursory reading of this subject yesterday, but my understanding is, that the hue-saturation mapping is an addition to the primary color space conversion, which remains unchanged (based on the conversion matrixes).

I can not judge, if this addition is enough to achieve an adequate conversion. At first sight, one could say this allows for any transformation; however, that is not so. The first, matrix based transformation leads to invalid results for many source co-ordinates, i.e. many colors can not be converted in XYZ, thus the following hue-saturation conversion is helpless. While this may not be totally avoidable by any method, I don't see it as given, that a sensor-specific conversion instead of the conversion by matrix would not yield a result preferrable to some customers.
[a href=\"index.php?act=findpost&pid=210472\"][{POST_SNAPBACK}][/a]

Thank you for your view that is not biased by relations to any particular software company.

Of course, everybody who stands up for his point of view, will have to take a certain amount of abuse of people who would love to ignore how opinions are formed and knowledge is gained: by discussion and competition out there in the real world.

But still, this the price of having a backbone and the ability to judge and draw conclusions is worth it. After all, only corpses drift with the flow.

So, Panopeeper, I really appreciate your standing up here and allowing readers to gain knowledge. People like you enrich this forum as a way to get knowledge, that is possible drowned out by the chorus otherwise.

Generally, the quest for maximum quality and practicability should never be underestimated. The age of digital photography is exiting its phase of infancy, and a solution will be found, one way or other.
« Last Edit: July 24, 2008, 08:07:56 pm by The View »
Logged
The View of deserts, forests, mountains. Not the TV show that I have never watched.

The View

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 1284
Converting to DNG - do you lose quality?
« Reply #47 on: July 24, 2008, 08:12:58 pm »

It is an honorable attempt for any company to look for a general standard, a kind of lingua franca of RAW conversion.

At the same time, while the attempt is good, it would be hubris for such a company and anybody who works for it, to proclaim such a standard, and pursue those who don't agree, as infideles.

Especially when that certain standard is not that future proof - and that's the way it looks if you attempt to read Panopeepers comment instead of just browsing to discard.

If Canon, Nikon, and others don't take part in DNG this has surely reasons that may be economical, "political" (market power related), or technical (does not give most possibilities, like color accuracy and the ability to store proprietary information).
Logged
The View of deserts, forests, mountains. Not the TV show that I have never watched.

The View

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 1284
Converting to DNG - do you lose quality?
« Reply #48 on: July 24, 2008, 08:26:43 pm »

There have been countless attempts for unity of any kind of language in any field.

Not only in M42 lens mounts...

Think of Esperanto. An artificial language, put together from English, Spanish, French, and other European languages. It provided a simple grammatical structure, simple declinations of the verbs, no "weird" (historically grown) rules, and a very practical, easy to memorize vocabulary.

And it was a failure. Esperanto was an honorable attempt, but failed, because it was an artificial standard, that did not include what people need in a language, and what goes beyond communicating information.

Technical standards can and have repeatedly hampered technological development, and to claim otherwise is actually more ignorant than the insolence of ignoring Adobe's supremacy in the development of software standards.

If someone invented a standard camera companies could customize to their needs, it would stand a chance, and it may happen.

Any other standard, that does not find support, is per definitionem, artificial. Because it only acts like a standard, but isn't. Standards become standards through widespread acceptance, not by definition.
Logged
The View of deserts, forests, mountains. Not the TV show that I have never watched.

DarkPenguin

  • Guest
Converting to DNG - do you lose quality?
« Reply #49 on: July 24, 2008, 11:30:55 pm »

I don't know that what panopeeper said was a direct problem with DNG.  I think it is fallout from the fact that a lot of camera makers software doesn't read DNG.
« Last Edit: July 24, 2008, 11:31:14 pm by DarkPenguin »
Logged

ErikKaffehr

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 11311
    • Echophoto
Converting to DNG - do you lose quality?
« Reply #50 on: July 25, 2008, 02:02:11 am »

Hi,

Historically there has been a problem with RAW-formats. Let's say that you are shooting Canon, assuming that you have D341. Are you sure that Canons DPP will support that camera for forever? Or do you think that you are going to be able to use present Canon DPP on all versions of Windows or Mac OS/X forever? World may even go Linux in 10 years ;-)

Now you can convert your pictures to TIFF. Even 16 bit TIFF and ProPhoto RGB will loose information (much more than you ever could loose with DNG). The question is which TIFF. All TIFF files cannot be read by all programs. You would probably go for Adobe TIFF, Windows byte order, no compression. If you use Adobe TIFF you are pretty close to Adobe DNG.

The other option is to go with JPEG. JPEG is much better than it's reputation, in my view. The problems with JPEGs are:

1) JPEG is not really intended for further processing
2) You cannot use a big color space with JPEG, because it's only 8 bits. Adobe is probably OK, Pro Photo RGB is not, because JPEG is only 8 bits and you lose a few bits when converting between color spaces.

Regarding the comparision with esperanto I don't really feel it is correct, it's more like saying: From now on everyone is speaking California dialect of American English!

Erik

Quote
There have been countless attempts for unity of any kind of language in any field.

Not only in M42 lens mounts...

Think of Esperanto. An artificial language, put together from English, Spanish, French, and other European languages. It provided a simple grammatical structure, simple declinations of the verbs, no "weird" (historically grown) rules, and a very practical, easy to memorize vocabulary.

And it was a failure. Esperanto was an honorable attempt, but failed, because it was an artificial standard, that did not include what people need in a language, and what goes beyond communicating information.

Technical standards can and have repeatedly hampered technological development, and to claim otherwise is actually more ignorant than the insolence of ignoring Adobe's supremacy in the development of software standards.

If someone invented a standard camera companies could customize to their needs, it would stand a chance, and it may happen.

Any other standard, that does not find support, is per definitionem, artificial. Because it only acts like a standard, but isn't. Standards become standards through widespread acceptance, not by definition.
[a href=\"index.php?act=findpost&pid=210526\"][{POST_SNAPBACK}][/a]
« Last Edit: July 25, 2008, 02:02:43 am by ErikKaffehr »
Logged
Erik Kaffehr
 

madmanchan

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 2115
    • Web
Converting to DNG - do you lose quality?
« Reply #51 on: July 28, 2008, 09:31:26 pm »

Gabor, you are correct that the lookup table(s) follows the initial conversion using a color matrix. I am confused by your description of "sensor-specific conversion" -- please clarify this.

All raw converters ultimately have to take the demosaiced raw camera coordinates (RGB or GMCY) and map them to XYZ or Lab, though it may be only an intermediate step. This is especially true for raw converters that use ICC profiles, where either XYZ or Lab D50 is the working space. The final mapping from XYZ to any of the standard RGB spaces is just a linear transform plus an encoding curve. The lookup tables provide a mechanism for non-linear corrections (e.g., if you need to tweak deep saturated reds differently than less-saturated reds, something the earlier matrix-only profiles obviously could not do).

I think maybe your concern is that if the initial color matrix from camera space to XYZ causes clipping in XYZ, then you've lost colors that cannot be retrieved later. This is valid and thus care needs to be taken in profile construction to minimize such clipping. In practice it is rarely an issue because the extreme camera values that would lead to such clipping cannot occur (for example you normally can't get a RGB coordinate of (1,0,0) from a camera).

However, until we have DNG 1.2 profiles I won't be able to provide you with any concrete image examples. Luckily we won't have to wait too long.
Logged
Eric Chan

Schewe

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 6229
    • http:www.schewephoto.com
Converting to DNG - do you lose quality?
« Reply #52 on: July 28, 2008, 11:53:16 pm »

Quote
Technical standards can and have repeatedly hampered technological development, and to claim otherwise is actually more ignorant than the insolence of ignoring Adobe's supremacy in the development of software standards.
[a href=\"index.php?act=findpost&pid=210526\"][{POST_SNAPBACK}][/a]


Yes...when the "standard" was developed for the benefit of a cartel or small group who wished to maintain total control over a specific standard for their own benefit...as it relates to industry adopted standard, I think you'll have to cite some cases for me to think that opinion is anything other than hogwash...

Photography has a long history of adopting industry wide standards. Neither Nikon nor Canon created their own film size standards...35mm film came (well, I suppose it still does) in a set specification for length, width and number and placement of sprocket holes–other wise, how the heck would you have loaded the same film in different manufactures?

Kodak and Fuji cooperated in the development of the E-6 color film processing film and processing...even though each manufacture was free to develop films that adhered to the standards but deliver different results on film. The type type, size and processing was standardized.

Now, maybe you have an anti-Adobe feather up your butt, I don't know, but until and unless the digital photo industry adopts industry wide standards for raw film formats, we'll still be in the "wild west" where individual companies can play fast and loose and not have their toes held to the fire for blowing out standards...heck, even Nikon and Canon adhere to SOME standards such as EXIF (which unfortunately still has some wriggle room that causes problems).

And if you think it's big old bad Adobe trying to take over and control the DNG standard, I'll point out that Adobe has recently been in discussions on turning over DNG to the ISO for adoption...I can't tell you the results of the most recent meetings at the moment cause I haven't heard about the results...but it's encouraging to note the interest on the part of Adobe and the ISO. Sorry, I think you completely miss the point regarding the importance of standards on the behalf of the individual camera user...it seriously is NOT in YOUR best interests to let Nikon and Canon play with your file formats...
Logged

Nick Rains

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 705
    • http://www.nickrains.com
Converting to DNG - do you lose quality?
« Reply #53 on: July 29, 2008, 04:38:22 am »

Quote
And if you think it's big old bad Adobe trying to take over and control the DNG standard, I'll point out that Adobe has recently been in discussions on turning over DNG to the ISO for adoption...I can't tell you the results of the most recent meetings at the moment cause I haven't heard about the results...but it's encouraging to note the interest on the part of Adobe and the ISO. Sorry, I think you completely miss the point regarding the importance of standards on the behalf of the individual camera user...it seriously is NOT in YOUR best interests to let Nikon and Canon play with your file formats...
[a href=\"index.php?act=findpost&pid=211305\"][{POST_SNAPBACK}][/a]

Correct me if I'm wrong by is not TIFF an Adobe format originally? If so then their track record of allowing free and open use without exploitation is pretty good.
Logged
Nick Rains
Australian Photographer Leica

Gary Brown

  • Full Member
  • ***
  • Offline Offline
  • Posts: 211
Converting to DNG - do you lose quality?
« Reply #54 on: July 29, 2008, 10:32:08 am »

Quote
Correct me if I'm wrong by is not TIFF an Adobe format originally? If so then their track record of allowing free and open use without exploitation is pretty good.
[{POST_SNAPBACK}][/a]
Ditto for PDF:
[a href=\"http://www.iso.org/iso/pressrelease.htm?refid=Ref1141]PDF format becomes ISO standard[/url]
Logged

Schewe

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 6229
    • http:www.schewephoto.com
Converting to DNG - do you lose quality?
« Reply #55 on: July 29, 2008, 10:43:36 am »

Quote
Correct me if I'm wrong by is not TIFF an Adobe format originally? If so then their track record of allowing free and open use without exploitation is pretty good.
[a href=\"index.php?act=findpost&pid=211340\"][{POST_SNAPBACK}][/a]

Actually, TIFF was started by Aldus (and MSFT along with others) and was deeded into Adobe stewardship when Adobe bought Aldus. And yes, Adobe has indeed shown that they can take care of things long term (even when they didn't start it). Adobe even allowed the ISO to to adopt it for use in TIFF-EP, which of course, the camera companies have adopted for use in NEF and CR2 files. The utter irony of this is that both recent NEF and CR2 files are so well formed TIFF-EP formats that they might as well be DNGs (and the cameras could prolly receive the ability to write DNGs with merely a firmware upgrade).

See, between TIFF-EP and DNG, Adobe put Nikon and Canon to school...teaching them how to make well formed files. Course they still refuse to document their formats :~(

Which pretty much completely deflates the contention that a standard will retard innovation. In the case of DNG, the standard actually taught the camera companies a thing or two. Which actually points to DNG's success (rather than failure).
« Last Edit: July 29, 2008, 10:44:06 am by Schewe »
Logged

DarkPenguin

  • Guest
Converting to DNG - do you lose quality?
« Reply #56 on: July 29, 2008, 11:24:10 am »

Quote
However, until we have DNG 1.2 profiles I won't be able to provide you with any concrete image examples. Luckily we won't have to wait too long.
[a href=\"index.php?act=findpost&pid=211284\"][{POST_SNAPBACK}][/a]

So make like with the examples.

Also, does the existence of these profiles mean that Magne Neilsen is going to get more of my money?
Logged

Brad Proctor

  • Full Member
  • ***
  • Offline Offline
  • Posts: 150
Converting to DNG - do you lose quality?
« Reply #57 on: July 29, 2008, 01:27:31 pm »

This discussion about standards reminds me of open source software and the movement behind it.  Businesses exist for the sole purpose of making a profit.  That is what they do.  That is what they are for.  So every time a company says they care about you, about innovation, about the world or anything else, remember that all they really care about is making a profit.

In general, standards (true standards) are great for us.  They promote competition among companies which leads to innovation.  Mainly this comes from the ability of multiple companies to write competing products that all work on the same standards.

When a company keeps the information private, no one else can develop for it.  Even if it isn't directly competing.  Imagine if computer components were all specific to a particular brand.  You can only plug this USB device into a Dell computer.  How about if you had to buy a specific power cord for everything in your house depending on who made the outlet.

Once a companies owns an important standard (and doesn't share).  Innovation comes to a grinding halt.  Why would the company spend money on development when they are the only place for consumers to go.

So, to make a long story short (too late)... DNG = good.
Logged
Brad Proctor

Schewe

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 6229
    • http:www.schewephoto.com
Converting to DNG - do you lose quality?
« Reply #58 on: July 29, 2008, 03:18:55 pm »

Quote
So every time a company says they care about you, about innovation, about the world or anything else, remember that all they really care about is making a profit.
[a href=\"index.php?act=findpost&pid=211482\"][{POST_SNAPBACK}][/a]


In general I can't fault your logic here...but there ARE exceptions to the rule. Adobe, as a company obsesses over always trying to "do the right thing"...and for most everything relating to technical considerations (that stuff under the influence of the engineers and not the suits), I can say that Adobe does indeed strive to do those things that it views as in the interests of the industry (sometimes to their own discomfort).

On the biz side, they do often do stooopid things...not on purpose, but usually by accident. But DNG is a prime example of Adobe (read Thomas Knoll, Zalman Stern, Eric Chan and Peter Merrill as the engineering team) "doing the right thing"...
Logged

NikosR

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 622
    • http://
Converting to DNG - do you lose quality?
« Reply #59 on: July 31, 2008, 10:03:42 am »

Jeff,

That's all nice and dandy but, taking Panokeeper's and Chan's parallel discussion into consideration, what would have happened if Nikon for example had decided to take up DNG and abandon their proprietary files 2-3 years ago? Would we all have to suffer the intolerable colours ACR sometimes comes up with (used to??) from Nikon files regardless the raw converter we would use? Would Nikon D3/D300/D700 be able to take advantage (if they opted to) as they can do now of excellent automatic correction of colour aberrations in their raw files?

I don't know the answer to these questions but all I can say is that I can see no standard being really viable unless it is developed and agreed upon by all major parties involved AND manages to both protect intellectual property and confidentiality AND does not indirectly hinder innovation (or what each party views as their strategic or competitive advantage).
« Last Edit: July 31, 2008, 10:20:19 am by NikosR »
Logged
Nikos
Pages: 1 2 [3] 4   Go Up