Luminous Landscape Forum
Raw & Post Processing, Printing => Adobe Lightroom Q&A => Topic started by: Luvntravln on March 30, 2009, 12:28:30 am
-
Hi,
Do you convert your RAW images to DNG before processing? If you do, why; if you don't, why?
-
Not anymore.
Three reasons ...
The converter bakes in panasonic lens corrections so you can't get a DNG that hasn't already been demosaiced.
Keeping changes confined to external .xmp (read: small) files rather than entire DNG (read: large) files makes for smaller backups which is particularly handy when using online backup services like Mozy Home.
The Canon raw converter offers some features that are more of a pain to do in DNG supporting apps.
-
Do you convert your RAW images to DNG before processing? If you do, why; if you don't, why?
Yes. The reasons are stated here:
http://www.ppmag.com/reviews/200709_adobedng.pdf (http://www.ppmag.com/reviews/200709_adobedng.pdf)
-
Yes. The reasons are stated here:
http://www.ppmag.com/reviews/200709_adobedng.pdf (http://www.ppmag.com/reviews/200709_adobedng.pdf)
Sorry Andrew, this is not so. Your paper deals correctly with the aspects of metadata editing. However, that has nothing to do with DNG; it works equally with native raws.
The difference is in reality where the metadata will be stored. You see embedding the metadata in the raw file as an advantage; I see it as a violation of the very basic principle, namely let the original untouched. If one does not want to archive the adjustment metadata, this is no problem; but if one does need that, the embedded metadata is disadvantage pure.
As to the file size: in times of terrabyte capacity on hard disk, the difference between the sizes resulted by two methods of compression is hardly a decivive reason. In fact, there are cases when DNG is larger than the native raw, but I would not mention this as an advantage of the native raw format.
-
The difference is in reality where the metadata will be stored. You see embedding the metadata in the raw file as an advantage; I see it as a violation of the very basic principle, namely let the original untouched. If one does not want to archive the adjustment metadata, this is no problem; but if one does need that, the embedded metadata is disadvantage pure.
You are entitled to that opinion. I don't share it, nor do others. The very point of the DNG container IS to store this additional data. You or anyone else is entitled to save off the original Raw proprietary data. I want an up-dateable preview and one that I can control for printing outside a Raw converter (in fact, I look forward to multiple such previews).
As for the time to update the DNGs when backing up, its a moot point for me as well. I have this all happening while I'm asleep at night. I happen to feel that computers should do mundane work when we humans are doing other useful things.
-
You are entitled to that opinion. I don't share it, nor do others
Correctly it is nor do some others; but some other others do share my opinion. This is largely a question of workflow, but not only. Some computer ill- and semiliterates do not understand the implications; there are some, who don't even archive the original at all.
One of the implications is, that several, different versions of the entire file can be created. For example if someone creates a backup of the archive on red-only storage, one can end up with different versions of 20 MB files, where the difference is only a tag and perhaps the preview.
Let me quote Erik Chan, a Camera Raw Engineer, from an Adobe forum:
To be clear, Adobe did __not__ create DNG in the hopes that photographers shooting non-DNG raw files would suddenly convert them all to DNG files and then throw away their original non-DNG raw files. Instead, Adobe created DNG has an example of a documented format (a set of TIFF extensions) that would improve interoperability among hardware and software vendors, as well as be suitable for archiving
-
Let me quote Erik Chan, a Camera Raw Engineer, from an Adobe forum:
To be clear, Adobe did __not__ create DNG in the hopes that photographers shooting non-DNG raw files would suddenly convert them all to DNG files and then throw away their original non-DNG raw files. Instead, Adobe created DNG has an example of a documented format (a set of TIFF extensions) that would improve interoperability among hardware and software vendors, as well as be suitable for archiving
That one can take a rendered TIFF or JPEG and place that into a DNG is a move I wish Adobe had not done. Its called Digital Negative. Adding the ability to place non Raws into the container really confused a lot of people needlessly.
-
That one can take a rendered TIFF or JPEG and place that into a DNG is a move I wish Adobe had not done. Its called Digital Negative. Adding the ability to place non Raws into the container really confused a lot of people needlessly.
We need a digital positive. Have a single container for a stack of conversions.
-
Yes. The reasons are stated here:
http://www.ppmag.com/reviews/200709_adobedng.pdf (http://www.ppmag.com/reviews/200709_adobedng.pdf)
The "reasons" in that article for converting to DNG don't make any sense if you bother to think about them a bit.
1) Store the metadata with in the same file. Duh, it's a digital NEGATIVE, meant to preserve all of the original image data WITHOUT CHANGES.
2) Non-proprietary format. Will all those who think that Nikon or Canon will go out of business or stop supporting their RAW format please raise their hands?
3) Smaller files. 1.5 terabyte hard disk = $140.
A possible reason for staying wth RAW is that the camera manufacturer might improve their software for processing their own RAW format, and we cannot be sure that these improvements will be available in rendering algorithms from Adobe or others.
Peter
-
The "reasons" in that article for converting to DNG don't make any sense if you bother to think about them a bit.
1) Store the metadata with in the same file. Duh, it's a digital NEGATIVE, meant to preserve all of the original image data WITHOUT CHANGES.
2) Non-proprietary format. Will all those who think that Nikon or Canon will go out of business or stop supporting their RAW format please raise their hands?
3) Smaller files. 1.5 terabyte hard disk = $140.
A possible reason for staying wth RAW is that the camera manufacturer might improve their software for processing their own RAW format, and we cannot be sure that these improvements will be available in rendering algorithms from Adobe or others.
Peter
Re: Item 2.
*raises hand*
-
Re: Item 2.
*raises hand*
That's right; this is not an impossible scenario.
However, the relevant question is not if a big manufacturer can go out of business, but if the existing native raw files would be affected by that event.
NO. Although the manufacturer's own raw processor may become obsolate after some system changes, other, independent raw processors will not stop supporting the old file formats, nor will the DNG converter stop. In other words, the worst case is, that one needs to convert the native raw files in DNG if they can not be processed directly.
-
2) Non-proprietary format. Will all those who think that Nikon or Canon will go out of business or stop supporting their RAW format please raise their hands?
They don't have to go out of business, just stop supporting the format. Kodak is still in business. Can I send you some of the very old DCS proprietary files I have that I can no longer open?
-
Can I send you some of the very old DCS proprietary files I have that I can no longer open?
Why can't you open them with the old software?
Anyway, the existence of the DNG converter eliminates this problem.
-
Why can't you open them with the old software?
It only runs under OS9, no machines here run OS9! In fact, very old DCS files require even older OS's.
Same issue with PhotoCD scans. You have to run the acquire module under Rosetta, something that very soon will not be possible on newer hardware.
-
Just to be clear, Andrew, the quote of mine that Gabor posted above wasn't intended by me to support the notion of putting non-raw data in a DNG container.
Instead, I was trying to say that while some users may benefit from using the Adobe DNG Converter to produce DNGs from non-DNG raw files, in my opinion the real benefit of DNG (or at least some standard, documented raw format, whether it be DNG, TIFF/EP, or something else) is when they are generated directly by the camera.
In the case of DNG, for instance, an in-camera DNG can contain an image checksum, which DNG readers can use to check for image integrity after images have been read off the card (or at any other time). As another example, the makers can embed 1 or more color profiles to be used for portable default color rendering across DNG readers; Casio and Pentax have done this quite effectively, without having to divulge what recipes they used to cook up those profiles in their labs.
My original comment about TIFF extensions was simply about the fact that the low-level file format (in terms of its structure) looks like a TIFF, which makes it relatively easy to support for software developers who are already used to dealing with TIFF. Developers also have the option of just using the DNG SDK which is a basic DNG reader and writer.
-
Just to be clear, Andrew, the quote of mine that Gabor posted above wasn't intended by me to support the notion of putting non-raw data in a DNG container.
Understood (and expected <g>). Before your time too.
As we've discussed, the integrity checking is another big plus for DNG.
-
The difference is in reality where the metadata will be stored. You see embedding the metadata in the raw file as an advantage; I see it as a violation of the very basic principle, namely let the original untouched. If one does not want to archive the adjustment metadata, this is no problem; but if one does need that, the embedded metadata is disadvantage pure.
Gabor
Can you point to any evidence that shows that the output from a DNG file is in any way different to that from it's original RAW when both are processed in the same way with the same RAW converter? Your stance on the 'purity' of the original RAW data seems to me to be ideology rather than reality, but since you are obviously a well informed kinda guy I'd be interested to find out if you can show firm examples why you have this apparently deep aversion to DNG files.
In other threads Andrew and I have pointed out some practical advantages which are pretty well indisputable, all we have had back are what appear to be purely theoretical objections. If these are meaningful in the 'real world' then fine, I'd like to know more, but if not then they just confuse people..
Show me the Money!
-
Can you point to any evidence that shows that the output from a DNG file is in any way different to that from it's original RAW when both are processed in the same way with the same RAW converter?
I find it important to remove the condition processed in the same way with the same RAW converter for a variety of reasons:
1. different raw converters may interpret the same data differently (this is obvious for anyone, who tried the manufacturer's raw processor and any other one),
2. "processed in the same way" is past all boundaries of mushiness; what is the meaning of "same way" for example relating to noise reduction?
Your stance on the 'purity' of the original RAW data seems to me to be ideology rather than reality, but since you are obviously a well informed kinda guy I'd be interested to find out if you can show firm examples why you have this apparently deep aversion to DNG files
1. I do not have any "aversion" to DNG files. The DNG design is a small-minded patchwork, but that is nothing special. As the matter of fact, my related product, Rawnalyze started out with supporting only DNG files.
2. While I have quite a few objections re the DNG specifications, the subject in this forum is something different, namely if to convert native raw files to DNG.
3. I am not only well-informed, but I am experienced enough not to *absolutely* trust anything, which "comes out of computer", not even if I myself have programmed it.
4. A specific example for the raw to DNG conversion being "inventive" for the lack of DNG ways expressing something present in the raw file: Canon's sRaw files.
5. A specific example for the Adobe DNG converter causing destruction: see around post #26 in this Adobe forum (http://www.adobeforums.com/webx?14@@.59b79390)
You are free to believe, that software coming out of the Adobe (or any other) door is free of error, but you are not free to expect, that any experienced person takes your considerations seriously.
-
2. "processed in the same way" is past all boundaries of mushiness; what is the meaning of "same way" for example relating to noise reduction?
You are free to believe, that software coming out of the Adobe (or any other) door is free of error, but you are not free to expect, that any experienced person takes your considerations seriously.
OK, I'd say that 'in the same way' meant exactly that. Open the file in the raw processor and export it using all the same parameters as the other file. is that rigourous enough?
Same NR value, same brightness value, same black point value, same, same etc etc etc. Same version of the same RAW processor too in case we are not clear. if this is not 'samey' enough for you then you are operating in realms of pedantry and sophistry way beyond us mere mortals.
"but you are not free to expect, that any experienced person takes your considerations seriously."
I was keeping this discussion respectful, as is my usual way, however this comment could be taken as rather patronising, maybe you meant otherwise...
-
OK, I'd say that 'in the same way' meant exactly that. Open the file in the raw processor and export it using all the same parameters as the other file. is that rigourous enough?
Somehow I "skipped" your with the same RAW converter; I was thinking of conversions with different raw converters.
However, this is irrelevant; the criterion is not that the results be the same but that they are not erroneous. The error I showed in the example was identical when processing the NEF or the DNG.
I was keeping this discussion respectful, as is my usual way, however this comment could be taken as rather patronising, maybe you meant otherwise...
Have you actually read the post I linked to? You asked for evidence, and I provided one; note, that this is not the only case when the DNG converter created an erroneous output.
Anyone, who blindly believes in any program's error free functioning and throws away the original is naive, to say the least. This is not an issue with Adobe; Nikon's Capture X too made error when storing the adjustment parameters in the NEF file. Not a serious error, but enough to demonstrate, that the product did not go through a rigorous test.
-
Have you actually read the post I linked to? You asked for evidence, and I provided one; note, that this is not the only case when the DNG converter created an erroneous output.
Of course I did. Please don't be so quick to assume things...what part of my post led you to think that? I made no reference to it at all, just to your 'tone'.
Anyway, moving on.. Yes, that was in interesting link, thanks, I will consider it further.
-
Hi just read the DNG spec and have a question
Does Adobe purport that camera manufacturers replace the native RAW file with the DNG spec to give say, JPEG and DNG instead of currently JPEG and native RAW?
While I agree a common file spec would be useful I cannot see the camera manufacturer giving up their costly device characterisation data that easily.
-
papa, this is a common misconception. Most of the makers' "costly device characterisation" is stored in maker notes. Whether a maker note is stored in a CR2 container, a NEF container, a DNG container, or a TIFF/EP container does not matter; the raw conversion software can read the same (proprietary) information regardless of the container. In fact, when the Adobe DNG Converter creates a DNG from a non-DNG raw file, it copies over the maker note, even though it doesn't most of the contents of that note. Cameras today that shoot DNG files directly also store characterization data in the DNG maker note; only the makers of those cameras understanding the meaning of those notes. Those makers have "given up" nothing.
Different makers will take different approaches to a common raw format.
Example 1: Some makers like Ricoh, Leica, and Casio are opting for the DNG-only route, i.e., their cameras do not offer a non-DNG raw mode. They are fine with putting as-shot white balance (i.e., maker neutral), embedded color profiles, etc. in public tags. Whatever they feel uncomfortable with sharing goes into the DNG maker note.
Example 2: Other makers like Pentax are opting for two raw modes, one that is DNG, one that is non-DNG (for Pentax, that would be PEF).
-
have I got this right? In order to process a RAW file be it a DNG or native RAW file, the device RGB to CIEXYZ matrix or transforms need to be used.
The .NEF files for example, do not contain the Device RGB to CIEXYZ transforms, these are in the accompanying proprietary software.
For eg .ACR has to use its own 'home made' camera characterizations as these are not available in the 'maker notes', not that I could find at least.
I you could point me to where this information is I would be grateful.
-
have I got this right? In order to process a RAW file be it a DNG or native RAW file, the device RGB to CIEXYZ matrix or transforms need to be used
Not right. The transformation between the camera's color space and the target (sRGB or Adobe RGB) does not need to be done by matrix operations. Adobe decided for this way in order to make the transformation more formalistic. Other raw processors, typically the manufacturers' own products do not use this method. The other method is direct transformation (often but incorrectly called LUT).
The inferiority of ACR's color reproduction was due primarily to this method. Now, with the additional profiling option (DNG Profile Editor) this is better, but it is still questionable if this method can ever yield equally good result as the more direct transformation.
Note, that the transformation depends on the illumination; there are different matrixes for different illumination.
-
I thought that the Device RGB ( or coordinates) had to be transformed to CIEXYZ BEFORE any further colour space transformations take place.
Is this not usually done using a 3x3 matrix?
-
Not right. The transformation between the camera's color space and the target (sRGB or Adobe RGB) does not need to be done by matrix operations. Adobe decided for this way in order to make the transformation more formalistic. Other raw processors, typically the manufacturers' own products do not use this method. The other method is direct transformation (often but incorrectly called LUT).
The inferiority of ACR's color reproduction was due primarily to this method. Now, with the additional profiling option (DNG Profile Editor) this is better, but it is still questionable if this method can ever yield equally good result as the more direct transformation.
Note, that the transformation depends on the illumination; there are different matrixes for different illumination.
I don't quite agree ...
In most cases, the makers' default color transforms are actually using a matrix as well, i.e., in addition to lookup tables. The matrix used for initial rotation and scaling to a canonical space (usually XYZ with a D50 white point), as well as to account for per-device characterization. The LUT is mostly used to build in color preference.
I do not see why you question whether a DNG profile can achieve quality equivalent to your so-called "direct transformation." A DNG profile can perform a matrix and a LUT operation and hence can effectively perform an arbitrary 3D to 3D color transformation.
-
I thought that the Device RGB ( or coordinates) had to be transformed to CIEXYZ BEFORE any further colour space transformations take place.
Is this not usually done using a 3x3 matrix?
papa, if the goal is to get fairly accurate scene colorimetry reproduction, then a 3x3 matrix to CIE XYZ space (or other standard space like RIMM) from device coordinates is generally sufficient. This is a reasonable first step before performing additional color operations, i.e., get the image into a color space with a well-understood set of primaries.
However, this is not a requirement. An image processing program can actually perform RGB color operations while in the native RGB linear camera system (or some derived space, e.g., after applying some gamma encoding). On the other hand, exposing editing operations to the user that manipulate the image in such a space is usually not a good idea, because it means that the controls will behave differently on different cameras (e.g., a RGB tone curve would behave very differently on a Nikon D70 versus a Nikon D3).
-
Yes. The reasons are stated here:
http://www.ppmag.com/reviews/200709_adobedng.pdf (http://www.ppmag.com/reviews/200709_adobedng.pdf)
As Gabor has pointed out, the DigitalDog's referenced assay is concerned mainly with the advantages of parametric editing, which can be done with raw files as well as DNGs. The advantage of DNG is that the metadata for the edits are stored directly in the DNG rather than in a separate XMP file or database as is the case when one is editing raw files with ACR. In the latter instance, the metadata can become separated from the raw file and the edits are then lost.
With my Nikon D3 using 14 bit losslessly compressed NEFs, converting to DNG results in a slightly smaller file, unless the option of embedding the raw file in the DNG is selected, in which case the resulting DNG is approximately double the size of the NEF. If one has only a DNG without the embedded raw file, the ability to edit the file with Nikon Capture NX or other third party software that does not support DNG is lost. Currently, ACR is my preferred editor, but there are advantages to Nikon Capture NX. NX can automatically correct for chromatic aberration with Nikkor and third party lenses and for light falloff with lenses that have a CPU. With ACR, one must do these corrections manually by entering the appropriate parameters, and this can be cumbersome with zoom lenses where the parameters differ with focal length and aperture. In the future, it is likely that additional lens defects will be corrected in software and it could be important not to lose this ability. If one chooses to take advantages of DNG, I think that it is important to save or embed the original raw file.
Parenthetically, I note that Andrew still thinks that the raw file is grayscale and that color is magically created by the raw converter. This feat is only slightly less astounding than the conversion of water to wine at the wedding at Cana.
Bill
-
As Gabor has pointed out, the DigitalDog's referenced assay is concerned mainly with the advantages of parametric editing, which can be done with raw files as well as DNGs. The advantage of DNG is that the metadata for the edits are stored directly in the DNG rather than in a separate XMP file or database as is the case when one is editing raw files with ACR.
I guess the last paragraph in column one discussing sidecar files wasn't obvious enough for some readers!
-
I guess the last paragraph in column one discussing sidecar files wasn't obvious enough for some readers!
Perhaps so, or perhaps the title of the article was ill-chosen.
Bill
-
Perhaps so, or perhaps the title of the article was ill-chosen.
You are entitled to that opinion, that said, I find many compelling features of DNG and find it a powerful option given the alternatives.
I'll tell the magazine you might be interested in becoming an editor....
-
Can you point to any evidence that shows that the output from a DNG file is in any way different to that from it's original RAW when both are processed in the same way with the same RAW converter?
http://forums.dpreview.com/forums/read.asp...essage=30275878 (http://forums.dpreview.com/forums/read.asp?forum=1014&message=30275878)
-
Hmm....that is an interesting post.
However, Borg states that the original RAW process (NX for example) should be archived with the original raw files otherwise a similar situation would develop. Sounds kinda hokey to me. I think one might be able to say...no problem...archive your DNG with whatever version of ACR is being used at the time (unless I misunderstood what he said).
I think the gist of that thread on "that other forum" is that a RAW file or a DNG contains your BASE information, whatever RAW processor you use (whether camera manufacturer or 3rd party such as Adobe) will in fact change over time as improvements are made.
Does that sound correct?
Cheers...
Todd in Chicago
http://forums.dpreview.com/forums/read.asp...essage=30275878 (http://forums.dpreview.com/forums/read.asp?forum=1014&message=30275878)
-
Yeah, but it's bullshit. It was fine in ACR when you could easily toss a copy of ACR next to your files and get back to the last rendering. In the case of lightroom you have a choice of having ALL of your files change whenever you change LR (to a version that changes the rendering) or exporting rendered copies of all your files. This kind of defeats the purpose of metadata editing. Heck if you're using ACR doesn't it defeat the purpose of smart objects, too?
-
LOL.....hehehehe.... a very slippery slope indeed!
Eric Chan indicated in that other thread that if you wanted to archive/preserve an image with the enhancements so that it DID not change over time; archive a TIFF version with the parametrics baked in.
:-)
Cheers...
Todd in Chicago
Yeah, but it's bullshit. It was fine in ACR when you could easily toss a copy of ACR next to your files and get back to the last rendering. In the case of lightroom you have a choice of having ALL of your files change whenever you change LR (to a version that changes the rendering) or exporting rendered copies of all your files. This kind of defeats the purpose of metadata editing. Heck if you're using ACR doesn't it defeat the purpose of smart objects, too?
-
Can you point to any evidence that shows that the output from a DNG file is in any way different to that from it's original RAW when both are processed in the same way with the same RAW converter?
http://forums.dpreview.com/forums/read.asp...essage=30275878 (http://forums.dpreview.com/forums/read.asp?forum=1014&message=30275878)
The original question in that form was not really relevant, i.e. this is not the real issue; nor is that post on DPreview relevant regarding the main issue.
DNG is (supposed to be) partly a format change of the original data. Although the encoding adopted by DNG in the compression is the same one adopted in CR2 files, there are small differences. The differences are greater with Nikon compressed image data; spme other formats are totally different.
The only relevant questions are,
1. if the image data remains unchanged or directly (i.e. one to one) convertible between the native format and DNG,
2. if all relevant info is converted or saved in DNG format.
Condition 1 is fulfilled mostly but not always. The exceptions are worthy dio be discussed on their own;
condition 2 is fulfilled by savng MakerNote in the DNG file.
Note, that I discounted the option of saving the entire original raw data in the DNG file, for that is plain garbage.
Back to the subject: it is NOT important, that the same result be generated from the different formats. DIfferent versions of any raw converter may yield different results from the same raw file, no matter which format.
It is important to understand, that the DNG file contains not only the "natural data" (i.e. that, what is taken from the native raw file) but its interpretation is added. Particularly, the color reproduction is an issue, which is entirely in the domain of the raw processor and can be changed. Thus there is nothing strange on the fact, that different versions of ACR or of the DNG converter generate different outputs.
Again, the main issue is, if the raw image data remains the same except for formal changes. The example I provided demonstrate, that this is not always the case. That error has been removed, but all related images converted from NEF in DNG format in that period are permanently lost if the NEF was not kept.
I repeate it: only computer illiterates would believe, that any software, no matter from which corner it comes, is guaranteed error free and one can blindly trust to throw away the original data.
-
I also recommending archiving the original raws as they came off the camera (whether these are camera-generated DNGs or non-DNGs) and have no issue with Gabor's view on this matter.
Regarding the dpreview thread linked to by Daniel, my take with Iliah's concern is that rendering changes are an issue in any parametric editing environment and is independent of DNG (and I stated this opinion in that thread). For example, suppose you process a CR2 file in your preferred raw conversion software today and adjust its controls the way you like. Then next year a new version of the software comes out with an improved color interpolation algorithm that reduces speckle artifacts. Presumably this is a Good Thing, in that if the algorithm is good, then your images simply look better. But if you go to process that CR2 again using the new version of the software, then even though you didn't touch any controls, the appearance of that CR2 has changed since you last looked at it. Same thing applies if, for instance, the raw converter "upgrades" to a 32-bit or 64-bit-per-component floating point pipeline (up from, say, 16 bits per component), well the internal math will now be done at higher precision so that results will be numerically different than what you got previously. Iliah suggests archiving the software version with the image. Well, ok, if you want to go that far, but keep in mind you'd have to archive the computer as well, and required associated hardware, since computers of the future may not run the old software. (Try running Kodak's raw converters for the DCS 460 on OS X or Vista ...)
-
For example, suppose you process a CR2 file in your preferred raw conversion software today and adjust its controls the way you like. Then next year a new version of the software comes out with an improved color interpolation algorithm that reduces speckle artifacts. Presumably this is a Good Thing...
Its a great thing! Its what makes Raw so damn useful. We didn't have that with film (keeping latent image around wasn't a good idea). Its not going to happening much if at all with rendered images. I'm all for reprocessing a hero image if indeed, it will visibly benefit the final output.
Well, ok, if you want to go that far, but keep in mind you'd have to archive the computer as well, and required associated hardware, since computers of the future may not run the old software. (Try running Kodak's raw converters for the DCS 460 on OS X or Vista ...)
That's been my experience and its not a good one. That said, being on the Mac since 1988, I've got lots of files I can no longer access (proprietary calender programs). My own fault for not updating as I go. But that data is far, far less important than image data. Its pretty easy to export a text doc from those older calendar applications and re-import into a newer product. Not so easy with a proprietary Raw file!
I have one really old Mac laptop that can boot OS9, I'm thinking I might just archive it.
-
Doesn't matter if the rendering is improved. All the work was based on a different rendering so the image might require manual intervention to actually be improved. In the case of something like a smart object that could be a lot of layers and a lot of work.
Now that I understand the lie of parametric editing I'm fine with this.
-
Thanks for sharing, Gabor. By the way, your posts are often rife with opportunity to quote out of context:
...any software, no matter from which corner it comes, is guaranteed error free...
Reminds me of Knuth for some reason: "Beware of bugs in the above code; I have only proved it correct, not tried it."
But if you go to process that CR2 again using the new version of the software, then even though you didn't touch any controls, the appearance of that CR2 has changed since you last looked at it.
Thanks for the post, Eric. The appearance changing with new versions is expected and normal. The reason I quoted that thread was because it shows that the appearance changes differently for DNG than for the original raw, contradicting Nick Rains' position that archiving the DNG would be the same.
-
Now that I understand the lie of parametric editing I'm fine with this.
Can you please clarify? In my view, parametric editing has never been about preserving rendered results (even approximately). Rather, it is largely about preserving the original raw data intact, regardless of how the software chooses to interpret it. That is, the software can continue to evolve and improve, but the original data should not change.
-
Can you please clarify? In my view, parametric editing has never been about preserving rendered results (even approximately). Rather, it is largely about preserving the original raw data intact, regardless of how the software chooses to interpret it. That is, the software can continue to evolve and improve, but the original data should not change.
Why do you need parametric editing for that?
-
"Beware of bugs in the above code; I have only proved it correct, not tried it."
I do swear that my programs are error free, but I would not bet on that.
-
The reason I quoted that thread was because it shows that the appearance changes differently for DNG than for the original raw, contradicting Nick Rains' position that archiving the DNG would be the same.
Seems to me that if you use different software to try and get the same results, you may or may not be successful. I admit to being surprised that a 3 year old DNG produced different results compared to processing the original CR2 RAW. Maybe Eric has an opinion on why this might be so.
However, just to check my facts, I just did this:
Convert a file to DNG.
Load both this DNG and it's corresponding CR2 into ACR5.2
Open both into PSCS4 having synched the settings, or left them both on default as I did.
Compare using difference blending mode.
Zero difference.
So I stand by my assertion that converting to DNG makes no difference; but evidently only using exactly the same software for both DNG and CR2. Therefore it's not the DNG format that is the 'culprit', it's the software that processes it.
-
Why do you need parametric editing for that?
You don't.
Parametric editing is not a requirement for non-destructive workflows. However, it offers several advantages over pixel-based edits. First, the edits themselves are much more compact. Hence you can work with multiple renderings of a given raw file (e.g., a color version, a neutral B&W version, tinted B&W version, versions with different color profiles, etc.) with little overhead; syncing edits (e.g., white balance, a given split-tone setting, hue twist, etc.) across a large number of images is also very efficient.
Another advantage, in my opinion, is that the image processing application (e.g., a raw converter) can automatically arrange the image processing operations in the most effective and/or efficient order, regardless of the order in which you make the edits. This all simply falls out of the idea that the application is working with a set of instructions/parameters. This has significant implications for users because it relieves users from accidentally being, for instance, in the wrong color space when performing certain types of operations (often you want to be in a linear RGB space for certain types of operations, in a non-linear opponent space for other types of operations, etc.). Having the user perform these conversions back-and-forth in a direct pixel-based representation, whether manually or automatically via an action/macro, is quite error-prone and can have many unwanted side effects.
-
Follow up.
I cannot reproduce the results of that DPReview thread.
I dug back into my archives, I archive the original RAW to DVD and it's corresponding DNG to external HD.
OK, DNG created back in 2006, opened along with it's original CRW file at exact same settings in ACR5.2 using the old ACR2.4 profiles.
Zero difference, completely black difference blend.
So, by my own experience and testing, DNG and RAW are entirely equivalent, even from old D60 files, as long you use the same RAW processor. AFAIC that's the end of it, if there is some subtle technical issue lurking here then it does not show up in the real world and I'm not really interested.
-
I stand by my assertion that converting to DNG makes no difference; but evidently only using exactly the same software for both DNG and CR2. Therefore it's not the DNG format that is the 'culprit', it's the software that processes it.
This is not, what you have proven. You have shown, that ACR interprets the native raw file identically as the DNG file. Big deal.
What about trying this with other raw converters? Some of them process (or processed in the past) the raw file "normally", i.e. as the raw processor's designers/programmers decided, while processing the DNG file as Adobe decided. This is relevant regarding the color reproduction.
However, again, in my eyes this is not an argument for or against DNG. This is the question of interpretation: if you select the path of DNG created by Adobe's DNG converter, you decide for Adobe's interpretation, which is the same as if you convert the native raw file directly with ACR. If you process the raw files (native and DNG) with another raw converter, the results may or may not be identical.
I remember there was a time, when Aperture processed DNG files, but only those created from natively supported raw files; this indicated, thet Aperture supported the format but did not give a damn for Adobe's interpretation of that raw data.
-
This is not, what you have proven. You have shown, that ACR interprets the native raw file identically as the DNG file. Big deal.
What about trying this with other raw converters? Some of them process (or processed in the past) the raw file "normally", i.e. as the raw processor's designers/programmers decided, while processing the DNG file as Adobe decided. This is relevant regarding the color reproduction.
However, again, in my eyes this is not an argument for or against DNG. This is the question of interpretation: if you select the path of DNG created by Adobe's DNG converter, you decide for Adobe's interpretation, which is the same as if you convert the native raw file directly with ACR. If you process the raw files (native and DNG) with another raw converter, the results may or may not be identical.
.
I have demonstrated what others have been disputing, that DNGs are not equivalent to the original RAW. They are, as expected.
As to processing in different RAW converters, well of course the results are going to be different. What's wrong with that? It's quite common to use different converters to process different files, some have strengths in different areas - that's why people like to keep their original RAWs, just in case they need to use, say, DPP for some subtle edit. In this case DNG is not the appropriate format, I've no problem with that.
I happen to use ACR almost exclusively- it's convenient to do so - and so DNG is an appropriate format to archive into - for me.
-
papa, this is a common misconception. Most of the makers' "costly device characterisation" is stored in maker notes. Whether a maker note is stored in a CR2 container, a NEF container, a DNG container, or a TIFF/EP container does not matter; the raw conversion software can read the same (proprietary) information regardless of the container. In fact, when the Adobe DNG Converter creates a DNG from a non-DNG raw file, it copies over the maker note, even though it doesn't most of the contents of that note. Cameras today that shoot DNG files directly also store characterization data in the DNG maker note; only the makers of those cameras understanding the meaning of those notes. Those makers have "given up" nothing.
Different makers will take different approaches to a common raw format.
Example 1: Some makers like Ricoh, Leica, and Casio are opting for the DNG-only route, i.e., their cameras do not offer a non-DNG raw mode. They are fine with putting as-shot white balance (i.e., maker neutral), embedded color profiles, etc. in public tags. Whatever they feel uncomfortable with sharing goes into the DNG maker note.
Example 2: Other makers like Pentax are opting for two raw modes, one that is DNG, one that is non-DNG (for Pentax, that would be PEF).
Eric,
The question is whether DNG allows the companies hide their proprietary color LUT and yet allow the user get the same colors using any DNG RAW converter.
At the end, that's what we want, the camera colors at any RAW converter.
-
The question is whether DNG allows the companies hide their proprietary color LUT and yet allow the user get the same colors using any DNG RAW converter.
At the end, that's what we want, the camera colors at any RAW converter.
That depends on what you mean by "hide" and "proprietary" in this context.
Camera vendors can certainly store color profiles (with matrices and complex LUTs, curves, etc.) in DNG images. However, as the format of these color profiles is described in the DNG spec, and both the spec and SDK are free to download and use, a developer/engineer could easily "peek" inside the file and grab the profile contents to see what's going on.
On the other hand, Adobe has provided so-called Camera Matching profiles for several years for several cameras. These profiles are stored in the same format and also produce "camera colors". Developers/engineers can poke inside these profiles in exactly the same way.
So, from my point of view, none of this stuff is particularly hidden to begin with.
-
Eric,
The question is whether DNG allows the companies hide their proprietary color LUT and yet allow the user get the same colors using any DNG RAW converter.
At the end, that's what we want, the camera colors at any RAW converter.
Not really. Actually really not. I want my colors and not those of nikon canon or whatever make.
When the endresult is there the camera used is of no importance.
-
Eric,
You get very close to the manufactures color.
Yet not in every situation and set of parameters.
Let's say I'm Nikon.
I want to give my customers, owners of Nikon cameras, the option to have the same colors they get using Capture NX in ACR / Lightroom.
Yet I don't want the matrices / LUT's to be exposed.
Just be hidden as they are hidden in my own NEF files.
Does DNG have that capability?
As far as I know, it doesn't, that's the main barrier...
-
Just be hidden as they are hidden in my own NEF files.
Nikon software creates .icc profiles during its work and those profiles are being extracted during that process by people to be used by their own purposes... the only way to have undocumented info in a documented file format is to store all that in a special tag intended for storing undocumented manufacturer's info and that will be a major PR blow to DNG concept if Adobe starts to use that info itself... alternatively Adobe might update a DNG spec allowing encryption of information in documented tags (so that you can have encrypted content in documented fields) and then Nikon can give certain software developers keys to decrypt, but as history teaches that is a futile effort... the keys will be stolen
-
As far as I know, it doesn't, that's the main barrier...
the main barrier to what ? it is possible to imitate color rendering quite close w/o asking Nikon... a good software developer (certainly not that every single one or every other) can create a synthetic raw file (or several) to cover with all possible (or reasonably necessary) combinations of R G1 B G2 values, convert it using Nikon software, get resulting tiff, then create a .dcp profile with LUTs imitating the same conversion (might be a huge one though) from that raw to Adobe's internal working color space (certainly adjusted to the fact that Nikon is not generating tiffs in the same colorspace as Adobe uses internally and that Adobe deals with demosaicked RGB data, not RGBG raw values for their color transforms)... something like this... "VitNovak" @ http://forums.adobe.com/community/cameraraw did something like this for his Canon's - search for his postings.
-
Let's say I'm Nikon.
I want to give my customers, owners of Nikon cameras, the option to have the same colors they get using Capture NX in ACR / Lightroom.
Yet I don't want the matrices / LUT's to be exposed.
Just be hidden as they are hidden in my own NEF files.
Does DNG have that capability?
As far as I know, it doesn't, that's the main barrier...
As Eric mentioned in post #14...
...in my opinion the real benefit of DNG (or at least some standard, documented raw format, whether it be DNG, TIFF/EP, or something else) is when they are generated directly by the camera.
In the case of DNG, for instance, an in-camera DNG can contain an image checksum, which DNG readers can use to check for image integrity after images have been read off the card (or at any other time). As another example, the makers can embed 1 or more color profiles to be used for portable default color rendering across DNG readers; Casio and Pentax have done this quite effectively, without having to divulge what recipes they used to cook up those profiles in their labs.
Eric,
You get very close to the manufactures color.
Yet not in every situation and set of parameters.
That's because you have two preview renditions to compare against with each reflecting how their rendering recipe deals with the sensor's non-linear behavior to any given scene captured. You're asking too much of both the device and software with regard to consistency between the two.
If you had only the Nikon software generated preview to assess color consistency shot to shot then you wouldn't notice the inconsistencies.