Luminous Landscape Forum
Raw & Post Processing, Printing => Digital Image Processing => Topic started by: Ellis Vener on February 28, 2014, 12:49:36 pm
-
The link is here: http://www.youtube.com/watch?v=_fikTm8XIt4
Andrew discusses:
-What they are and how they differ from ICC profiles.
-Why you don't need to make more than two or possibly three per camera unless you are shooting under weird spectrum lighting like fluorescent or mercury vapor.
-How to create them
-That they are useful for at least one non-Adobe raw processing program.
-How to automate applying them during import.
I've been making and using DNG profiles for a few years now and find it really does make a difference in color rendering and I learned a couple of things.
One thing he doesn't explicitly mention is that your choice of a camera profile is just an option to get the results you want. That inherent in the power and flexibility of using your camera's raw format
-
Andrew discusses:
-What they are and how they differ from ICC profiles.
did he still say that you need to make a tiff first ?
-
did he still say that you need to make a tiff first ?
NO. He did say that the raw image of the target must first be converted to DNG in order to construct the profile. Thereafter the profile can be applied to any raw format image supported by your raw converter.
-
did he still say that you need to make a tiff first ?
NO. He did say that the raw image of the target must first be converted to DNG in order to construct the profile. Thereafter the profile can be applied to any raw format image supported by your raw converter.
YES, he says that for an ICC profile you need to convert it to a rendered TIFF, to be fed into a profiling software.
That's why he states that the ICC profile is output(!?) referred, and that the DNG profile is more Raw (Rawer) and scene referred.
Some beg to differ with those qualifications. ICC profiles are used for scanners and Digital Still camera's, very much at the input/capture side of the imaging pipeline/equation and therefore oblivious of the output side. They are pretty much scene referred ICC profiles, and can be profiled without the need to produce Rendered TIFF files if necessary. A program like Rawdigger can produce CGATS output files (http://www.rawdigger.com/usermanual/sampleswindow/save) from the Raw sensor data, as input for Profiling software.
I have yet to test how well that works, because I'm happy with the current ICC profile for my Scanner and DSLR, so I'm not in any particular hurry.
Cheers,
Bart
-
NO. He did say that the raw image of the target must first be converted to DNG in order to construct the profile. Thereafter the profile can be applied to any raw format image supported by your raw converter.
YES, he says that for an ICC profile you need to convert it to a rendered TIFF, to be fed into a profiling software.
That's why he states that the ICC profile is output(!?) referred, and that the DNG profile is more Raw (Rawer) and scene referred.
Some beg to differ with those qualifications. ICC profiles are used for scanners and Digital Still camera's, very much at the input/capture side of the imaging pipeline/equation and therefore oblivious of the output side. They are pretty much scene referred ICC profiles, and can be profiled without the need to produce Rendered TIFF files if necessary. A program like Rawdigger can produce CGATS output files (http://www.rawdigger.com/usermanual/sampleswindow/save) from the Raw sensor data, as input for Profiling software.
I have yet to test how well that works, because I'm happy with the current ICC profile for my Scanner and DSLR, so I'm not in any particular hurry.
Cheers,
Bart
Rodney is still in denial, how sad...
-
Rodney is still in denial, how sad...
Looking at the page for Rawdigger it appears that for it to work the raw data will have to be converted or processed to produce a file containing the values of the raw file in a format Rawdigger can use. The program will also costs $89.99
Have you done an apples to apples comparison of the results from the two approaches that you are willing to share with us that proves justifies the cost and time involved in learning and using Rawdigger and that it will produce a markedly better result than DNG profiling? As someone who both truly does want to find the technical limits of what my photography equipment can produce but isn't interested in "how many angels can dance on the head of a pin" arguments, I look forward to the seeing the comparison.
-
Looking at the page for Rawdigger it appears that for it to work the raw data will have to be converted or processed to produce a file containing the values of the raw file in a format Rawdigger can use.
RawDigger opens raw file directly, no need in any conversion.
-
Quote from: Vladimirovich on February 28, 2014, 10:46:23 AM
did he still say that you need to make a tiff first ?
NO. He did say that the raw image of the target must first be converted to DNG in order to construct the profile. Thereafter the profile can be applied to any raw format image supported by your raw converter.
Since some have not bothered to view Andrew's video, let me clarify my answer:
He did not say that you need to first make a tiff in order to create a DNG (dcp) camera profile. As to whether DNG profiles are to be preferred to icc profiles; I have no dog in that fight.
-
Since some have not bothered to view Andrew's video, let me clarify my answer:
He did not say that you need to first make a tiff in order to create a DNG (dcp) camera profile. As to whether DNG profiles are to be preferred to icc profiles; I have no dog in that fight.
Hi,
Poster 'Vladimirovich' asked the question with regards to ICC profiles, and they do not strictly need a rendered TIFF, nor are they output referred profiles..., by common definitions. Andrew, who is otherwise known to provide good information (as a consultant should), somehow and for some reason only known to himself (judging from his posts here on LuLa) seems (!) to have decided to promote Adobe's DNG initiative, instead of keeping an open mind towards other market participants. It can't be just his personal situation that dictates that, or can it?
That's all fine though, if full disclosure is available, and truthful information is given, which 'appears' not to be the case. The responses have become more political, in the sense that one doesn't downright lie but just doesn't tell the complete (only selective) truth either. Hence the questions (to assert the veracity of his comments), not because people have watched the video or not. He should be pleased with the fact that people at least are trying to verify the value of his opinion, rather than simply label it as biased and irrelevant.
So, currently, there are two options on the table. Either Andrew didn't know that one can convert Raw data, before even demosaicing, to CGATS data for profiling (which would be surprising for a consultant of his reputation), or he deliberately left that aspect out of the supplied information (if so, why?) ...
I hope Andrew joins in the discussion, because I'd rather discuss matters with, than about, people. I value his information about color management, and would like to continue doing that.
So Andrew (if he's even reading this), what's up?
Kind regards,
Bart
-
before even demosaicing
demosaicking still leaves the data as "scene referred" "color"-wise ("color" in quotes because whatever post-demosaic RGB numbers are - they are not yet coordinates in any proper color space)... Adobe is using just matrices to get into the first PCS (whatever LUTs are in DCP container - those are not operating on scene referred data, but on the data that was already color transformed into cieXYZ D50 with purely matrix operations) - so how is it better vs raw converters that are using icc containers and some hardcoded logic is not very clear to me... now Adobe boast an abilitity to have data for dual illuminants in one DCP container (and that part is missed in non DCP implementations - but technically nothing prevents non DCP implementations to use 2 icc containers like Adobe use dual illuminant profiles where everything is stored in 1 dcp container, surely that is nice for Adobe to do this)... now somebody with a greater knowledge please correct what I wrote here if I was wrong.
-
Hi,
DNG profiles pretty much define the colour handling in the raw conversion process. So the format is documented and well defined.
ICC is intended to handle conversion between colour spaces, as far as I can see, but it doesn't specify the raw conversion process.
There have been thread here: http://www.luminous-landscape.com/forum/index.php?topic=82891.0 where amongst others an engineer working at Phase One (Esben HR) and who is said to maintain the processing pipeline clearly says that an amount of processing is done before applying an ICC profile.
As far as I understand his postings, the early stages of raw processing convert the image into a specific colour space, and the ICC profiles are applied to an image in that colour space:
"You will need to process the image in Capture One first, and select "Embedded" color profile. This will apply all the pre-processing steps, including non-linear curves, to the image, but the ICC profile will not be applied (it is embedded in the file). You want to create a color profile that replaces the embedded ICC, so the profiling tool needs to ignore the embedded ICC profile.
An image with an embedded profile will also contain the now-standard TIFF tag ProfileToneCurve (to go back to linear space). In theory, this should make it a much easier job for a profile creator, but they must obviously be aware of that fact. It could even require a different set of algorithms, depending on the exact way the software builds the profile.
It does require some fiddling to get this to work with tools I am aware of. X-Rite did a good job of making the process easy to apply for the supported Adobe products; it is not a simple thing to do from an engineering point of view. The quality of the resulting profiles are, I suppose, a subjective matter."
Although I have limited knowledge of the colour conversion process, that approach makes sense to. QPCard had some technical info indication that their ICC profile was not applicable to C1 and also stated that the reason was the need to go over TIFF. They say the next version which they have in beta test will support tiff reading and modifications to be able to support Capture One. This is entirely consistent with Esben's writing.
That just says it is a bit complex to generate ICC profiles for C1. It doesn't say it is impossible to use ICC profiles for conversion all the way, but I guess that you need to know what is the starting point.
Best regards
Erik
Hi,
Poster 'Vladimirovich' asked the question with regards to ICC profiles, and they do not strictly need a rendered TIFF, nor are they output referred profiles..., by common definitions. Andrew, who is otherwise known to provide good information (as a consultant should), somehow and for some reason only known to himself (judging from his posts here on LuLa) seems (!) to have decided to promote Adobe's DNG initiative, instead of keeping an open mind towards other market participants. It can't be just his personal situation that dictates that, or can it?
That's all fine though, if full disclosure is available, and truthful information is given, which 'appears' not to be the case. The responses have become more political, in the sense that one doesn't downright lie but just doesn't tell the complete (only selective) truth either. Hence the questions (to assert the veracity of his comments), not because people have watched the video or not. He should be pleased with the fact that people at least are trying to verify the value of his opinion, rather than simply label it as biased and irrelevant.
So, currently, there are two options on the table. Either Andrew didn't know that one can convert Raw data, before even demosaicing, to CGATS data for profiling (which would be surprising for a consultant of his reputation), or he deliberately left that aspect out of the supplied information (if so, why?) ...
I hope Andrew joins in the discussion, because I'd rather discuss matters with, than about, people. I value his information about color management, and would like to continue doing that.
So Andrew (if he's even reading this), what's up?
Kind regards,
Bart
-
Hi,
I love raw digger, because it helps me to find out how I expose. It is a great tool for learning. No doubt it is useful for generating profiles, I feel that with profiles there is a great chance of messing up things. Also, if I use DNG Profile Editor it will just adjust the base profile, i don't now about ColorChecker Passport, but modifying an existing base profile makes a lot of sense compared to making a profile from just a few patches.
Best regards
Erik
Looking at the page for Rawdigger it appears that for it to work the raw data will have to be converted or processed to produce a file containing the values of the raw file in a format Rawdigger can use. The program will also costs $89.99
Have you done an apples to apples comparison of the results from the two approaches that you are willing to share with us that proves justifies the cost and time involved in learning and using Rawdigger and that it will produce a markedly better result than DNG profiling? As someone who both truly does want to find the technical limits of what my photography equipment can produce but isn't interested in "how many angels can dance on the head of a pin" arguments, I look forward to the seeing the comparison.
-
Hi,
I love raw digger, because it helps me to find out how I expose. It is a great tool for learning. No doubt it is useful for generating profiles, I feel that with profiles there is a great chance of messing up things. Also, if I use DNG Profile Editor it will just adjust the base profile, i don't now about ColorChecker Passport, but modifying an existing base profile makes a lot of sense compared to making a profile from just a few patches.
Best regards
Erik
The XRite ColorChecker Passport has three targets. A standard 24-patch XRite ColorChecker - useful for profiling and as a general reference; a neutral gray target for setting in camera white balance; and a target that has both gray scale patches and two sets of tinted biasing" gray patches, one for landscape and one for portrait work. I almost never use the third target, but it is handy as a short cut in some situations.
You are correct in saying that the "(Adobe) DNG Profile Editor it will just adjust the base profile." That is what the CCPP software does as well. Adobe's software allows for many more adjustments than Xrite's does and I when doing very technical work like copying art or anything where subject coloration rendering is extremely important (think commercial product colors like Coca-Cola red or Pepsi blue) it is very useful at getting the raw image processed very close to accurate color so that changes in post-processing are minimized. Different colors of gold are particularly difficult to render.
My perspective is that of a working professional photographer who doesn't have a full time or even part time colorist/post processor on staff. I find the DNG profiling solutions to be immensely helpful as I can apply the profile to to an entire shoot and if I shoot with multiple cameras, I use them to minimize the differences not only between different makes and models but also between individual cameras of the same model. While creating a profile with Rawdigger may enable a technically more accurate profile (I am still waiting for proof) is it a workable efficient tool for dealing an entire take or is it better for none-off situations?
-
There have been thread here: http://www.luminous-landscape.com/forum/index.php?topic=82891.0 where amongst others an engineer working at Phase One (Esben HR) and who is said to maintain the processing pipeline clearly says that an amount of processing is done before applying an ICC profile.
and so does Adobe... and Esben HR escaped w/o answering the direct questions... we can repeat the question here again - Does C1 do any color transforms (per channel black level subtractions, correcting for stitched sensor parts, etc, etc do not count as a color transform and demosaick does not count either) from scene referred coordinates into any color space before starting applying the transforms guided by the data in their OEM icc containers or not.
-
QPCard had some technical info
QPCard has the following factual error here - http://www.qpcard.com/en_b2c/dcp_icc_profile - "...Then the Matrix + LUT corrections takes care of the [scene referred] color and white balance, and translates it into a specified PCS (profile connection space) which in the case of Adobe Camera Raw is ProPhoto RGB... ", they kind of forgot that matrix part alone does the color transform into cieXYZ D50 and after that everything (all LUTs 3D, 2.5D, 1D whatever) is non scene referred realm... just an example about how precise they are.
-
While creating a profile with Rawdigger may enable a technically more accurate profile (I am still waiting for proof) is it a workable efficient tool for dealing an entire take or is it better for none-off situations?
that's not the point, the point is/was that DCP profiles allow for something better in principle... and one of the statements made by Rodney many times here was that _all_ tools existing to build "icc camera profiles" can't work with scene referred data... now even a tiff file that is just a container for demosaicked data that was not actually color transformed into anything else still has scened referred data, per channel manipulations and demosaick do not make the data output referred.
-
Hi,
I think that Esben said that he could not talk about their processing pipeline without an NDA, but my interpretation of his writing is "yes" and that happens also make a lot of sense.
By the way, you forgot about white balance…
Best regards
Erik
and so does Adobe... and Esben HR escaped w/o answering the direct questions... we can repeat the question here again - Does C1 do any color transforms (per channel black level subtractions, correcting for stitched sensor parts, etc, etc do not count as a color transform and demosaick does not count either) from scene referred coordinates into any color space before starting applying the transforms guided by the data in their OEM icc containers or not.
-
What bothers me the most is that you can only use the 24 patch card for a DNG profile but I can make ICC profiles with the Digital SG chart which has far better ability to deal with subtle colour transitions.
-
Hi,
I agree that Andrews presentation is good. With DCP profiles we have a couple of tools that make a decent job of profiling. No doubt, better tools may exist but they may also need better understanding of what you need to achieve. Using those tools in Lightroom helped me save my bacon on my P45+ with minimal effort.
I could have used tools like Argyll, but than I may need better understanding of colour management and it's trap doors.
Best regards
Erik
The link is here: http://www.youtube.com/watch?v=_fikTm8XIt4
Andrew discusses:
-What they are and how they differ from ICC profiles.
-Why you don't need to make more than two or possibly three per camera unless you are shooting under weird spectrum lighting like fluorescent or mercury vapor.
-How to create them
-That they are useful for at least one non-Adobe raw processing program.
-How to automate applying them during import.
I've been making and using DNG profiles for a few years now and find it really does make a difference in color rendering and I learned a couple of things.
One thing he doesn't explicitly mention is that your choice of a camera profile is just an option to get the results you want. That inherent in the power and flexibility of using your camera's raw format
-
What bothers me the most is that you can only use the 24 patch card for a DNG profile but I can make ICC profiles with the Digital SG chart which has far better ability to deal with subtle colour transitions.
The SG is quite expensive. Is there an example of the difference in quality of profiling from a standard 24 patch to the SG?
-
Rather than viewing the utube version, I would recommend looking at the full resolution (http://www.digitaldog.net/files/DNG%20Camera%20profile%20video.mov) tutorial on his web site.
Bill
-
You may be interested to see for yourself, Imaging Resource are publishing raw files of shots that contain both targets. They call those Multi Target. See, for example, here http://www.imaging-resource.com/PRODS/M9/M9THMB.HTM
-
I think that Esben said that he could not talk about their processing pipeline without an NDA, but my interpretation of his writing is "yes" and that happens also make a lot of sense.
I think that it was simply his way out ;) and no, it does not make any sense (see Iliah's matrix profile for C1 that I referred in another topic)
By the way, you forgot about white balance…
per channel WB is not a color transform - it leaves the data in the original "space"...
-
What bothers me the most is that you can only use the 24 patch card for a DNG profile but I can make ICC profiles with the Digital SG chart which has far better ability to deal with subtle colour transitions.
that's because Adobe gives you a prebuilt base profile to rest upon (when you use colorchecker and Adobe PE).
-
that's because Adobe gives you a prebuilt base profile to rest upon (when you use colorchecker and Adobe PE).
Once again, I'd like to see some proof rather than just words.
-
Well, I watched the video and waited the entire time for an example of why a dng profile in LR or ACR (which I also understand to be built on top of the Adobe canned camera profiles) is any better than, say, the direct icc approach of Capture One. All I saw were comparisons between Adobe standard profiles in LR/ACR and the tweaked version as represented by the new dng profile, which may or may not be an improvement. Clearly, in at least on example shown, the dng version was visibly worse, and in others it was just different, and in yet others, not different enough to matter.
-
The SG is quite expensive. Is there an example of the difference in quality of profiling from a standard 24 patch to the SG?
We found for our repro setup, building ICC profiles for C1 (Leaf Aptus-II 8) that there was a significant difference. Which is why we bought the SG. I don't have those test files any more I'm afraid, the testing was done some 19 months ago.
-
Once again, I'd like to see some proof rather than just words.
Ellis, how about doing this : google "Eric Chan" or google "Madmanchan" (within LuLa forums or http://forums.adobe.com/community/cameraraw) and actually reading straight from the person instead of watching promotional videos from 3rd party sources ?
-
I have all my files in the native RAW format with 100% of the data. I have some that have been processed and saved as TIFF files for print labs and some saved as JPEG for Web use. Why would I want to translate all my RAW files into a proprietary Adobe format that holds me hostage to this company that has shown for the past 20 years that it is power crazy, buying up competitors and killing their products again and again and again, and now forcing its customers to rent the software month to month.
I also lose all the camera manufacturer specific settings as with Nikon and it Active D Lighting. I have had problems with the Adobe Camera Raw converter, which is reverse engineered from the output file of the cameras, where I get banding or loss of dynamic range and other problems and I resolved them every time by opening the file using the Nikon RAW converter and then saving as a TIFF.
TIFF by the way is the only non-proprietary lossless file format that exists. I can send a TIFF to any lab or graphics or print shop anywhere in the world. The same is not true for a DNG file.
-
Just to mention, DNG has an option to store the original raw file. The files will be bigger but you can always restore the original file. It will be bit for bit identical.
By the way, TIFF is also a format specified by Adobe and DNG is just a TIFF variant. My remembrance tells me TIFF was developed by Aldus and taken over by Adobe.
Best regards
Erik
I have all my files in the native RAW format with 100% of the data. I have some that have been processed and saved as TIFF files for print labs and some saved as JPEG for Web use. Why would I want to translate all my RAW files into a proprietary Adobe format that holds me hostage to this company that has shown for the past 20 years that it is power crazy, buying up competitors and killing their products again and again and again, and now forcing its customers to rent the software month to month.
I also lose all the camera manufacturer specific settings as with Nikon and it Active D Lighting. I have had problems with the Adobe Camera Raw converter, which is reverse engineered from the output file of the cameras, where I get banding or loss of dynamic range and other problems and I resolved them every time by opening the file using the Nikon RAW converter and then saving as a TIFF.
TIFF by the way is the only non-proprietary lossless file format that exists. I can send a TIFF to any lab or graphics or print shop anywhere in the world. The same is not true for a DNG file.
-
@ligthsmith: IIRC [watched the video when this thread started] you do not need to convert all files to DNG, you need one or two that the colorchecker software uses to create camera calibration settings to use in LR.
Of course, if you are of the 'Adobe's number is 666' persuasion, this point is moot
(and as far as I remember, no TIF's were mentioned at all)