Raw photo analysis software

[Chris Pollock]

When I'm shooting static subjects, I usually do at least a 3 shot bracket and keep the best exposure. The best exposure, of course, is the longest one that doesn't clip important highlights.

The problem is that determining which parts of a photo have clipped in the raw data isn't all that easy. Adobe Camera Raw shows highlight clipping in the processed image, which may or may not correspond to clipping in the raw data. With the 2010 process in ACR I could quickly reject obviously over or under exposed shots, then reduce the exposure to look for lost highlight detail in more marginal shots. The 2012 process, with its automatic highlight recovery, makes the process of identifying the best exposure harder.

What I'd really like is a program that can display a rough rendering of the raw data, and show exactly which areas have clipped in each colour channel. A raw histogram would also be nice. Does anyone know of a program like this? I tried Google, but it didn't turn up anything.

If it doesn't already exist, would many of you find such a program useful? I'm tempted to try writing it myself using Adobe's DNG API. It doesn't require any complicated algorithms - quite the opposite in fact.

[francois]

When I'm shooting static subjects, I usually do at least a 3 shot bracket and keep the best exposure. The best exposure, of course, is the longest one that doesn't clip important highlights.

The problem is that determining which parts of a photo have clipped in the raw data isn't all that easy. Adobe Camera Raw shows highlight clipping in the processed image, which may or may not correspond to clipping in the raw data. With the 2010 process in ACR I could quickly reject obviously over or under exposed shots, then reduce the exposure to look for lost highlight detail in more marginal shots. The 2012 process, with its automatic highlight recovery, makes the process of identifying the best exposure harder.

What I'd really like is a program that can display a rough rendering of the raw data, and show exactly which areas have clipped in each colour channel. A raw histogram would also be nice. Does anyone know of a program like this? I tried Google, but it didn't turn up anything.

If it doesn't already exist, would many of you find such a program useful? I'm tempted to try writing it myself using Adobe's DNG API. It doesn't require any complicated algorithms - quite the opposite in fact.

Have a look at RawDigger and let us know what you think.

[bjanes]

Have a look at RawDigger and let us know what you think.

Rawdigger is my preferred program for looking at raw files, but currently it does not have a display for clipped channels. The raw histogram will show clipped channels, but the display will not indicate where in the image that the clipping is taking place. Rawnalize also shows a raw histogram and it also has a clipping indicator. Unfortunately, the developer has passed away and the program is not being updated for new cameras. However, it does operate with DNG files.

Here is the clipping indicator display showing clipped green channels in a Stouffer wedge. The display shows nonclipped channels (red and blue), so the display is magenta.

Regards,

Bill

[Chris Pollock]

Have a look at RawDigger and let us know what you think.

Thanks for the link. I've installed it, had a brief play around, and read a little bit of the information on the web site.

I've already learned one interesting fact. Previously I'd naively assumed that the digital values would reach the maxiumum number that could be represented by the number of bits per channel. e.g. For a 14 bit camera like my 5D Mark II, I'd assumed that clipped channels would have a value of 16383. Looking at a grossly overexposed image (from an HDR sequence) I see that the values actually top out at 15760 for the red and blue channels, and 15761 for the green channel.

Interestingly, selecting a large area that was totally blown in all 3 channels shows that there is absolutely no variation in the blown pixel data - the minimum and maximum values for all 3 channels were exactly the same. I can only assume that the AD converter maxes out at a number slightly less than the number that can be stored by the number of bits. (Presumably it's not analogue saturation, or there should be a slight difference between pixels.) This would slightly complicate the task of finding blown highlights, but all that's necessary is to find the maximum value for each camera, which should be an easy task.

I have to say that RawDigger doesn't look like it would suit my purpose however. It seems to be designed for a deep analysis of files, not for working quickly. It requires you to open the files one at a time, which would be far too slow. It also lacks a way to display blown highlights, as noted in the post above.

I need something similar to ACR that lets you quickly scroll through a list of files and scan each one for blown highlights.

[b2martin]

Why not continue to use the PV2010 process to identify clipped images and then switch to PV2012 to process the images you keep?  You can select which process to use in Camera RAW (PV2003, PV2010, or PV2012).

[bjanes]

Why not continue to use the PV2010 process to identify clipped images and then switch to PV2012 to process the images you keep?  You can select which process to use in Camera RAW (PV2003, PV2010, or PV2012).

PV2010 with a linear tone curve does a decent job reflecting the raw file, but only after one applies the proper BaselineExposure offset. For the Nikon D3, this is +0.5 EV, so one has to apply -0.5 EV exposure in ACR. The offset varies among cameras and can be found by examining the exif file of the DNG. Also, clipping of the blue and red channels can occur during white balance, since the WB multipliers for these channels are greater than unity. For example, this can occur in shots of a saturated flower. Here is an example of a yellow flower. The red channel is short of clipping, but when the WB multiplier for the red channel of about 1.7 is applied, the red channel will be clipped. Unless you are using UniWB, this will be difficult to detect in ACR.



Regards,

Bill

[Bart_van_der_Wolf]

Also, clipping of the blue and red channels can occur during white balance, since the WB multipliers for these channels are greater than unity. For example, this can occur in shots of a saturated flower. Here is an example of a yellow flower. The red channel is short of clipping, but when the WB multiplier for the red channel of about 1.7 is applied, the red channel will be clipped. Unless you are using UniWB, this will be difficult to detect in ACR.

Hi Bill,

Since we're in the "Other Raw Converters" forum, I wonder how 'RawTherapee' (which BTW also has a Raw histogram, with vertical scale adjustment) would handle that, since AFAIK it processes data in floating point which should prevent clipping upon rounding to 16-bit integers. I'm just not sure if the WB+demosaicing is already done in Floating point, I'll have to dig up some images of tulips ...

Cheers,
Bart

[deejjjaaaa]

I have to say that RawDigger doesn't look like it would suit my purpose however. It seems to be designed for a deep analysis of files, not for working quickly. It requires you to open the files one at a time, which would be far too slow. It also lacks a way to display blown highlights, as noted in the post above.

that is on TODO list = http://forums.dpreview.com/forums/readflat.asp?forum=1041&message=41385706

[deejjjaaaa]

Hi Bill,

Since we're in the "Other Raw Converters" forum, I wonder how 'RawTherapee' (which BTW also has a Raw histogram, with vertical scale adjustment) would handle that, since AFAIK it processes data in floating point which should prevent clipping upon rounding to 16-bit integers. I'm just not sure if the WB+demosaicing is already done in Floating point, I'll have to dig up some images of tulips ...

Cheers,
Bart

RPP ( www.raw-photo-processor.com ) might do that... albeit it is a  not a superfast... you can select a raw histogram mode, settings illustrated below :





and go through files w/ clipping indicator checkbox on

[Chris Pollock]

Why not continue to use the PV2010 process to identify clipped images and then switch to PV2012 to process the images you keep?  You can select which process to use in Camera RAW (PV2003, PV2010, or PV2012).

I thought of that, and may well do so, but it's still unreliable compared to looking at the raw data.

[ejmartin]

Hi Bill,

Since we're in the "Other Raw Converters" forum, I wonder how 'RawTherapee' (which BTW also has a Raw histogram, with vertical scale adjustment) would handle that, since AFAIK it processes data in floating point which should prevent clipping upon rounding to 16-bit integers. I'm just not sure if the WB+demosaicing is already done in Floating point, I'll have to dig up some images of tulips ...

Cheers,
Bart

Yes, RawTherapee carries out all operations in float, until the end of the imaging pipeline, when it clips and converts to 8-bit or 16-bit for output.  There are a few exceptions, if you start turning on various image processing tools whose operation requires data be clipped somewhere along the way.  To avoid that, use the 'neutral' processing profile option.

[mouse]

PV2010 with a linear tone curve does a decent job reflecting the raw file, but only after one applies the proper BaselineExposure offset. For the Nikon D3, this is +0.5 EV, so one has to apply -0.5 EV exposure in ACR. The offset varies among cameras and can be found by examining the exif file of the DNG.
Regards,
Bill

Bill-
Can you tell me where to look in the DNG EXIF file to determine the offset?
Thanks.

[bjanes]

Bill-
Can you tell me where to look in the DNG EXIF file to determine the offset?
Thanks.

The easiest way to find the BaselineExposure offset is to use the Rawdigger exif utility and copy the data to the clipboard and then paste into your word processor (I used Microsoft Word) and search for "Baseline". It is about a fifth of the way down from the beginning of the file.

Regards,

Bill

[mouse]

Bill-

Got it
Many thanks

[bjanes]

Hi Bill,

Since we're in the "Other Raw Converters" forum, I wonder how 'RawTherapee' (which BTW also has a Raw histogram, with vertical scale adjustment) would handle that, since AFAIK it processes data in floating point which should prevent clipping upon rounding to 16-bit integers. I'm just not sure if the WB+demosaicing is already done in Floating point, I'll have to dig up some images of tulips ...

Cheers,
Bart

Bart,

I spent some time working with RawTherapee and was impressed with its power and flexibility. It is a techie's dream. The raw histogram easily shows clipping in the raw file. Here is the raw histogram of the file I previously posted with Rawnalize. The red channel is well short of clipping, but it becomes clipped when rendering into a white balanced space. It is necessary to use ProPhotoRGB to avoid saturation clipping. The screen capture is from a NEC wide gamut monitor and it is uploaded as AdobeRGB and it may not display well on some systems. It would be helpful if ACR also provided a raw histogram.



For interested readers, here is the file rendered into ProPhotoRGB with no adjustments. The red is strongly clipped.



To avoid clipping with white balance, one can use WB multipliers of less than unity as Guillermo explains in his essay on DCRaw where one uses the -H 1 switch, which forces one multiplier to be 1.0 and ensures that the others are less than 1. As far as I know, one can obtain similar results by using negative exposure as shown below. Unfortunately the resulting image appears somewhat dull as compared to the other images.



Regards,

Bill

[ejmartin]

Bill, you might use the 'raw white point' slider in the Raw tab.  When there are no clipped channels, it is safe to use a value less than one.  One use of the tool is to set the raw white point to a higher value in situations such as the one you have with this image.  I am a little surprised that the histogram is so peaked in the red channel when one dials down the exposure in the 'Exposure' tab instead of adjusting the raw white point; that looks like a bug, they should be equivalent apart from possible issues when the input profile uses a LUT that runs out of range.  Would you be willing to share the raw file?  I believe you have my email.

[Bart_van_der_Wolf]

Bart,

I spent some time working with RawTherapee and was impressed with its power and flexibility. It is a techie's dream. The raw histogram easily shows clipping in the raw file.

Hi Bill,

It is indeed a techie's dream. At the same time that vast array of possibilities poses a user interface challenge, and therefore it may look a bit intimidating to newbies. But having so much control over the whole conversion process is IMHO a blessing for those who know what they're doing.

Cheers,
Bart

[bjanes]

Bill, you might use the 'raw white point' slider in the Raw tab.  When there are no clipped channels, it is safe to use a value less than one.  One use of the tool is to set the raw white point to a higher value in situations such as the one you have with this image.  I am a little surprised that the histogram is so peaked in the red channel when one dials down the exposure in the 'Exposure' tab instead of adjusting the raw white point; that looks like a bug, they should be equivalent apart from possible issues when the input profile uses a LUT that runs out of range.  Would you be willing to share the raw file?  I believe you have my email.

Emil,

Thanks for the tip. The spike in the red channel does not appear when one uses the raw white point control. I misplaced your email so I uploaded the file to usendit. The Image has little artistic value so I am not concerned about anyone having access to it and they can use it to explore the features of the program. Thank you very much for your work in this excellent piece of software.

Regards,

Bill

https://www.yousendit.com/download/QlVpL0dPYSs1bmlKUmNUQw