Luminous Landscape Forum

Raw & Post Processing, Printing => Adobe Lightroom Q&A => Topic started by: Hans Kruse on April 22, 2012, 09:44:03 am

Title: ETTR and LR 4 (and DxO)
Post by: Hans Kruse on April 22, 2012, 09:44:03 am
In the past I felt there was a reasonable consistency between what my camera showed as overexposed by blinkies and what I saw in Lightroom. Now with PV 2012 i LR4 this consistency has totally gone! I see in a number of pictures that a full stop higher exposure will show sometimes less blinkies than the one stop lpwer exposure on the LCD screen on the camera. My camera is a Canon 1Ds mkIII and I assume that a blinky is generated if at least one channel is blown out. I also assume that an area shown as overexposed in Lightroom (by pushing the J-key) also is one where at least one channel is blown out. In other words all other areas in the photo not shown as overexposed in LR do not have any channel blown out. So now the question: How can an area that is shown is partially blown out in LR3 or PV2010 be shown as not blown out at all in PV2012?

So if what PV2012 shows as not blown out at all is correct (which I assume) then there is a really new situation with ETTR! I have started bracketing all my landscape shots and now choose the one that is ETTR in Lightroom. Quite often this gives me one stop extra.

It could seem that since I understand that the DxO measurements are based on the DxO raw converter, that the dynamic range of a given camera is not necessarily what the DxO number says.... (assuming that the DxO rawconverter most likely is more like PV2010 than PV2012 in this respect).
Title: Re: ETTR and LR 4 (and DxO)
Post by: luxborealis on April 22, 2012, 10:22:56 am
Interesting... you seem to have identified 3 different definitions of "clipped highlights": (1) the camera's definition (as shown by blinking highlights in playback - which may be the least accurate if the playback image has been processed in-camera as they often are), (2) the PV2010 definition and (3) the PC2012 definition (as shown by highlight clipping in LR). Of course, DxO will have their own definition and, no doubt, Capture One and Aperture may have different definitions again.

Rather than having to assume, it would be helpful to know what the exact definition is in each case (98%+, 99.9%+ or over 100% in one, two or three channels, or...?) Of course, the threshold value will vary with each camera raw version. This just might bring renewed requests for the option of setting our own clipping threshold value within LR which, over the years, photographers have been asking for.

Great question, Hans!
Title: Re: ETTR and LR 4 (and DxO)
Post by: Hans Kruse on April 22, 2012, 10:35:02 am
Well, actually part of it is really a question for Schewe :)
Title: Re: ETTR and LR 4 (and DxO)
Post by: Bart_van_der_Wolf on April 22, 2012, 11:23:48 am
In the past I felt there was a reasonable consistency between what my camera showed as overexposed by blinkies and what I saw in Lightroom. Now with PV 2012 i LR4 this consistency has totally gone! I see in a number of pictures that a full stop higher exposure will show sometimes less blinkies than the one stop lpwer exposure on the LCD screen on the camera. My camera is a Canon 1Ds mkIII and I assume that a blinky is generated if at least one channel is blown out. I also assume that an area shown as overexposed in Lightroom (by pushing the J-key) also is one where at least one channel is blown out. In other words all other areas in the photo not shown as overexposed in LR do not have any channel blown out. So now the question: How can an area that is shown is partially blown out in LR3 or PV2010 be shown as not blown out at all in PV2012?

Hi Hans,

All three clipping indications are based on the resulting image after rendering the Raw data. The camera bases it's blinkies on the JPEG data (so after applying your camera presets for colorspace, contrast, sharpening, WB, etc.). Lightroom apparently does so as well, because you can make the clipping warnings go away with adjustments. The main difference is between PV2010 and PV2012. PV2010 was probably quite close to the truth when using 'as-shot' settings and the other controls at default for the camera. PV2012 on the other hand will treat partially clipped data (which you can only determine with a tool like 'Rawdigger' which shows the actual Raw data) as somewhat repairable. It's unfortunately not possible to set that threshold ourselves, so we'll have to follow what Adobe determines good enough.

From their point of view it's somewhat understandable to only warn for clipping if it cannot be resolved easily, and PV2012 can do a convincing job recovering some of the really lost info, but I prefer more control and understanding of the real data that's available to work from. That's why I use bracketing in critical situations, use ETTR, and inspect the Raw data with Rawdigger. That also allows me to better interpret the camera blinkies, and make better exposures.

Quote
So if what PV2012 shows as not blown out at all is correct (which I assume) then there is a really new situation with ETTR! I have started bracketing all my landscape shots and now choose the one that is ETTR in Lightroom. Quite often this gives me one stop extra.

PV2012 can only make assumptions when it recovers lost/clipped data. That may lead to duller and less accurate highlight rendering than from a proper exposure. I would suggest to first learn to interpret the PV2012 indicators, based on Rawdigger's real assessment, rather than blindly following the PV2012 indications. You may get better conversions based on real data, even if that is a bit before PV2012 starts giving warnings.

We're all going through a learning curve here with the new PV2012.

Quote
It could seem that since I understand that the DxO measurements are based on the DxO raw converter, that the dynamic range of a given camera is not necessarily what the DxO number says.... (assuming that the DxO rawconverter most likely is more like PV2010 than PV2012 in this respect).

No, I think DxO generally gives reliable info, consistent with the underlying Raw data. I've had a similar issue with Capture One, where it's 'filmcurve' tonality suggested highlight clipping where the real Raw data still has headroom, but when switched to a linear tone curve the clipping indication became reliable, and consistent with the real data. Fortunately C1 does allow to set the threshold for the clipping indications.

Cheers,
Bart
Title: Re: ETTR and LR 4 (and DxO)
Post by: bjanes on April 22, 2012, 02:45:18 pm
In the past I felt there was a reasonable consistency between what my camera showed as overexposed by blinkies and what I saw in Lightroom. Now with PV 2012 i LR4 this consistency has totally gone! I see in a number of pictures that a full stop higher exposure will show sometimes less blinkies than the one stop lpwer exposure on the LCD screen on the camera. My camera is a Canon 1Ds mkIII and I assume that a blinky is generated if at least one channel is blown out. I also assume that an area shown as overexposed in Lightroom (by pushing the J-key) also is one where at least one channel is blown out. In other words all other areas in the photo not shown as overexposed in LR do not have any channel blown out. So now the question: How can an area that is shown is partially blown out in LR3 or PV2010 be shown as not blown out at all in PV2012?

I agree with all of what Bart posted, but would like to add a few points. According to Eric Chan, PV2012 automatically performs highlight recovery and Jeff Schewe appears to be ambiguous on this topic. As usual, tests with your own camera are in order. As an example, here are a few tests I did with the Nikon D3 by exposing a Stouffer wedge and comparing the actual raw histogram as shown by Rawnalize and Rawdigger with the camera histogram and blinking highlights as well as the ACR histogram and highlight indicator. I used the Nikon standard picture control (normal contrast).

This image demonstrates that the Nikon histogram is slightly conservative, but close enough for practical work.
(http://bjanes.smugmug.com/Photography/Exposure/D3Histograms/CameraVsRawHistogram/500020980_br84F-XL.png)

The blinking highlights are a bit more conservative.
(http://bjanes.smugmug.com/Photography/Exposure/D3Histograms/Composite/574559130_YeNHy-L.gif)

The ACR histograms in PV2010 are complicated by the +0.5 EV exposure offset used by ACR for this camera. To compensate for this offset, it is necessary to decrease the ACR exposure by 0.5 EV.

Here is PV2010 with no exposure compensation for image 05 which is slightly below clipping as shown by the Rawnalize histogram. It appears blown in ACR.
(http://bjanes.smugmug.com/Photography/Exposure/D3Histograms/i-DZb7cSC/0/XL/05PV2010Default-XL.png)

Using the -0.5 EV compensation, the image still appears slightly overexposed.
(http://bjanes.smugmug.com/Photography/Exposure/D3Histograms/i-22vGvpw/0/XL/05PV2010ExpNegHalf-XL.png)

PV2012 shows a more accurate histogram for the properly exposed image. The baseline offset seems not to be in effect.
(http://bjanes.smugmug.com/Photography/Exposure/D3Histograms/i-Rtw3qfz/0/XL/05PV2012Default-XL.png)

Image 03 has clipped green channels. The blue channel is at clipping and the red channel is just short of clipping (this is with a 5000K light box).
(http://bjanes.smugmug.com/Photography/Exposure/D3Histograms/i-sb4V6ZM/0/O/D3Stouf0003-Sel-1206-1306.png)

The PV2012 histogram exhibits minimal clipping, indicating the effect of highlight recovery.



I don't know what baseline offset PV2010 uses for your camera (this can be determined by converting to DNG and using an exif reader), but for my own camera, PV2012 shows improvement.
(http://bjanes.smugmug.com/Photography/Exposure/D3Histograms/i-DpC2QZG/0/XL/03PV2012Default-XL.png)

Regards,

Bill
Title: Re: ETTR and LR 4 (and DxO)
Post by: Schewe on April 22, 2012, 03:10:02 pm
According to Eric Chan, PV2012 automatically performs highlight recovery and Jeff Schewe appears to be ambiguous on this topic.

I guess it all depends on what you consider to be "highlight recovery"...the new image adaptive PV 2012 has an auto-ranging capability which can appear as "recovery" but I don't think it really is in a classic sense. What I think is going on is that the way the highlights are rendered in PV 2012 has a different roll off which extracts more highlight detail. Note that in a linear capture, the brightest area of the capture has a lot of data even if there is no clipping. Recurving the tone mapping to extract more usable image data is certainly what PV 2012 is doing. That's what I think people are seeing...and I don't call that "highlight recovery" which to me means extracting image detail when one or more channels are clipped. I'm pretty sure there is a small degree of classic highlight recovery going on even when all Basic panel controls are at zero which I think is what Eric is talking about and I think it's the new tone mapping that is making the biggest differences. But a minus Exposure or Highlights is clearly when the classic "highlight recovery" really kicks in and that has been substantially improved in PV 2012.

I'm still coming to grips with PV 2012 and I've been working with it in various forms since late last summer...what I think is going on is that this new tone mapping logic is really, really leading edge and traditional descriptions and terms fail to really describe what going on under the hood. I've read the post on Lightroom Journal titled Magic or Local Laplacian Filters? (http://blogs.adobe.com/lightroomjournal/2012/02/magic-or-local-laplacian-filters.html) and that stuff is too deep for me to understand...but what I will say it that PV 2012 has made a major improvement in the ability to tone map raw captures. I can't say it's "perfect" (there are a couple of types of images where tone bleeding can occur) but compared to PV 2010 it's clear that one can get a lot more image detail out of highlights and shadows. I hesitate to call this a change in the dynamic range of the image but is sure does make it easier to deal with higher contrast scenes...I'm going to be in the process of writing about this stuff in the very near future...the way I usually do that is ask some questions of Eric and Thomas, swirl it around and spit it out and see if they agree with me. I'll let ya all know how that goes :~)
Title: Re: ETTR and LR 4 (and DxO)
Post by: Schewe on April 22, 2012, 04:37:31 pm
In the past I felt there was a reasonable consistency between what my camera showed as overexposed by blinkies and what I saw in Lightroom.

If by that you mean that most all cameras' display an sRGB (or Adobe RGB) ripped preview and then apply a very, very conservative indicator for clipping (for Canon about 1 stop) then that would be generally correct. The camera LCD and histogram are 8 bit renderings based on the camera company's raw to JPEG SDK rendering. For the 1DS MII (which I also shoot) it's an easy 1 stop conservative clipping warning. If you are doing ETTR of a scene whose contrast range is about stop or 2 lower than the sensor's dynamic range it can be very tricky to nail the optimal exposure. I tend to bracket +- 2/3 stop around an exposure compensated meter reading...usually the plus 2/3 is fine.

And yes, PV 2012 CAN make far better use of highlight detail in a raw capture. The new tone mapping extracts a lot more highlight textural detail and can go into the shadows a lot better without artifacts. That will have an impact on how you think about doing ETTR and HDR...and with today's sensors the need for ETTR has, I think been reduced but not eliminated. It's gotten to the point where rather than worry over much about ETTR I'm more concerned about nailing a "correct" exposure. For high contrast scenes that's deciding whether or not highlight texture is visually important, if it is, I'll cut exposure to preserve highlights or do a bracket so I can blend 2 or more layers to retain texture. For a lower contrast scene I'll still push exposure to the right as long as I can maintain the require F-stop & shutter speed to get the technical capture optimal. Sometimes even pushing the ISO a bit (depending on the camera's noise signature).
Title: Re: ETTR and LR 4 (and DxO)
Post by: deejjjaaaa on April 22, 2012, 10:15:11 pm
It could seem that since I understand that the DxO measurements are based on the DxO raw converter

absolutely not - they sell a separate software = http://www.dxo.com/intl/image_quality/dxo_analyzer
Title: Re: ETTR and LR 4 (and DxO)
Post by: deejjjaaaa on April 22, 2012, 10:20:29 pm
Interesting... you seem to have identified 3 different definitions of "clipped highlights": (1) the camera's definition (as shown by blinking highlights in playback - which may be the least accurate if the playback image has been processed in-camera as they often are), (2) the PV2010 definition and (3) the PC2012 definition (as shown by highlight clipping in LR). Of course, DxO will have their own definition and, no doubt, Capture One and Aperture may have different definitions again.

some raw converters will show the real raw histogram, for example rawtherapee, rpp - which is the real situation with the data and you can see what is really clipped (which channel(s)), unlike ACR/LR where you have only approximation (in some cases good enough for you though).

Title: Re: ETTR and LR 4 (and DxO)
Post by: eliedinur on April 23, 2012, 07:27:29 am
The LR histogram is far from an approximation since it represents image data after the application of WB, which changes the data set fundamentally and can in itself cause clipping that was not present prior to the application. Daylight WB, for instance, can multiply red channel values by more than x2. Tungsten WB will increase the blue channel by even more. WB is the primary reason why both camera and LR histograms are remote from Raw reality.
Title: Re: ETTR and LR 4 (and DxO)
Post by: bjanes on April 23, 2012, 08:23:47 am
The LR histogram is far from an approximation since it represents image data after the application of WB, which changes the data set fundamentally and can in itself cause clipping that was not present prior to the application. Daylight WB, for instance, can multiply red channel values by more than x2. Tungsten WB will increase the blue channel by even more. WB is the primary reason why both camera and LR histograms are remote from Raw reality.

One main difference between the monochrome camera histogram and the ACR/LR histogram is that the former is a luminosity histogram, whereas the ACR/LR histogram is a composite RGB histogram (see Cambridge in Color (http://www.cambridgeincolour.com/tutorials/histograms2.htm)). In Photoshop, one can select the type of histogram. The luminosity histogram is largely determined by the green component. The more advanced cameras have color histograms showing the RGB channels separately after white balance. To avoid the white balance multipliers, one can load a UniWB profile into the camera, but saturation clipping can still occur, since the widest color space available with most cameras is Adobe RGB. It is time that the camera makers should offer a true raw histogram with a log base 2 scale (f/stops) on the x-axis. A log scale is often better for the y-axis as well.

Regards,

Bill
Title: Re: ETTR and LR 4 (and DxO)
Post by: Hans Kruse on April 23, 2012, 08:29:35 am
Thanks for all the answers and comments. I have used Rawdigger to analyze some of my pictures and compared to both LR PV2010 and PV2012 clipping indications using a completely neutral setting in the develop module. One thing is the fairly big difference between PV2010 and PV2012 which bothers me as to how to choose a bracketed shot that is optimal and without any channel clipped. I knew, of course, that the histogram changes and also the clipping indicator when I change e.g. the exposure slider in LR (as well as many other sliders of course), but I somehow had the belief that in a neutral setting the clipping indicator would be a good indicator for any clipped RAW data and wrong I was!

So not only can we not expose to the right when using our cameras as Michael has written about here http://www.luminous-landscape.com/tutorials/optimizing_exposure.shtml but we are also stuck in post processing unless we check the RAW histogram using a tool like Rawdigger for each picture since we cannot rely on the clipping indicators in any way in LR. This may sound a bit overstated, but from my analysis of some test shots done today it is clear that there is no safe way to know that a channel is clipped just by looking at LR histogram and clipping indicators (which I understand is purely a function of the histogram).

So what do we do? Well if the picture looks good is there a problem? Maybe we don't see the problem on our screens when doing the pp. Maybe we see it when a large print is made and we can see in the sky that areas are not rendered right due to clipping. One thought come to mind that I would be very desirable to have an option in LR to show clipping based on RAW data and not just the rendered data.
Title: Re: ETTR and LR 4 (and DxO)
Post by: bjanes on April 23, 2012, 08:36:12 am
Thanks for all the answers and comments. I have used Rawdigger to analyze some of my pictures and compared to both LR PV2010 and PV2012 clipping indications using a completely neutral setting in the develop module. One thing is the fairly big difference between PV2010 and PV2012 which bothers me as to how to choose a bracketed shot that is optimal and without any channel clipped. I knew, of course, that the histogram changes and also the clipping indicator when I change e.g. the exposure slider in LR (as well as many other sliders of course), but I somehow had the belief that in a neutral setting the clipping indicator would be a good indicator for any clipped RAW data and wrong I was!

Hans,

My experience with PV2012 is similar to yours. With respect to PV2010, I have found that a linear tone curve (sliders on main tab zeroed and point curve to linear) gives an accurate indication of clipping if one compensates for the BaselilneExposure offset that ACR/LR uses (+0.5 EV for the Nikon D3, so one must use -0.5 EV exposure in ACR).

Regards,

Bill
Title: Re: ETTR and LR 4 (and DxO)
Post by: Hans Kruse on April 23, 2012, 09:11:26 am
As an example of the test shots I did and where Rawdigger shows clipping and what the clipping indicators were in LR using PV2010 and PV2012 with neutral settings and WB as AWB from the camera (close to daylight). The metering in the camera (Canon 1Ds mkIII) using matrix metering (aka. evaluative) would naturally underexpose this scene, so I bracketed with 1/3 stop between between EV 0 to EV +2. Only at EV+2 were there clipping in the RAW data (_MG_4747) and LR PV2012 showed no clipping and PV2010 a very large clipped area.  Screen shots from LR for _MG_4746 are not attached but LR PV2010 showed clipping and PV2012 not (of course).
Title: Re: ETTR and LR 4 (and DxO)
Post by: Hans Kruse on April 23, 2012, 09:22:16 am
If by that you mean that most all cameras' display an sRGB (or Adobe RGB) ripped preview and then apply a very, very conservative indicator for clipping (for Canon about 1 stop) then that would be generally correct. The camera LCD and histogram are 8 bit renderings based on the camera company's raw to JPEG SDK rendering. For the 1DS MII (which I also shoot) it's an easy 1 stop conservative clipping warning. If you are doing ETTR of a scene whose contrast range is about stop or 2 lower than the sensor's dynamic range it can be very tricky to nail the optimal exposure. I tend to bracket +- 2/3 stop around an exposure compensated meter reading...usually the plus 2/3 is fine.

And yes, PV 2012 CAN make far better use of highlight detail in a raw capture. The new tone mapping extracts a lot more highlight textural detail and can go into the shadows a lot better without artifacts. That will have an impact on how you think about doing ETTR and HDR...and with today's sensors the need for ETTR has, I think been reduced but not eliminated. It's gotten to the point where rather than worry over much about ETTR I'm more concerned about nailing a "correct" exposure. For high contrast scenes that's deciding whether or not highlight texture is visually important, if it is, I'll cut exposure to preserve highlights or do a bracket so I can blend 2 or more layers to retain texture. For a lower contrast scene I'll still push exposure to the right as long as I can maintain the require F-stop & shutter speed to get the technical capture optimal. Sometimes even pushing the ISO a bit (depending on the camera's noise signature).

Thanks! I'm looking forward to your writeup. I can imagine that your Adobe friends (and you as well, maybe?) would be against adding a RAW histogram option to an already fairly rich interface. Maybe a button or short cut (of the many still available :)) to display the RAW histogram and the clipping for the RAW data. For Canon sensors and a high DR scene that one would prefer to pp in LR as a single image one really need to have the optimum exposure not to loose some of that precious DR that the Canon does not have too much of. For Sony Exmor sensors it's a lesser need, but still optimizing noise performance and avoid blending can be quite desirable. Especially with the great working of the highlight and shadows sliders in both the detail panel and in the grad filter and brush.

So a reliable (RAW) clipping indicator would be really good to have in LR.
Title: Re: ETTR and LR 4 (and DxO)
Post by: bjanes on April 23, 2012, 10:04:02 am
As an example of the test shots I did and where Rawdigger shows clipping and what the clipping indicators were in LR using PV2010 and PV2012 with neutral settings and WB as AWB from the camera (close to daylight). The metering in the camera (Canon 1Ds mkIII) using matrix metering (aka. evaluative) would naturally underexpose this scene, so I bracketed with 1/3 stop between between EV 0 to EV +2. Only at EV+2 were there clipping in the RAW data (_MG_4747) and LR PV2012 showed no clipping and PV2010 a very large clipped area.  Screen shots from LR for _MG_4746 are not attached but LR PV2010 showed clipping and PV2012 not (of course).

Hans,

Did you use the proper baseline offset for the 1DsMIII for the PV2010 rendering? It is +0.35 EV, so you would have to dial in -0.35EV exposure. This can be determined with the Rawdigger Exif utility with a DNG file.

Regards,

Bill
Title: Re: ETTR and LR 4 (and DxO)
Post by: John R Smith on April 23, 2012, 10:07:08 am
How do you find out what the LR baseline offset is for a Hasselblad 3FR file?

John
Title: Re: ETTR and LR 4 (and DxO)
Post by: Hans Kruse on April 23, 2012, 10:15:35 am
Hans,

Did you use the proper baseline offset for the 1DsMIII for the PV2010 rendering? It is +0.35 EV, so you would have to dial in -0.35EV exposure. This can be determined with the Rawdigger Exif utility with a DNG file.

No, I didn't. The large clipping area goes away on _MG_4747 with -0.65EV and on 4746 it goes away with -0.35EV. I'll check later what Rawdigger says as I'm going out shooting now :)

However I still would prefer an easy to use clipping indicator that says what RAW data is clipped. Sometimes one could choose to ignore it or crop the area away.
Title: Re: ETTR and LR 4 (and DxO)
Post by: bjanes on April 23, 2012, 11:01:31 am
How do you find out what the LR baseline offset is for a Hasselblad 3FR file?

John,

You would convert the raw file to DNG (if it isn't already in DNG) and use an exif reader to check the baseline offset. Rawdigger can read the exif.

Regards,

Bill
Title: Re: ETTR and LR 4 (and DxO)
Post by: Hans Kruse on April 23, 2012, 03:50:33 pm
Hans,

Did you use the proper baseline offset for the 1DsMIII for the PV2010 rendering? It is +0.35 EV, so you would have to dial in -0.35EV exposure. This can be determined with the Rawdigger Exif utility with a DNG file.

Regards,

Bill

I checked and it is correct (of course) that the baseline offset is 0.35EV for the 1Ds mkIII.
Title: Re: ETTR and LR 4 (and DxO)
Post by: eliedinur on April 23, 2012, 07:09:51 pm
Quote
So a reliable (RAW) clipping indicator would be really good to have in LR
..
In the meantime .... I have found that putting the Temperature and Tint sliders all the way over to the left (2000, -150) will produce a histogram that is very much like a Uni-WB histogram. It may not be a true null WB, but it is close. (I often shoot with a Uni-WB and at the "As Shot" setting LR's sliders go to 2050 and -150). It also is quite close to histograms in Raw Digger and Rawnalyze and I believe it gives a fairly good representation of the nature of the Raw capture, especially with regard to clipping.
Title: Re: ETTR and LR 4 (and DxO)
Post by: Hans Kruse on April 24, 2012, 07:04:08 am
I'm somewhat embarrased not to have seen Uni-WB mentioned before and not knowing about it....does anybody have a file for the 1Ds mkIII that could be used for setting the Uni-WB in camera? I'm not sure I would like to shoot with that color tempeture (I shoot RAW of course), but it would be interesting to check it. Also if this helps checking in LR this would be good to have. I do think LR PV2012 does a beautiful job in handling the situations where just channel is clipped and I have checked some pictures where the sky (clouds) contained clippings and I checked one version not clipped versus one clipped and I could not see the difference in the areas clipped. But the fact that LR PV2012 gives me no warnings about clipping is a concern as mentioned a couple of times. Now I know that just changing to PV2010 will give a hint in some cases and a fairly quick turn around for critical pictures via Rawdigger is fairly quick by right clock an image and choose show in Finder and the open the RAW file with Rawdigger. But if LR could do it directly that would be nice :)
Title: Re: ETTR and LR 4 (and DxO)
Post by: eliedinur on April 24, 2012, 07:36:36 am
Back several years ago when I first started messing with Uni-WB, people were creating magenta jpgs that could be loaded into the camera and used to create a Uni-WB custom WB setting. Then somebody (E-K at the POTN forum) discovered a much easier way. He reasoned that if the (Canon) camera were given a target image that was already perfectly balanced (R=G=B), the calculated custom WB multipliers would be 1/1/1. There are two ways to easily create such a target; either shoot a black frame (0/0/0) by leaving the lens cap on or completely overexpose a frame (255/255/255). I prefer the black frame. When you use it to make the CWB, the camera will warn you that the result will not be pretty. Ignore that.
Title: Re: ETTR and LR 4 (and DxO)
Post by: Hans Kruse on April 24, 2012, 08:18:04 am
Back several years ago when I first started messing with Uni-WB, people were creating magenta jpgs that could be loaded into the camera and used to create a Uni-WB custom WB setting. Then somebody (E-K at the POTN forum) discovered a much easier way. He reasoned that if the (Canon) camera were given a target image that was already perfectly balanced (R=G=B), the calculated custom WB multipliers would be 1/1/1. There are two ways to easily create such a target; either shoot a black frame (0/0/0) by leaving the lens cap on or completely overexpose a frame (255/255/255). I prefer the black frame. When you use it to make the CWB, the camera will warn you that the result will not be pretty. Ignore that.

Thanks, just did this and the WB setting in LR (as shot) becomes 2700, -150. For some testshots bracketed by 1/3 there was a perfect relation between blinkies on the camera LCD and the clipping in Rawdigger. LR did with PV2012 show clipping at the as shot WB although small and PV2010 larger. Of course that WB could not be used, so going to e.g. daylight there were no clipping warnings in LR. Will do some more testing.
Title: Re: ETTR and LR 4 (and DxO)
Post by: eliedinur on April 24, 2012, 09:20:56 am
Thanks, just did this and the WB setting in LR (as shot) becomes 2700, -150. For some testshots bracketed by 1/3 there was a perfect relation between blinkies on the camera LCD and the clipping in Rawdigger. LR did with PV2012 show clipping at the as shot WB although small and PV2010 larger. Of course that WB could not be used, so going to e.g. daylight there were no clipping warnings in LR. Will do some more testing.
When I use the eye-dropper WB tool in LR on a black frame it goes to 2700/-150. I think the important factor is the Tint -150 because in normal broad spectrum light the green channel will always be most exposed. If the green is not clipped the chances are very good that the other two are also not clipped.
Title: Re: ETTR and LR 4 (and DxO)
Post by: Hans Kruse on April 25, 2012, 03:51:01 pm
I have checked further and found the clipping warning on the LCD pretty accurate compared to Rawdigger when setting the camera WB to Uni-WB. And even better this means that the live histogram in live view when the Uni-WB is set is a real RAW histogram (or at least very close to). The only bad thing is that libe view on the 1Ds mkIII do not have electronic first curtain ;) So ETTR is easy in live view in this way.
Title: Re: ETTR and LR 4 (and DxO)
Post by: Hans Kruse on May 12, 2012, 07:35:38 am
I guess it all depends on what you consider to be "highlight recovery"...the new image adaptive PV 2012 has an auto-ranging capability which can appear as "recovery" but I don't think it really is in a classic sense. What I think is going on is that the way the highlights are rendered in PV 2012 has a different roll off which extracts more highlight detail. Note that in a linear capture, the brightest area of the capture has a lot of data even if there is no clipping. Recurving the tone mapping to extract more usable image data is certainly what PV 2012 is doing. That's what I think people are seeing...and I don't call that "highlight recovery" which to me means extracting image detail when one or more channels are clipped. I'm pretty sure there is a small degree of classic highlight recovery going on even when all Basic panel controls are at zero which I think is what Eric is talking about and I think it's the new tone mapping that is making the biggest differences. But a minus Exposure or Highlights is clearly when the classic "highlight recovery" really kicks in and that has been substantially improved in PV 2012.

I'm still coming to grips with PV 2012 and I've been working with it in various forms since late last summer...what I think is going on is that this new tone mapping logic is really, really leading edge and traditional descriptions and terms fail to really describe what going on under the hood. I've read the post on Lightroom Journal titled Magic or Local Laplacian Filters? (http://blogs.adobe.com/lightroomjournal/2012/02/magic-or-local-laplacian-filters.html) and that stuff is too deep for me to understand...but what I will say it that PV 2012 has made a major improvement in the ability to tone map raw captures. I can't say it's "perfect" (there are a couple of types of images where tone bleeding can occur) but compared to PV 2010 it's clear that one can get a lot more image detail out of highlights and shadows. I hesitate to call this a change in the dynamic range of the image but is sure does make it easier to deal with higher contrast scenes...I'm going to be in the process of writing about this stuff in the very near future...the way I usually do that is ask some questions of Eric and Thomas, swirl it around and spit it out and see if they agree with me. I'll let ya all know how that goes :~)

Not to push you ;) and I don't I could even if I tried, but it would be really great with a more comprehensive writeup on what the new PV2012 algorithms do in the highlight are and in the shadows as well. Since I came home from my Tuscany trip, I have looked at many of the older captures where I have kept overexposed ones for exposure blending and found that in many cases the overexposed image (clipped by typically one channel in the RAW histogram) which was not usable in PV2010 now looks indistinguishable in 1:1 view from the exposure one stop lower (and under the clipping limit according to the RAW histogram) in the highlights. This means, of course, that in high DR scenes I can in many cases do tone mapping in LR4 on a single exposure rather than use Photomatix Pro, Photoshop HDR Pro (or manual blending in Photoshop) or other tools. This is wonderful news compared to LR3 and in my opinion alone a strong reason to upgrade to LR4 (if anyone still doubts that LR3 to LR4 is a worth while upgrade!!).

So do I need a writeup to continue using LR4 for what I have now found to be an amazing new capability? No, certainly not, but I'm curious to see an explanation of what it really does and where the limits are so to speak! For example, can it recover more than one channel clipped? Does it work from interpolation across an area where clipping occurs and estimate color and luminosity from neighboring areas to fill in the gaps in the clipped areas? Etc. etc. and it is probably much more advanced that what I hint at here. It would be good to know how this works rather than reverse engineer the understanding from working with the pictures alone.

Using a Canon 1Ds mkIII this is very good news and does away with many of the situations where I previously would do a form of exposure blending, since I can now edit the picture in LR4 alone. So for any Adobe engineers reading this, a huge thanks for this new capability in LR4.
Title: Re: ETTR and LR 4 (and DxO)
Post by: jrsforums on May 12, 2012, 10:04:21 am
If I may, let me share some of the things, I think, I have learned and some of the exchanges I have had with Eric...

First, I do not believe there is any easy or cookbook way to rationalize what your camera histogram shows and how far you can push ETTR to not blow highlights in LR.  Each camera system and usually each camera is going to be a little different. 

You need to learn how you camera is going to work....i.e. a calibrate it.  Lee Varis, in his book 'Mastering Exposure..." (or a free PDF extract from it) shows how one can calibrate it...i.e. how much room do you have after you hit the blinkies and right side of the chart. This, obviously, needs to be done in oncert with the raw converter you are using.

One thing I noticed, and is cussed with Eric, is that 2010 and 2012 are quite different in what hey tell you about over exposed highlights.  In 2010, if you have red indicators of over exposure in your image, that does not tell you if they are truly blown or just that the other settings have pushed them there.  Only later, if you try to recover hem will you find that some are really blown and have lost all, or most, detail.

I have found 2012 to be quite wonderful in this area.  At the initial default settings, the only areas which show red are are those that are totally or partially "blown", i.e. misinterpret detail.  This makes it much easier to determine if these areas are significant to the image...and if so...to not even bother to try to work on the image as these areas will never look good and will usually at best provide a silvery or grayish area with no texture.

Anyway, this was first on my list of "likes" about LR 4.  The more I use it, the longer the list becomes.

John
Title: Re: ETTR and LR 4 (and DxO)
Post by: Hans Kruse on May 12, 2012, 10:32:00 am
In the past I found a pretty ok correlation between the blinkies on the camera and in LR. That could be improved a bit using the Uni-WB setting which I tried.

Using LR4 PV2012 there is no longer such a relationship as mentioned. If you take a picture that has a blown channel seen in the RAW histogram using e.g. Rawdigger then often LR4 PV2012 will not show any clipping in the default input and when adjusted for the baseline offset. So what is interesting is how I can rely on the clipping indicators in LR4 PV2012 or not. In essence what are the algorithms in lay mans terms that make it possible to not show clipping indicators in areas where there is a blown channel (and yes, I know that the white balance will change this hugely). What I do in practical terms is to compare pictures from an exposure that is not clipped with one that is clipped but LR4 PV2012 says is ok and see if there is any visible difference in the areas of clipping (which I can check in addition in PV2010 using the Uni-WB (2700, -150) which corresponds well with the Raw histogram).
Title: Re: ETTR and LR 4 (and DxO)
Post by: jrsforums on May 12, 2012, 11:14:25 am
In the past I found a pretty ok correlation between the blinkies on the camera and in LR. That could be improved a bit using the Uni-WB setting which I tried.

Using LR4 PV2012 there is no longer such a relationship as mentioned. If you take a picture that has a blown channel seen in the RAW histogram using e.g. Rawdigger then often LR4 PV2012 will not show any clipping in the default input and when adjusted for the baseline offset. So what is interesting is how I can rely on the clipping indicators in LR4 PV2012 or not. In essence what are the algorithms in lay mans terms that make it possible to not show clipping indicators in areas where there is a blown channel (and yes, I know that the white balance will change this hugely). What I do in practical terms is to compare pictures from an exposure that is not clipped with one that is clipped but LR4 PV2012 says is ok and see if there is any visible difference in the areas of clipping (which I can check in addition in PV2010 using the Uni-WB (2700, -150) which corresponds well with the Raw histogram).

Since the camera histogram is showing the jpeg, there should not be a correlation with truly blown highlights in LR.  This is the exact problem I mentioned in LR 3.  At the default settings, the red over exposure indications were not necessarily truly blown....and in most cases, most were not.

In 2012, all the red areas (you may need to blow up to 100 or 200%) to ensure all the area is red) will have RGB of 100, possible 99 which Eric said could be a rounding problem).  Any RGB. Pixel lower than that will not show red....unless you force it with slider changes.

I am kind of a pragmatic person.  While I like to understand what is happening under the covers, practically, if the area under consideration has correct color and textural detail, I really don't care how it was arrived at or whether it was partially blown and recovered.  I do want to know quickly what cannot be recovered so I do not waste my time in pp.

So I am happy with LR 4, even if I need to sorta guess how much I can push past the warnings of the jpeg blinkies and histogram.  A little exploring/comparing allows me to pretty much nail the ETTR for 2012.  If it is critical, I will bracket.
Title: Re: ETTR and LR 4 (and DxO)
Post by: Hans Kruse on May 12, 2012, 11:48:41 am
Since the camera histogram is showing the jpeg, there should not be a correlation with truly blown highlights in LR.  This is the exact problem I mentioned in LR 3.  At the default settings, the red over exposure indications were not necessarily truly blown....and in most cases, most were not.

In 2012, all the red areas (you may need to blow up to 100 or 200%) to ensure all the area is red) will have RGB of 100, possible 99 which Eric said could be a rounding problem).  Any RGB. Pixel lower than that will not show red....unless you force it with slider changes.

I am kind of a pragmatic person.  While I like to understand what is happening under the covers, practically, if the area under consideration has correct color and textural detail, I really don't care how it was arrived at or whether it was partially blown and recovered.  I do want to know quickly what cannot be recovered so I do not waste my time in pp.

So I am happy with LR 4, even if I need to sorta guess how much I can push past the warnings of the jpeg blinkies and histogram.  A little exploring/comparing allows me to pretty much nail the ETTR for 2012.  If it is critical, I will bracket.

When setting the Uni-WB and the baseline offset there is a good correlation between tje JPG blinkies on the camera and what LR shows with PV2010. Try it. Getting the exposure right is not really difficult, I bracket for landscape shooting to get the optimal exposure, especially in high DR situations. But I do it most of the time anyway because this gives me the largest flexibility in post processing where I often edit a lot to get the look I'm after.

I'm also quite a pragmatic person, but I like to have the guess work out of the place if possible. In fact what I miss is RAW clipping indicator (different color) to overlay the normal LR colored clipping indicator area. In this way I would be able to see where the problems might be. I can use a virtual copy set with the above Wb, offset and PV2010, but then I have to switch back and forth between them. This is doable but a bit cumbersome.

Despite being pragmatic I like to know how things work, a left over from the IT profession in the past, I guess :)
Title: Re: ETTR and LR 4 (and DxO)
Post by: jrsforums on May 12, 2012, 12:05:25 pm
I did try UniWB and did not like it (to each their own)....and some guess work still remained.  I do try to get the jpeg histogram closer (less guess) by lowering the saturation and contrast setting as much as possible...I should also lower. Sharpening, but actually raise it to better judge image.

At default settings, 2012 red indicators are the "RAW clipping indicators" at least as far as LR conversion is concerned.  In your case, however, I expect the UniWB throws that off.

John
Title: Re: ETTR and LR 4 (and DxO)
Post by: Hans Kruse on May 12, 2012, 12:24:32 pm
At default settings, 2012 red indicators are the "RAW clipping indicators" at least as far as LR conversion is concerned.  In your case, however, I expect the UniWB throws that off.


No, that's not true and you can easily see this by looking at a RAW image in Rawdigger. I do not use UniWB when checking in PV2012 and in general do not use UniWB. Just checked it.
Title: Re: ETTR and LR 4 (and DxO)
Post by: jrsforums on May 12, 2012, 03:34:36 pm
No, that's not true and you can easily see this by looking at a RAW image in Rawdigger. I do not use UniWB when checking in PV2012 and in general do not use UniWB. Just checked it.

Not sure I understand or am missing your point.

I specifically said "as far as LR conversion was concerned".  I don't kow enough about intricacies of raw conversion, but I would think than comparing raw conversion between LR and RAwdigger would be apples and oranges.

John
Title: Re: ETTR and LR 4 (and DxO)
Post by: Hans Kruse on May 12, 2012, 03:44:56 pm
Not sure I understand or am missing your point.

I specifically said "as far as LR conversion was concerned".  I don't kow enough about intricacies of raw conversion, but I would think than comparing raw conversion between LR and RAwdigger would be apples and oranges.

John

No, it is not apples and oranges. Rawdigger indicates the true histogram and where clipping occurs. That's why I'm saying that rather than using an approximation using PV 2010 and Uni-WB og base level offset, it would be much better to be able to see where the RAW clipping is since LR will not show clipping in areas where there are clipping in the RAW data. I hope it is clear now ;) Just having LR show where LR will clip because of the editing is not enough for me at least.
Title: Re: ETTR and LR 4 (and DxO)
Post by: Bart_van_der_Wolf on May 12, 2012, 05:45:34 pm
[...] it would be much better to be able to see where the RAW clipping is since LR will not shot clipping in areas where there are clipping in the RAW data.

Yes, I agree with Hans. The Exposure control has stopped being a post-exposure gain control, and in PV2012 has been turned into a mid-tone control. It is not a reliable tool to check exposure anymore, because the connection of highlight rendering and underlying Raw data in not obvious. The simple solution is, like implemented in "RawTherapee", an optional Raw histogram (and with adjustable vertical scale).

Cheers,
Bart
Title: Re: ETTR and LR 4 (and DxO)
Post by: aduke on May 12, 2012, 06:28:28 pm
It seems to me that a RAW histogram would be useful in LRx, where x > 4.

Alan