Luminous Landscape Forum

Raw & Post Processing, Printing => Digital Image Processing => Topic started by: Tim Lookingbill on December 16, 2018, 04:15:04 pm

Title: Optimize In Camera Jpeg White Balance RGB Histogram For Better ETTR.
Post by: Tim Lookingbill on December 16, 2018, 04:15:04 pm
This may be old news to some folks here but I just found a way to make the jpeg histogram more useful for ETTR. I tested this out just now just to make sure I could affect my Pentax K200D's individual RGB jpeg histogram display rendered according to white balance setting and exposure.

Just discovered my used 2008 Pentax camera offers an adjustable Custom White balance to make it green, blue, purple, amber etc.

So I set the Custom WB at neutral 5000K/+6 Tint as read in ACR's As Shot and took a shot of my LED display showing a light gray field. As would be expected the incamera individual RGB histograms are all the same shape where the non-clipped highlights line up perfectly between each other. Took another shot of my LED display opening one third stop which had all three highlights slammed and spiked to the right.

Adjusted the Custom WB by moving the color temp color graph map to ALL GREEN and took the shot again using the same exposure that clipped the highlights with the previous neutral WB exposure. Guess what! The RGB histogram highlights weren't slammed and spiked to right. In fact they were nice ant hill shapes with no spikes.

Thought I'ld share this with those who have cameras with an adjustable Custom White balance and individual RGB histograms that reflect the affects of changing exposure and white balance.

Has anyone tried this?

This also could work by setting a custom WB using a color target that would make the green channel even more biased and then go back into the WB color adjust graph map and make it even more green which would give more predictable exposure latitude for ETTR. ACR As Shot WB does show the effects of distorting the in camera WB this way.

I did a search on this subject to make sure this wasn't already covered and came up with nothing.

Title: Re: Optimize In Camera Jpeg White Balance RGB Histogram For Better ETTR.
Post by: BAB on December 17, 2018, 08:34:28 am
I think you would need RAW histogram by channel view to achieve proper ETTR? Then the camera is not cooking the file output. Your test may only be correct for a given situation of EV.
Title: Re: Optimize In Camera Jpeg White Balance RGB Histogram For Better ETTR.
Post by: digitaldog on December 17, 2018, 09:22:12 am
I think you would need RAW histogram by channel view to achieve proper ETTR? Then the camera is not cooking the file output. Your test may only be correct for a given situation of EV.
You are correct about a raw Histogram. No need for JPEG Histogram hacks either. At least for photographers who had ever shot transparency film without a Histogram anywhere near the camera. Far from difficult and photography 101 (optimal exposure). Get something like RawDigger to run exposure tests to actually understand how your sensors optimally expose the raw data, understand how light meters actually operate, ignore the LCD lies about data you are NOT capturing. ETTR is an old and poor term that needs to go away. The correct term is correct exposure; something most photographers have achieved for well over 100 years.
Has anyone tried this?  ;D
Title: Re: Optimize In Camera Jpeg White Balance RGB Histogram For Better ETTR.
Post by: DP on December 17, 2018, 09:26:01 am
I did a search on this subject to make sure this wasn't already covered and came up with nothing.

search for UniWB... it was covered miriad times
Title: Re: Optimize In Camera Jpeg White Balance RGB Histogram For Better ETTR.
Post by: TonyW on December 17, 2018, 11:06:38 am
This may be old news to some folks here but I just found a way to make the jpeg histogram more useful for ETTR. I tested this out just now just to make sure I could affect my Pentax K200D's individual RGB jpeg histogram display rendered according to white balance setting and exposure.

Just discovered my used 2008 Pentax camera offers an adjustable Custom White balance to make it green, blue, purple, amber etc.

So I set the Custom WB at neutral 5000K/+6 Tint as read in ACR's As Shot and took a shot of my LED display showing a light gray field. As would be expected the incamera individual RGB histograms are all the same shape where the non-clipped highlights line up perfectly between each other. Took another shot of my LED display opening one third stop which had all three highlights slammed and spiked to the right.

Adjusted the Custom WB by moving the color temp color graph map to ALL GREEN and took the shot again using the same exposure that clipped the highlights with the previous neutral WB exposure. Guess what! The RGB histogram highlights weren't slammed and spiked to right. In fact they were nice ant hill shapes with no spikes.

Thought I'ld share this with those who have cameras with an adjustable Custom White balance and individual RGB histograms that reflect the affects of changing exposure and white balance.

Has anyone tried this?

This also could work by setting a custom WB using a color target that would make the green channel even more biased and then go back into the WB color adjust graph map and make it even more green which would give more predictable exposure latitude for ETTR. ACR As Shot WB does show the effects of distorting the in camera WB this way.

I did a search on this subject to make sure this wasn't already covered and came up with nothing.
Guillermo Lujik who does post here occasionally wrote (and devised?) UniWB.  The thrust of your thread seems to be similar?
http://www.guillermoluijk.com/tutorial/uniwb/index_en.htm

I believe the main problem with trying to achieve ETTR in camera for JPEG is apart from the fact that a "correctly" exposed raw will likely be an "incorrectly" exposed JPEG is that the camera metering system may let you down - unless you are able to actually meter a highlight ROI that you must hold detail and then set exposure accordingly. 

Cannot speak for your Pentax but... Auto metering may be problematical in as much as you really do not know exactly what areas are being measured and what 'clever' algorithms being applied by the manufacturer (Nikon Matrix).  Centre-weighted is also an issue as it is not always evident how much weighting given to the overall measuring area.  Spot metering should be best but again need to know the exact area of measurement i.e. size and position in relation to what you see through the viewfinder or in LV.  In addition, you need to establish a meter calibration point to find out how much you need to allow for exposure correction once you have measured your most important highlight ROI e.g. +2, +2.5, +3 EV's to ETTR

Other than that relying on the camera screen LCD and RGB and composite histograms will give you an 'acceptable' capture quite/very often, and if you have the time to test exposures (not the decisive moment of course because you will miss it  ;) ) by manual or auto bracketing then you may achieve a degree of satisfaction for ETTR JPEG by seeing the histogram stacked to the right. 
Title: Re: Optimize In Camera Jpeg White Balance RGB Histogram For Better ETTR.
Post by: Tim Lookingbill on December 17, 2018, 12:03:14 pm
search for UniWB... it was covered miriad times
I've heard of UniWB several years back in this forum. I'm not trying to win any converts for true ETTR with this post.

I'm simply relaying a procedure I stumbled upon after toying around with Custom WB that allows adjusting hue to force the jpeg green channel forward ahead of the red and blue. I find it to be a good enough demonstration of how the hue of WB can affect exposure parameters close to full saturation when shooting Raw for cameras that don't have an in camera Raw histogram.

And to be clear the magenta tinted test shot that severely clipped the histogram on all channels was recoverable but there were linearity issues where after I white balanced in ACR showed slightly darker tones around 200RGB were yellowish and those lower were neutral. This is in PV2010 reducing the Exposure slider where doing this with the Brightness slider compressed the detail that forced the yellow to stay neutral at the sacrifice of detail.

That's the bit of detail left out of ETTR discussions is what exposing this close to saturation does to linearity according to the PV version and what highlight recovery slider one uses. PV2012 just moved this preserve linearity without compression feature over to another slider.

I might have over exposed the magenta tinted test image 1/3 more stop where it would extremely clip in the jpeg histogram but then recovery in ACR or LR might just compress the highlight detail which doesn't look good shooting brightly lit white bilious clouds.

Mine is an old camera with less dynamic range capabilities so newer cameras would most likely allow more recoverable detail with exposures closer to saturation.

But what is the quality of what is recovered. Preserve highlight color detail and linearity or accept compression to be the only solution.
Title: Re: Optimize In Camera Jpeg White Balance RGB Histogram For Better ETTR.
Post by: digitaldog on December 17, 2018, 12:11:49 pm
WB has no effect on raw data let alone exposure. Only the amount of light striking the sensor: shutter and aperture.
Title: Re: Optimize In Camera Jpeg White Balance RGB Histogram For Better ETTR.
Post by: DP on December 17, 2018, 12:22:21 pm
Guillermo Lujik who does post here occasionally wrote (and devised?) UniWB. 

UniWB predates Guillermo Lujik (but he contributed to popularization)
Title: Re: Optimize In Camera Jpeg White Balance RGB Histogram For Better ETTR.
Post by: Tim Lookingbill on December 17, 2018, 12:26:00 pm
Something else I discovered shooting wide dynamic range scenes that force an exposure of  1/250/s, f/16, ISO 200 on a sunny day to preserve the highlights of houses where I got a lot of noise in the shadow areas of houses that wound up being more midrange instead of deep shadows meaning there was more bounced fill light in the sides of houses not getting direct sunlight on such a bright sunny day.

But around close to dusk and in the shade I exposed at 1/80's, f/8, ISO 800 and there's NO NOISE in the shadows of a macro shot of some old dead leaf laying on green clover. Explain that!

My Pentax came with a CCD sensor (not CMOS) which doesn't do so well with heat and the ISO 800 shot at dusk was a bit cooler around that time.
Title: Re: Optimize In Camera Jpeg White Balance RGB Histogram For Better ETTR.
Post by: digitaldog on December 17, 2018, 12:41:19 pm
The true way to know if highlights are 'preserved' due to exposure is to examine the raw Histogram.
The true way to know if there is no noise is first to bracket exposure and examine the noise instead of assuming none exists in any.
Explained but accepted?  ;)
I'm not sure why a few (amateur/hobbyists) have to make something as fundamental as idealized exposure for any media (neg, transparency, raw, JPEG) more difficult than it has to be.
Next we'll hear how difficult and complicated it is to focus (prior auto focus lens).  ;)
Title: Re: Optimize In Camera Jpeg White Balance RGB Histogram For Better ETTR.
Post by: DP on December 17, 2018, 12:50:47 pm
1/250/s, f/16, ISO 200
1/80's, f/8, ISO 800

1/80@f8 is a lot more exposure (incoming light) than 1/250 @ f16 - so for deep shadows in both shots it might not be a difference that 1/250 & f16 was a sunny day and 1/80 & f8 was not... you might preserve the details in the important highlights on a sunny day, but severely underexposed your shadows vs that shot during the dusk... spot meter shadows next time to see if they are exposed equally between two scenarios.

plus a good chance that analog gain in off sensor ADC in your Pentax @ ISO800 is more beneficial than @ ISO200 -> amplifying analog signal before ADC up to a certain limit is good - result looks like lesser readout related noise

ambient temperature might be a factor too

Pentax K100D probably has the same 6mp CCD & off sensor ADC = http://www.photonstophotos.net/Charts/PDR_Shadow.htm#Pentax%20K100D ... see that going above nominal ISO200 is beneficial for deep shadows (if you can't saturate the sensor with exposure for whatever reason)

Title: Re: Optimize In Camera Jpeg White Balance RGB Histogram For Better ETTR.
Post by: Tim Lookingbill on December 17, 2018, 02:55:40 pm
1/80@f8 is a lot more exposure (incoming light) than 1/250 @ f16 - so for deep shadows in both shots it might not be a difference that 1/250 & f16 was a sunny day and 1/80 & f8 was not... you might preserve the details in the important highlights on a sunny day, but severely underexposed your shadows vs that shot during the dusk... spot meter shadows next time to see if they are exposed equally between two scenarios.

plus a good chance that analog gain in off sensor ADC in your Pentax @ ISO800 is more beneficial than @ ISO200 -> amplifying analog signal before ADC up to a certain limit is good - result looks like lesser readout related noise

ambient temperature might be a factor too

Pentax K100D probably has the same 6mp CCD & off sensor ADC = http://www.photonstophotos.net/Charts/PDR_Shadow.htm#Pentax%20K100D ... see that going above nominal ISO200 is beneficial for deep shadows (if you can't saturate the sensor with exposure for whatever reason)

Some very convincing points I hadn't considered.

Most of my shots at ISO 800 regardless of the exposure always gave me lots of noise.

The macro shot at dusk was not only dim lighting but the majority of the scene was of dark objects like the deep green clover which usually requires I set exposure to let in more light, but even then I should've gotten much more noise. ACR defaults has the bulk of the data occupying the left 1/3 of the histogram meaning I underexposed a somewhat dark scene.

But I think I'm going to go with heat as the cause because I just remembered shooting a 5 sec. long exposure to capture a snow scene at night at my apt. lit only by a couple of porch lights from nearby tenants. I set it at ISO 200, the lowest with the K100D and there was very little noise in the darkest shadows but it was very cold out. ETTR worked the best shooting this way with a tripod which I used.
Title: Re: Optimize In Camera Jpeg White Balance RGB Histogram For Better ETTR.
Post by: digitaldog on December 17, 2018, 02:59:50 pm
And my ISO 800 images show LESS noise than ISO 100 in this case because ISO doesn't have anything to do with exposure; the amount of light striking the sensor. It does have something to do with dumb meters and not too intelligent photographers who don't understand how to expose:


(http://digitaldog.net/files/100vs800iso.jpg)
Title: Re: Optimize In Camera Jpeg White Balance RGB Histogram For Better ETTR.
Post by: smahn on December 18, 2018, 11:59:41 am
And my ISO 800 images show LESS noise than ISO 100 in this case because ISO doesn't have anything to do with exposure; the amount of light striking the sensor. It does have something to do with dumb meters and not too intelligent photographers who don't understand how to expose:


(http://digitaldog.net/files/100vs800iso.jpg)

Since it's the same amount of light in both cases, why do they differ?
Title: Re: Optimize In Camera Jpeg White Balance RGB Histogram For Better ETTR.
Post by: Frans Waterlander on December 18, 2018, 12:00:22 pm
And my ISO 800 images show LESS noise than ISO 100 in this case because ISO doesn't have anything to do with exposure; the amount of light striking the sensor. It does have something to do with dumb meters and not too intelligent photographers who don't understand how to expose:


(http://digitaldog.net/files/100vs800iso.jpg)

There you go again, ad hominem attack. Maybe you should explain the reasons why your 800 ISO shot has less noise, instead of attacking those you don't agree with.
Title: Re: Optimize In Camera Jpeg White Balance RGB Histogram For Better ETTR.
Post by: faberryman on December 18, 2018, 12:16:27 pm
Why do people make something which is simple and straightforward like exposure and make it unnecessarily complicated?
Title: Re: Optimize In Camera Jpeg White Balance RGB Histogram For Better ETTR.
Post by: digitaldog on December 18, 2018, 12:23:34 pm
There you go again, ad hominem attack.
Nope, simply facts backed up with an example.
Quote
Maybe you should explain the reasons why your 800 ISO shot has less noise, instead of attacking those you don't agree with.
Explain the obvious. To you again?
Title: Re: Optimize In Camera Jpeg White Balance RGB Histogram For Better ETTR.
Post by: digitaldog on December 18, 2018, 12:26:18 pm
Since it's the same amount of light in both cases, why do they differ?
Best explained in this post from way back in 2011 and the diagram:
https://forum.luminous-landscape.com/index.php?topic=56906.msg466484#msg466484
Title: Re: Optimize In Camera Jpeg White Balance RGB Histogram For Better ETTR.
Post by: stamper on December 18, 2018, 12:35:19 pm
Why do people make something which is simple and straightforward like exposure and make it unnecessarily complicated?

Exposure simple?
Title: Re: Optimize In Camera Jpeg White Balance RGB Histogram For Better ETTR.
Post by: Garnick on December 18, 2018, 12:40:11 pm
There you go again, ad hominem attack. Maybe you should explain the reasons why your 800 ISO shot has less noise, instead of attacking those you don't agree with.

Think back to film days.  If the subject seemed to call for more light hitting the film we had three alternatives.  When possible and feasible, we could perhaps literally add more light, as in a studio situation or something of the sort, where adding light and balancing the colour with filters was one alternative perhaps.  Of course the next approach would a combination of aperture and or shutter speed adjustments. The final approach was to use a higher ISO film, manufactured to be more sensitive to light.  The way I understand it, sensitivity to light is the most obvious reason for increasing the ISO, whether for film or for a digital camera sensor.  Of course the downside of using a higher ISO film was the grain factor, which was eventually overcome somewhat in some of the colour and B&W emulsions.  In the digital photography world, what we used to call "grain" is now referred to as "noise", or the S/N Ratio.  Now after all of this rambling I believe the phrase I used previously (sensitive to light) tells the tale as far as the apparent reduction of "noise" as we increase the ISO setting.  Even though the intensity of the light source does not change and nor does the shutter speed or aperture, as the ISO is increased the sensor's sensitivity to light increases, thus the reduction of noise, especially in the shadows and other darker areas of the image.  Again, as you at least approach the "Optimum Exposure Level" you will start to detect less noise in those areas normally most affected by a lack of exposure. 

I imagine Andrew might jump in and perhaps deflate my theory to some extent at least.  However, it's the way I see the situation and understand what's actually happening when I raise the ISO setting.  It does not increase the amount of light striking the sensor.  The sensor is then set to gobble up more of it, and for the most part that's a good thing.  I'm also certain that Andrew would offer a much more understandable way of putting it, which I look forward to.

Gary           

     
Title: Re: Optimize In Camera Jpeg White Balance RGB Histogram For Better ETTR.
Post by: digitaldog on December 18, 2018, 12:48:48 pm
Exposure simple?
"While intelligent people can often simplify the complex, a fool is more likely to complicate the simple."  -Gerald W. Grumet
Some of this discussion seems to back up that concept.
Those paying to subscribe and find this idea complex, this is a good start:
https://luminous-landscape.com/the-optimum-digital-exposure/ (https://luminous-landscape.com/the-optimum-digital-exposure/)
For Franz who may or may not be a subscriber (but should just for this):
Compare the noise from the camera’s “metered” exposure at 200 ISO (Section A) and the optimum exposure at 800 ISO (Section C). The noise from the optimum exposure at 800 ISO is the same or better than the 200 ISO at the metered exposure!   
As to ISO, depends on the camera design too:
https://photographylife.com/iso-invariance-explained
Title: Re: Optimize In Camera Jpeg White Balance RGB Histogram For Better ETTR.
Post by: faberryman on December 18, 2018, 01:31:44 pm
I imagine Andrew might jump in and perhaps deflate my theory to some extent at least.  However, it's the way I see the situation and understand what's actually happening when I raise the ISO setting.  It does not increase the amount of light striking the sensor.  The sensor is then set to gobble up more of it, and for the most part that's a good thing.  I'm also certain that Andrew would offer a much more understandable way of putting it, which I look forward to.
Except that as you increase ISO your camera meter tells you to use a faster shutter speed or smaller aperture; i.e. less exposure and less light striking the sensor.
Title: Re: Optimize In Camera Jpeg White Balance RGB Histogram For Better ETTR.
Post by: digitaldog on December 18, 2018, 01:37:41 pm
Except that as you increase ISO your camera meter tells you to use a faster shutter speed or smaller aperture; i.e. less exposure and less light striking the sensor.
Yes, IF you follow that 'recommendation' but as Bob's article and my example show, you don't have to do so.
It's akin to someone pointing a reflective meter at a white dog on snow or a black cat on coal and blindly accepting that exposure as being anything close to being correct or ideal. Meters (and resulting Histograms) can be easily fooled. Photographers who understand exposure, how differing meters work, what to meter on and how to test exposure for differing media don't get fooled. Going full circle as to why futzing with WB and JPEG Histograms along with easily fooled reflective meters is kind of a waste of time. Instead of simply testing the sensor/meter and viewing a raw Histogram to understand how to approach exposing in the field or otherwise, AFTER testing.
As many of us did with transparency film, decades before Histograms existed.
Title: Re: Optimize In Camera Jpeg White Balance RGB Histogram For Better ETTR.
Post by: faberryman on December 18, 2018, 02:14:29 pm
Yes, IF you follow that 'recommendation' but as Bob's article and my example show, you don't have to do so.
It's akin to someone pointing a reflective meter at a white dog on snow or a black cat on coal and blindly accepting that exposure as being anything close to being correct or ideal. Meters (and resulting Histograms) can be easily fooled. Photographers who understand exposure, how differing meters work, what to meter on and how to test exposure for differing media don't get fooled. Going full circle as to why futzing with WB and JPEG Histograms along with easily fooled reflective meters is kind of a waste of time. Instead of simply testing the sensor/meter and viewing a raw Histogram to understand how to approach exposing in the field or otherwise, AFTER testing.
As many of us did with transparency film, decades before Histograms existed.
I have been a photographer for 45 years, shot a lot of black and while film and slides in that time. I learned early on how to meter for correct exposure. Served me well as I transitioned to digital. And I still shoot film as well. Never had to go through the gyrations outlined in the OP to get correct exposure. If people thought more about their images and less about their technique, we all would be better off.
Title: Re: Optimize In Camera Jpeg White Balance RGB Histogram For Better ETTR.
Post by: nirpat89 on December 18, 2018, 02:42:28 pm
Best explained in this post from way back in 2011 and the diagram:
https://forum.luminous-landscape.com/index.php?topic=56906.msg466484#msg466484

You are basically "pushing" the first image (or "pulling" the second) to compare them side-by-side.  If the noise was not more in the former, then the high iso would be redundant. 
Title: Re: Optimize In Camera Jpeg White Balance RGB Histogram For Better ETTR.
Post by: digitaldog on December 18, 2018, 02:48:52 pm
You are basically "pushing" the first image (or "pulling" the second) to compare them side-by-side.  If the noise was not more in the former, then the high iso would be redundant.
The first is 'pushed' as it isn't an ideal exposure for raw as 'recommended' by the meter. The 2nd appears far too bright until (using Michaels term from the original ETTR article) normalized. Had the first been optimally exposed, it too would need to be normalized and wouldn't be as noisy. But the main take away is the incorrect concept that higher ISO always produces more noise as the OP implied.
Title: Re: Optimize In Camera Jpeg White Balance RGB Histogram For Better ETTR.
Post by: DP on December 18, 2018, 02:54:24 pm
as the ISO is increased the sensor's sensitivity to light increases
increasing the nominal ISO in a best case scenario (there are exceptions) leads to a bigger analog gain pre ADC - and that is benefical in many implementations with off sensor ADC mostly (see old Canon cameras)... or in modern processors (based on Aptina's patent) at some point in switching off extra capacitance that also decreases readout related noise (pre ADC)... exceptions are numerous - for example sometimes, at some point, in some camera cases increasing nominal ISO will read just to writing a different tag - instructing converter to do multiplication behind the scenes... or firmware might actually do that multiplication when writing raw data (post ADC)... etc, etc... what does not happen in sensors so far used in consumer cameras is that no "sensor's sensitivity to light" is changing... albeit we can't exclude some technologies in the future that indeed will do just that... but not now, sorry
Title: Re: Optimize In Camera Jpeg White Balance RGB Histogram For Better ETTR.
Post by: nirpat89 on December 18, 2018, 03:09:14 pm
The first is 'pushed' as it isn't an ideal exposure for raw as 'recommended' by the meter. The 2nd appears far too bright until (using Michaels term from the original ETTR article) normalized. Had the first been optimally exposed, it too would need to be normalized and wouldn't be as noisy. But the main take away is the incorrect concept that higher ISO always produces more noise as the OP implied.

I think when most people think about noise and iso, it is in relation to 2 shots of the same scene, one with a slower shutter and low iso and the other with faster speed and higher iso (keeping the same "metered" exposure level) the former will have lower noise than the latter.  Under those conditions, the common perception is right.  Hopefully so, otherwise I have been lugging my heavy tripod for nothing... :)
Title: Re: Optimize In Camera Jpeg White Balance RGB Histogram For Better ETTR.
Post by: digitaldog on December 18, 2018, 03:12:33 pm
I don't know what most people think about ISO, exposure, noise etc. I believe a fair number don't understand how they all work. ;)
Title: Re: Optimize In Camera Jpeg White Balance RGB Histogram For Better ETTR.
Post by: Aram Hăvărneanu on December 18, 2018, 05:37:46 pm
The only cameras for which this matters anymore are Canon cameras. Almost (all?) every other manufacturer switched to badly-called ISOless sensors.

You can see here exactly how much it matters for your camera: http://www.photonstophotos.net/Charts/PDR_Shadow.htm

In any case, I do not see the relevancy of this to the initial posting.
Title: Re: Optimize In Camera Jpeg White Balance RGB Histogram For Better ETTR.
Post by: digitaldog on December 18, 2018, 05:41:06 pm
That maybe so. There a heckuva of Lotta Canon cameras out there.  :D
Title: Re: Optimize In Camera Jpeg White Balance RGB Histogram For Better ETTR.
Post by: DP on December 18, 2018, 06:00:41 pm
The only cameras for which this matters anymore are Canon cameras. Almost (all?) every other manufacturer switched to badly-called ISOless sensors.
new Sony sensors are not ISO-less just because of the extra capacitance switch off at certain nominal ISO (hence a noticeable difference in readout noise upon the switch off) - hence truly truly ISO-less sensors are dying breed.... what you are referring to is the migration to multiple small simple ADCs on sensor vs few off sensor ADCs and hence the decrease of importance of analog pre ADC gain...
Title: Re: Optimize In Camera Jpeg White Balance RGB Histogram For Better ETTR.
Post by: Frans Waterlander on December 18, 2018, 07:00:06 pm
Best explained in this post from way back in 2011 and the diagram:
https://forum.luminous-landscape.com/index.php?topic=56906.msg466484#msg466484

It seems to me that that thread, and hence this one, is misleading on the issue of noise without some clarification. Yes, noise will be lower (but by how much differs from camera to camera) if correctly exposed at a higher ISO than underexposed (with the same shutter speed and aperture) at a lower ISO. But, noise (and things like posterizsation) will be lowest when correctly exposed at the lowest ISO setting. The term "correctly exposed" is of course open to further explanation/interpretation, like ER, ETTR, etc.
Title: Re: Optimize In Camera Jpeg White Balance RGB Histogram For Better ETTR.
Post by: digitaldog on December 18, 2018, 07:18:03 pm
It seems to me that that thread, and hence this one, is misleading on the issue of noise without some clarification.
Because you can't understand it, doesn't make it misleading.  ;)  You see no difference in noise between the two examples shown here or the multiple one's shown on the other thread? Another display issue?
Quote
Yes, noise will be lower (but by how much differs from camera to camera) if correctly exposed at a higher ISO than underexposed
Unless you love non-image forming data, (noise), lower is better. But then if you're OK under exposing your image data too, I suspect you're the kind of photographer (to be kind) that doesn't care much about noise either.
Some of us do. Some of us were professional photographers. Others are just serous about the craft and quality of their images and image data. Some of us believe in GIGO:Garbage In Garbage Out. Some here should stick to in camera JPEGs or cell phones for image capture.
Quote
But, noise (and things like posterizsation) will be lowest when correctly exposed at the lowest ISO setting.
Expect when it isn't with respect to ISO and noise as shown and apparently misunderstood by some. I'll refrain from commenting on whatever posterizsation is :P
Quote
The term "correctly exposed" is of course open to further explanation/interpretation, like ER, ETTR, etc.
Only for those unable to understand how to view a raw Histogram.
AS was outlined in the 2011 post, with the cameras that operate as shown, you've clearly missed the advantages of HIGHER ISO with less noise:
One may use the same aperture and shutter speed (because you are somehow restricted to those settings), it can be advantageous to increase ISO because it can reduce the noise as seen above!
Achieving so called "ETTR" through ISO only makes sense when exposure is inadequate. This can occur in many shooting conditions like low light conditions, large DOF requirement, quick action, etc.
Title: Re: Optimize In Camera Jpeg White Balance RGB Histogram For Better ETTR.
Post by: digitaldog on December 18, 2018, 07:27:36 pm
new Sony sensors are not ISO-less just because of the extra capacitance switch off at certain nominal ISO (hence a noticeable difference in readout noise upon the switch off) - hence truly truly ISO-less sensors are dying breed.... what you are referring to is the migration to multiple small simple ADCs on sensor vs few off sensor ADCs and hence the decrease of importance of analog pre ADC gain...

There a heckuva of Lotta Sony cameras out there too.
Good to know, just got one coming from years shooting Canon, will have to play around with a similar test.
Title: Re: Optimize In Camera Jpeg White Balance RGB Histogram For Better ETTR.
Post by: Tim Lookingbill on December 18, 2018, 11:29:52 pm
I have been a photographer for 45 years, shot a lot of black and while film and slides in that time. I learned early on how to meter for correct exposure. Served me well as I transitioned to digital. And I still shoot film as well. Never had to go through the gyrations outlined in the OP to get correct exposure. If people thought more about their images and less about their technique, we all would be better off.

Except in the digital world at least for me I set exposure to capture the entire tonal scale of a scene and to do this I have to adjust exposure to preserve highlights. They'ld rather mention UniWB which is not what I was talking about or asking them to indicate whether they've tried what I've indicated in the topic.

In camera Jpeg based histograms have been a quick guide for me to make this more easier to calculate exposure on the fly "chimping" the LCD but jpeg histograms don't indicate max head room on how far to let in more light that won't wipe out a lot of detail in the highlights as an example bilious white clouds on bright sunny day.

My point in posting this topic is whether other cameras that have more head room can now use the RGB histogram in pushing exposure. No one as expected would indicate if they've ever tried this since they're cameras have a wider dynamic range than my old Pentax.
Title: Re: Optimize In Camera Jpeg White Balance RGB Histogram For Better ETTR.
Post by: TonyW on December 19, 2018, 07:07:59 am

Except in the digital world at least for me I set exposure to capture the entire tonal scale of a scene and to do this I have to adjust exposure to preserve highlights. They'ld rather mention UniWB which is not what I was talking about or asking them to indicate whether they've tried what I've indicated in the topic.

In camera Jpeg based histograms have been a quick guide for me to make this more easier to calculate exposure on the fly "chimping" the LCD but jpeg histograms don't indicate max head room on how far to let in more light that won't wipe out a lot of detail in the highlights as an example bilious white clouds on bright sunny day.

My point in posting this topic is whether other cameras that have more head room can now use the RGB histogram in pushing exposure. No one as expected would indicate if they've ever tried this since they're cameras have a wider dynamic range than my old Pentax.
Good luck in capturing the entire tonal scale of a scene with the 200D: there is just not enough sensor DR for many scenes so you make a compromise sacrifice shadow detail to capture highlight detail or vice versa or add light to the scene or use HDR.

Its a shame when threads turn out to be argumentative and others PoV are denigrated.

Two mentions here of UniWB, dismissed as irrelevant when the purpose of the OP seemed to indicate looking for ways to make the JPEG histogram more useful.
Quote
UniWB is a "trick". A camera set up to use UniWB is instructed to produce a distorted JPEG image. That JPEG will typically look horribly green when viewed on a display. However, the distorted image is specifically designed to produce a histogram whose right hand extremity closely matches that of a real RAW histogram.

I do not have a 10 year old Pentax to make a comparison but AFAIK all DSLR offered some form of Custom WB and also probably some form of Picture mode e.g. Natural, Bright, Vibrant etc.
Any of which when implemented by the user alter the rendering of the raw WITH the obvious change to the histogram.  This does not change the exposure but merely the rendering. 

If you decide to use the camera LCD view including histogram as an indicator of raw exposure then you may be better selecting one of the seven scene modes  of the 200D, maybe Portrait or  Landscape bearing in mind that your raw converter of choice will have added its own values and results will be different e.g. Adobe to RT etc.
Title: Re: Optimize In Camera Jpeg White Balance RGB Histogram For Better ETTR.
Post by: bjanes on December 19, 2018, 07:31:05 am
And my ISO 800 images show LESS noise than ISO 100 in this case because ISO doesn't have anything to do with exposure; the amount of light striking the sensor. It does have something to do with dumb meters and not too intelligent photographers who don't understand how to expose:

This is a nice demonstration of optimizing shadow detail with a camera having less read noise as the ISO is raised, but this strategy works only with a short scale scene. With a full scale scene, the highlights wold be blown out.

Regards,

Bill
Title: Re: Optimize In Camera Jpeg White Balance RGB Histogram For Better ETTR.
Post by: nirpat89 on December 19, 2018, 09:31:39 am
This is a nice demonstration of optimizing shadow detail with a camera having less read noise as the ISO is raised, but this strategy works only with a short scale scene. With a full scale scene, the highlights wold be blown out.

Regards,

Bill

Bingo!
Title: Re: Optimize In Camera Jpeg White Balance RGB Histogram For Better ETTR.
Post by: digitaldog on December 19, 2018, 09:53:12 am
This is a nice demonstration of optimizing shadow detail with a camera having less read noise as the ISO is raised, but this strategy works only with a short scale scene. With a full scale scene, the highlights wold be blown out.

Regards,

Bill
Exposure 101 for digital don’t blow out the highlights.
Again, the demonstration is to illustrate that higher ISO doesn’t necessarily equate to more noise.
Edit. Don’t blow out the highlights you the photographer in control of your exposure do not wish to blow out!
Title: Re: Optimize In Camera Jpeg White Balance RGB Histogram For Better ETTR.
Post by: kirkt on December 19, 2018, 11:40:36 am
All of these issues have been thoughtfully described, with the pros and cons inherent in the process, here:

https://www.fastrawviewer.com/blog/in-camera-histogram-doesn%27t-represent-exposure

https://www.rawdigger.com/howtouse/exposure-for-raw-or-for-jpegs

https://www.rawdigger.com/howtouse/beware-histogram

https://www.rawdigger.com/howtouse/lightmeter-calibration

Putting aside for a moment the idea that the histogram is influenced by the JPEG produced by the user's cameras settings, there are some assumptions that one is making when attempting to establish the camera's histogram as a useful tool for evaluating raw exposure - the most important being the white balance itself, as this will dramatically change the per-channel "exposure" (brightness) indicated by the histogram relative to the actual exposure of the sensor.  In daylight, the green channel is a good proxy for evaluating sensor saturation as red and blue will channels will receive less exposure than green in the raw data.  But consider non-daylit scenes - the prevailing lighting will affect the usefulness of the histogram as well and may completely skew the histogram away from the underlying raw exposure when white balanced, either toward green or toward the "correct" white balance.  Finally, because we want to expose, more or less, the sensor for highlight detail, we need to be careful with respect to the actual color of the critical highlights (either the inherent color of the highlight surface or the lighting falling on and reflecting off of the surface) when evaluating the per-channel clipping.

Unless one understands how the JPEG histogram relates to the raw exposure, simply setting a green WB (akin to UniWB) will only get you so far - other in-camera settings will affect where the tones are placed on the histogram such that, even if one fixes a constant exposure (aperture and SS), changes to the in-camera settings that affect the JPEG (color profiles, DR settings, etc.) will change the JPEG histogram even when sensor exposure remains constant.

I have found that I can get optimal exposure 99% of the time by metering the critical highlights (and it works better if these are close to neutral, clouds being a good example) with a spot meter and adding approximately 3 to 3.3 EV to the metered reading (usually be changing SS).  On my Sekonic meter I can enter this offset as a filter compensation value and, at the press of a button, get the compensation in SS that I need to expose for the metered highlights.

If you really want to push things to the edge, you can test your various raw converters and see how each responds to clipping in one or more channels (raw saturation) and evaluate how much extra compensation you can get away with if the highlights are neutral and the raw converter can RECONSTRUCT the partially clipped highlights from the intact raw channel data.  In some circumstances I can dial in over 4EV on my Fujifilm X-H1 with CaptureOne and reconstruct neutral highlights, for example.  I do not advise setting exposure based on your raw converter though, but it is good to know what result a combination of camera exposure plus software will give you for the times when you really need to turn exposure up to 11.

All of this is predicated upon studying the content of the raw data using Raw Digger or a similar tool.

Kirk
Title: Re: Optimize In Camera Jpeg White Balance RGB Histogram For Better ETTR.
Post by: faberryman on December 19, 2018, 12:01:40 pm
I have found that I can get optimal exposure 99% of the time by metering the critical highlights (and it works better if these are close to neutral, clouds being a good example) with a spot meter and adding approximately 3 to 3.3 EV to the metered reading (usually be changing SS).  On my Sekonic meter I can enter this offset as a filter compensation value and, at the press of a button, get the compensation in SS that I need to expose for the metered highlights.
In the old days we would call this placing the highlights in Zone VIII and letting the shadows fall where they may. It is how you expose transparency film. The more things change, the more they stay the same. Hardly an epiphany.
Title: Re: Optimize In Camera Jpeg White Balance RGB Histogram For Better ETTR.
Post by: Tim Lookingbill on December 19, 2018, 12:43:14 pm
Good luck in capturing the entire tonal scale of a scene with the 200D: there is just not enough sensor DR for many scenes so you make a compromise sacrifice shadow detail to capture highlight detail or vice versa or add light to the scene or use HDR.

Its a shame when threads turn out to be argumentative and others PoV are denigrated.

Two mentions here of UniWB, dismissed as irrelevant when the purpose of the OP seemed to indicate looking for ways to make the JPEG histogram more useful.
I do not have a 10 year old Pentax to make a comparison but AFAIK all DSLR offered some form of Custom WB and also probably some form of Picture mode e.g. Natural, Bright, Vibrant etc.
Any of which when implemented by the user alter the rendering of the raw WITH the obvious change to the histogram.  This does not change the exposure but merely the rendering. 

If you decide to use the camera LCD view including histogram as an indicator of raw exposure then you may be better selecting one of the seven scene modes  of the 200D, maybe Portrait or  Landscape bearing in mind that your raw converter of choice will have added its own values and results will be different e.g. Adobe to RT etc.

I know my Pentax can't capture all the tonal scale of a scene. I wrote that in my response! But thanks for writing out such a lengthy response to something I didn't say. It must've taken a lot  of effort and I appreciate it.

Now to ask again about YOUR camera and whether you've tried the jpeg RGB histogram experiment in my OP since I'm ASSUMING your camera is more modern and can take in MORE DYNAMIC RANGE THAN MY OLD PENTAX.

Let's see if I finally get one of you to answer this which was pointed out in my OP.
Title: Re: Optimize In Camera Jpeg White Balance RGB Histogram For Better ETTR.
Post by: digitaldog on December 19, 2018, 12:49:21 pm
Let's see if I finally get one of you to answer this which was pointed out in my OP.
This question was answered here (https://forum.luminous-landscape.com/index.php?topic=128092.msg1083987#msg1083987) long ago:

Quote
Has anyone tried this?
Title: Re: Optimize In Camera Jpeg White Balance RGB Histogram For Better ETTR.
Post by: TonyW on December 19, 2018, 01:03:26 pm
I know my Pentax can't capture all the tonal scale of a scene. I wrote that in my response! But thanks for writing out such a lengthy response to something I didn't say. It must've taken a lot  of effort and I appreciate it.
Sarcasm really you are already on a loser here why continue?  What do you think you did not say?

Quote
Now to ask again about YOUR camera and whether you've tried the jpeg RGB histogram experiment in my OP since I'm ASSUMING your camera is more modern and can take in MORE DYNAMIC RANGE THAN MY OLD PENTAX....
NO, I HAVE NOT TRIED NOR DO I NEED TO TRY YOUR VOODOO EXPERIMENTS. going down another damn pointless rabbit hole

Title: Re: Optimize In Camera Jpeg White Balance RGB Histogram For Better ETTR.
Post by: Tim Lookingbill on December 19, 2018, 02:11:07 pm
Sarcasm really you are already on a loser here why continue?  What do you think you did not say?
NO, I HAVE NOT TRIED NOR DO I NEED TO TRY YOUR VOODOO EXPERIMENTS. going down another damn pointless rabbit hole

Thanks for the confirmation and reply. Very helpful.
Title: Re: Optimize In Camera Jpeg White Balance RGB Histogram For Better ETTR.
Post by: bjanes on December 19, 2018, 02:21:09 pm
Exposure 101 for digital don’t blow out the highlights.
Again, the demonstration is to illustrate that higher ISO doesn’t necessarily equate to more noise.
Edit. Don’t blow out the highlights you the photographer in control of your exposure do not wish to blow out!

Of course, you don't want to blow out important highlights. However, if your scene dynamic range is 9 stops and the DR of your camera is only 8 stops, as it is with your camera at ISO 800 (according to Bill Claff), you can't capture the full DR of the scene. Depending on the subject, you can preserve the highlights or the shadows. However, if you had used base ISO you could have captured the full DR of the scene without clipping of highlights.
Title: Re: Optimize In Camera Jpeg White Balance RGB Histogram For Better ETTR.
Post by: digitaldog on December 19, 2018, 02:27:55 pm
Of course, you don't want to blow out important highlights. However, if your scene dynamic range is 9 stops and the DR of your camera is only 8 stops, as it is with your camera at ISO 800 (according to Bill Claff), you can't capture the full DR of the scene. Depending on the subject, you can preserve the highlights or the shadows. However, if you had used base ISO you could have captured the full DR of the scene without clipping of highlights.
I don't think anyone, certainly not I dispute this. The entire discussion about ISO 800 (higher ISO) is to dismiss the OP's comments and a general concept (also mentioned) that higher ISO equates to higher noise. For a lot of cameras, that isn't the case. There were also those useful comments from the 2011 post about where a higher ISO with less noise can be useful. Certainly one should take DR into account in such cases.This is a technique that isn't going to work for everyone in every situation. But it can be useful in some cases and more importantly dismisses the wrong idea about high ISO and noise. It's also important to understand your comments about DR and blowing out highlights; photographers simply have to understand how their specific tools work (and why). Some here clearly don't. So yeah, DR, ISO, what meters assume how differing meter technology operate (Reflective vs. Incident), and how to avoid clipping in raw are all important to understand for photographers who wish to obtain the highest quality captures. And far more important IMHO than attempting silly hacks with JPEG Histograms for raw shooters.
I suppose we could ask, why do we still not have a raw camera Histogram (with exception of a firmware 'hack' for Canon)?
Much of the above was true for film.
Title: Re: Optimize In Camera Jpeg White Balance RGB Histogram For Better ETTR.
Post by: TonyW on December 19, 2018, 02:32:58 pm
Thanks for the confirmation and reply. Very helpful.
So pleased to hear that you found this very helpful!
Title: Re: Optimize In Camera Jpeg White Balance RGB Histogram For Better ETTR.
Post by: kirkt on December 19, 2018, 06:47:36 pm
In the old days we would call this placing the highlights in Zone VIII and letting the shadows fall where they may. It is how you expose transparency film. The more things change, the more they stay the same. Hardly an epiphany.

No epiphany, but a simple reality of exposing for a digital sensor.

Computers aren’t an epiphany either, just a bunch of switches controlling the flow of electrons.

Kirk
Title: Re: Optimize In Camera Jpeg White Balance RGB Histogram For Better ETTR.
Post by: DP on December 20, 2018, 05:40:17 am
It seems to me that that thread, and hence this one, is misleading on the issue of noise without some clarification. Yes, noise will be lower (but by how much differs from camera to camera) if correctly exposed at a higher ISO than underexposed (with the same shutter speed and aperture) at a lower ISO.

I'd say this way: with the equal exposures S/N in deep shadows can be increased (when/if possible - for example w/o clipping important highlights) by selecting a certain higher nominal ISO values (before shooting !) _BUT_ more exposure when / if possible is always better than using higher gains (higher nominal ISOs) ...
Title: Re: Optimize In Camera Jpeg White Balance RGB Histogram For Better ETTR.
Post by: Garnick on December 20, 2018, 08:41:51 am
"BUT_ more exposure when / if possible is always better than using higher gains (higher nominal ISOs)"

I agree absolutely, but of course the caveat "if possible" always lingers in the background.  I have bracketing sequences set up for situations that I deem "possible" for using the "Optimal Exposure" routine as mentioned in Bob DiNitale's excellent article.  And, if the exposure I want to use for that shot has clipped some important area I simply stack it with the one containing that information and integrate it in the final image.  More or less akin to HDR I suppose, but it works quite well when necessary.  As a matter of fact, although somewhat OT, about 12 years ago I was scanning a number of Kodachromes of the slot canyons in the south west US.  As you might imagine, the suns rays were almost blown out at the top of the canyon.  I "underexposed" those chromes buy approximately 1.5 EV along with the proper exposure and stacked them.  I was then able to brush in the otherwise blown out areas very successfully.  Long before HDR, but it worked very well. 

Gary       
Title: Re: Optimize In Camera Jpeg White Balance RGB Histogram For Better ETTR.
Post by: bjanes on December 20, 2018, 09:46:16 am
I'd say this way: with the equal exposures S/N in deep shadows can be increased (when/if possible - for example w/o clipping important highlights) by selecting a certain higher nominal ISO values (before shooting !) _BUT_ more exposure when / if possible is always better than using higher gains (higher nominal ISOs) ...

+1. These special techniques are often used when more exposure is not feasible because of depth of field considerations or freezing of action requires decreasing the aperture (larger f/stop) and/or using faster shutter speeds. In this situation with an ISO-invariant camera such as the Nikon D800 with ISOs above 400, it us often advantageous to expose as much as possible using the aperture and shutter speed that conditions require and then use a lower than indicated ISO in order to protect the highlights. One then brightens the image in post. Jim Kasson discusses this strategy here (https://blog.kasson.com/d850/d850-exposure-strategy-manual-settings/).

This confirms Andrew's assertion that ISO has nothing to do with exposure, although a higher ISO will result in less exposure if one uses the metered value. Of course, one does not have to use the metered exposure. Andrew does not like the term ETTR and instead prefers "proper exposure", but the latter term does not indicate how that exposure is obtained. Jim restricts use of the term ETTR to base ISO. Above base ISO when one uses the smaller aperture and the faster shutter speed that conditions require as discussed above, moving the histogram to the right by increasing the ISO does not affect exposure, but it does reduce highlight headroom.

Regards,

Bill
Title: Re: Optimize In Camera Jpeg White Balance RGB Histogram For Better ETTR.
Post by: DP on December 20, 2018, 10:09:37 am
ISO has nothing to do with exposure
probably people shall start saying that "ISO" (quote marks intended) with the current state of technology in digital cameras is a part of a decision process about the exposure (while not part of that exposure process) which is done before exposure itself starts by a human operator and/or firmware based on metering and some other observations/experience/etc ... and we still can't say that never ever a technology will come that really change the "sensitivity" of the sensor or the amount of light reaching it (during the same exposure time & aperture setting) with a simple dial of ISO control... may be one day it happens... and then what ?
Title: Re: Optimize In Camera Jpeg White Balance RGB Histogram For Better ETTR.
Post by: digitaldog on December 20, 2018, 01:05:06 pm

Setting ISO speed does not change the sensitivity of the sensor to light (Photons striking it), just like volume control does not change the sensitivity of a radio receiving it's signal. In each cases the setting (ISO or volume) controls only the signal processing, while the input stage (sensor, antenna) provides the same input signal.
That's why when ISO setting is cranked up, automatic exposure can result in more noise - automatic exposure in this case decreases the exposure (that is, the combination of aperture and shutter speed is set to allow less light captured by the sensor). Less exposure, less light, more noise. Again, avoid automatic exposure; it's easily fooled. No Histogram necessary. For Franz who may have left the building (or Tim), in the case of my ISO 800vs.100 example, I didn't follow the automatic exposure for the ISO 800 capture and due to amplification in this type of camera, less noise.
Title: Re: Optimize In Camera Jpeg White Balance RGB Histogram For Better ETTR.
Post by: DP on December 20, 2018, 01:42:57 pm
Setting ISO speed does not change the sensitivity of the sensor to light

with the fine print - with the current technology ... you can't be sure what comes next or may be already lurking in the labs ...

PS: a funny example when somebody can claim that ISO is actually a part of exposure... shooting with speedlites (or strobes) with TTL... well there ISO you dial actually does control (TTL !) the amount of light emitted by speedlite/strobe and thus hitting the sensor with fixed exposure time and fixed aperture  ;D
Title: Re: Optimize In Camera Jpeg White Balance RGB Histogram For Better ETTR.
Post by: digitaldog on December 20, 2018, 01:47:24 pm
with the fine print - with the current technology ... you can't be sure what comes next or may be already lurking in the labs ...
Yeah, like anti-gravity machines and photon torpedoes; you're right. Let's stick to what's available to the people posting here today if we can.  ;)
Title: Re: Optimize In Camera Jpeg White Balance RGB Histogram For Better ETTR.
Post by: Garnick on December 20, 2018, 03:30:31 pm
Setting ISO speed does not change the sensitivity of the sensor to light (Photons striking it), just like volume control does not change the sensitivity of a radio receiving it's signal. In each cases the setting (ISO or volume) controls only the signal processing, while the input stage (sensor, antenna) provides the same input signal.
That's why when ISO setting is cranked up, automatic exposure can result in more noise - automatic exposure in this case decreases the exposure (that is, the combination of aperture and shutter speed is set to allow less light captured by the sensor). Less exposure, less light, more noise. Again, avoid automatic exposure; it's easily fooled. No Histogram necessary. For Franz who may have left the building (or Tim), in the case of my ISO 800vs.100 example, I didn't follow the automatic exposure for the ISO 800 capture and due to amplification in this type of camera, less noise.

Hmmm...I was expecting that Andrew would correct me on part of the two replies I offered, but I wasn't expecting that it would necessarily be this part.  The ISO part of my replies was from an article I read probably 5 years ago or more.  I have always remembered that and its association with why and how Film ISO actually works.  Therefore, since I am not a tech, nor have I ever pretended to be, it did seem plausible to me at the time.  Since I read this latest offering from Andrew I have been trying to locate that article, but to no avail unfortunately.  However, that is not important now, since it obviously contained some rather  egregious errors.  This particular topic had never been a focal point of my experience while using a digital DSLR until I read Bob DiNitale's article here on LuLa a couple of years ago, and then following his routine to prove it to myself.  Indeed, "the proof was in the pudding", or in Bob's case, "the milk".  Of course only those who have actually read the article will understand my reference to "milk". 

Thanks for correcting my error Andrew, and your explanation is certainly very understandable.  The only downside is that from this point on I'll have one more setting to deal with and hope I haven't overcooked that file.  But I have also "overexposed" from the meter reading by two stops and been able to bring it back with no apparent issues, and definitely NO noise that I can determine.  In ACR, if one is not pulling the "exposure" slider to a negative value, the file has not yet reached Optimum Exposure.  That's my personal standard for evaluating the exposure of an image.

Gary         
Title: Re: Optimize In Camera Jpeg White Balance RGB Histogram For Better ETTR.
Post by: DP on December 20, 2018, 04:40:36 pm
Yeah, like anti-gravity machines and photon torpedoes; you're right. Let's stick to what's available to the people posting here today if we can.  ;)
still with all those organic sensors in the making (like Panasonic/Fuji) - there is no assurance that sensitiviy of the sensor can't be changed... or Eric Fossum's new toy... where you can in fact control what an individual sensel (they call it - jot) will do (count or not) with an individual incoming photon so to say... so you can basically from the practical standpoint "control sensitivity" of the sensor based on those jots differently across the frame ___DURING THE EXPOSURE TIME__ ... and they have working prototypes - it is not photon torpedo
Title: Re: Optimize In Camera Jpeg White Balance RGB Histogram For Better ETTR.
Post by: digitaldog on December 20, 2018, 04:47:40 pm
still with all those organic sensors in the making (like Panasonic/Fuji) - there is no assurance that sensitiviy of the sensor can't be changed... or Eric Fossum's new toy... where you can in fact control what an individual sensel (they call it - jot) will do (count or not) with an individual incoming photon so to say... so you can basically from the practical standpoint "control sensitivity" of the sensor based on those jots differently across the frame ___DURING THE EXPOSURE TIME__ ... and they have working prototypes - it is not photon torpedo
https://futurism.com/we-might-actually-be-able-to-make-star-treks-photon-torpedoes
Title: Re: Optimize In Camera Jpeg White Balance RGB Histogram For Better ETTR.
Post by: Aram Hăvărneanu on December 20, 2018, 04:55:11 pm
with the fine print - with the current technology ... you can't be sure what comes next or may be already lurking in the labs ...

The quantum efficiency of modern sensors is something like 70%. If you could control the sensitivity to light, you could only make it less sensitive.
Title: Re: Optimize In Camera Jpeg White Balance RGB Histogram For Better ETTR.
Post by: Guillermo Luijk on December 20, 2018, 05:16:31 pm
A flaw of any technique trying to use the JPEG histogram as a proxy to mimic the genuine RAW histogram (of which UniWB is probably the most effective generally speaking), is that in the end of the camera RAW processing pipeline, a colour conversion from camera space (or whatever we like to call RAW data) to an output colour space (sRGB or Adobe RGB) takes place.

This 3D matrix conversion makes all three RGB input channels to individually contribute to each of the three output channels. Being G the most interesting (since it predominantly encodes higher RAW values), we could specify each output G value as a linear combination:

G_out = k1*R_in + k2*G_in + k3*B_in

G_out=1 could correspond to a G_in<1 (false positive RAW clipping)

G_out<1 could correspond to a G_in=1 (false negative RAW clipping)

In other words, no matter what we do there is never a guarantee of having a reliable RAW clipping warning through JPEG histograms.

Regards
Title: Re: Optimize In Camera Jpeg White Balance RGB Histogram For Better ETTR.
Post by: faberryman on December 20, 2018, 05:35:00 pm
In other words, no matter what we do there is never a guarantee of having a reliable RAW clipping warning through JPEG histograms.
Let's remember there are no guarantees in life about anything, and we just do the best job we can. I don't have a problem with blown highlights very often so I can't get too excited about all this. I bet most members don't routinely have these problems either. Maybe if you have a ten year old camera with limited dynamic you have to be more careful. I wish people would worry more about their composition and less about their histogram.
Title: Re: Optimize In Camera Jpeg White Balance RGB Histogram For Better ETTR.
Post by: digitaldog on December 20, 2018, 05:49:49 pm
Let's remember there are no guarantees in life about anything, and we just do the best job we can. I don't have a problem with blown highlights very often so I can't get too excited about all this. I bet most members don't routinely have these problems either. Maybe if you have a ten year old camera with limited dynamic you have to be more careful. I wish people would worry more about their composition and less about their histogram.
I believe his point is that one does not need to rely on a JPEG histogram to optimally expose raw. The history of photography seems to clearly back up that concept concerning exposure.
Title: Re: Optimize In Camera Jpeg White Balance RGB Histogram For Better ETTR.
Post by: TonyW on December 20, 2018, 06:17:44 pm
Let's remember there are no guarantees in life about anything, and we just do the best job we can. I don't have a problem with blown highlights very often so I can't get too excited about all this. I bet most members don't routinely have these problems either. Maybe if you have a ten year old camera with limited dynamic you have to be more careful. I wish people would worry more about their composition and less about their histogram.
I do not think this is solely or particularly about blown highlights, rather more about optimal exposure for digital capture including that of the scene DR fitting well inside the cameras DR capacity where you really have a chance to optimise exposure moving as far right as possible prior to clipping.  How important this is for noise reduction with modern sensors and processing algorithms I do not know but suspect not quite as much as in the past

My 10 year old Nikon had limited DR capability maybe 3-4 EV less than sensors of the last few years, but still more range than most transparency film IIRC and probably a little less than colour neg material and almost certainly less than the best B&W film/print combination

Title: Re: Optimize In Camera Jpeg White Balance RGB Histogram For Better ETTR.
Post by: nirpat89 on December 20, 2018, 07:31:11 pm
A practical question about the in-camera raw histogram if there was such a thing, how would one use it?  It would still have to be displayed on a scale so in order to know what is going on beyond this scale, one would need some way to handle the histogram as you would in CameraRaw or Lr.  Kind of awkward fidgeting with the camera display trying to figure out if you nailed the ETTR while shooting on location.  No?
Title: Re: Optimize In Camera Jpeg White Balance RGB Histogram For Better ETTR.
Post by: digitaldog on December 20, 2018, 07:35:25 pm
A practical question about the in-camera raw histogram if there was such a thing, how would one use it?  It would still have to be displayed on a scale so in order to know what is going on beyond this scale, one would need some way to handle the histogram as you would in CameraRaw or Lr.  Kind of awkward fidgeting with the camera display trying to figure out if you nailed the ETTR while shooting on location.  No?


Some examples from the way done on Canon with Magic Lantern:

http://hdr-photographer.com/2014/10/why-i-love-magic-lantern-several-years-later/ (http://hdr-photographer.com/2014/10/why-i-love-magic-lantern-several-years-later/)
https://www.magiclantern.fm/forum/index.php?topic=12096.0 (https://www.magiclantern.fm/forum/index.php?topic=12096.0)


Again, assuming you MUST have a Histogram to expose.  ;)
Title: Re: Optimize In Camera Jpeg White Balance RGB Histogram For Better ETTR.
Post by: Guillermo Luijk on December 20, 2018, 07:48:38 pm
A practical question about the in-camera raw histogram if there was such a thing, how would one use it?  It would still have to be displayed on a scale so in order to know what is going on beyond this scale, one would need some way to handle the histogram as you would in CameraRaw or Lr.  Kind of awkward fidgeting with the camera display trying to figure out if you nailed the ETTR while shooting on location.  No?

My preferred format would be log X axis (with EV divisions) + linear Y axis. Histograms like this would be tremendously intuitive giving precise information about the adjustments needed for accurate optimum exposure (ETTR):

(https://4.bp.blogspot.com/-XIK2EOPW3C4/WvIWVVwKG1I/AAAAAAAADdE/7HWZgFkIsCYBw5iPdWhnmplP0ZlB7uG3ACLcBGAs/s1600/histogramaraw1.png)

Just a glance to find out that adding +1,5EV means perfect ETTR here. Or just +1EV if you don't even want to sacrifice those very few G photosites.

Such histogram also provides a precise DR measurement of the scene; 8 stops in this case (number of vertical divisions with relevant content). A camera with N-bit RAW files encoding can measure DR of up to N stops, no matter if the sensor's capture DR is lower than N stops.

Regards
Title: Re: Optimize In Camera Jpeg White Balance RGB Histogram For Better ETTR.
Post by: nirpat89 on December 20, 2018, 08:08:30 pm
My preferred format would be log X axis (with EV divisions) + linear Y axis. Histograms like this would be tremendously intuitive giving precise information about the adjustments needed for accurate optimum exposure (ETTR):

(https://4.bp.blogspot.com/-XIK2EOPW3C4/WvIWVVwKG1I/AAAAAAAADdE/7HWZgFkIsCYBw5iPdWhnmplP0ZlB7uG3ACLcBGAs/s1600/histogramaraw1.png)

Just a glance to find out that adding +1,5EV means perfect ETTR here. Or just +1EV if you don't even want to sacrifice those very few G photosites.

Such histogram also provides a precise DR measurement of the scene; 8 stops in this case (number of vertical divisions with relevant content). A camera with N-bit RAW files encoding can measure DR of up to N stops, no matter if the sensor's capture DR is lower than N stops.

Regards

That makes perfect sense.  Thanks.  I hope Nikon is consulting you... :)
Title: Re: Optimize In Camera Jpeg White Balance RGB Histogram For Better ETTR.
Post by: nirpat89 on December 20, 2018, 08:17:20 pm

Some examples from the way done on Canon with Magic Lantern:

http://hdr-photographer.com/2014/10/why-i-love-magic-lantern-several-years-later/ (http://hdr-photographer.com/2014/10/why-i-love-magic-lantern-several-years-later/)
https://www.magiclantern.fm/forum/index.php?topic=12096.0 (https://www.magiclantern.fm/forum/index.php?topic=12096.0)


Again, assuming you MUST have a Histogram to expose.  ;)

Thanks, Andrew.  Had never heard of Magic Lantern.  It is not for Nikons, unfortunately, as yet so not useful for me, though.   May be some day.

Title: Re: Optimize In Camera Jpeg White Balance RGB Histogram For Better ETTR.
Post by: digitaldog on December 20, 2018, 08:44:50 pm
Thanks, Andrew.  Had never heard of Magic Lantern.  It is not for Nikons, unfortunately, as yet so not useful for me, though.   May be some day.
Another reason to simply learn to expose optimally without a Histogram. A raw Histogram could be useful. It's absolutely not required. Kodachrome, Velvia etc, didn't have one and somehow, many of us managed to expose that stuff.
Title: Re: Optimize In Camera Jpeg White Balance RGB Histogram For Better ETTR.
Post by: TonyW on December 21, 2018, 03:05:29 am
.....
Again, assuming you MUST have a Histogram to expose.  ;)
;D Agreed, although a nice to have feature it may be viewed as the post-mortem/inquest after the decisive moment has long passed
Title: Re: Optimize In Camera Jpeg White Balance RGB Histogram For Better ETTR.
Post by: TonyW on December 21, 2018, 03:24:23 am
My preferred format would be log X axis (with EV divisions) + linear Y axis. Histograms like this would be tremendously intuitive giving precise information about the adjustments needed for accurate optimum exposure (ETTR):

(https://4.bp.blogspot.com/-XIK2EOPW3C4/WvIWVVwKG1I/AAAAAAAADdE/7HWZgFkIsCYBw5iPdWhnmplP0ZlB7uG3ACLcBGAs/s1600/histogramaraw1.png)

Just a glance to find out that adding +1,5EV means perfect ETTR here. Or just +1EV if you don't even want to sacrifice those very few G photosites.

Such histogram also provides a precise DR measurement of the scene; 8 stops in this case (number of vertical divisions with relevant content). A camera with N-bit RAW files encoding can measure DR of up to N stops, no matter if the sensor's capture DR is lower than N stops.

Regards
Oh yes, I like that and would welcome the option to choose this type of display during capture.  If only there was a strong enough photographic lobby to overcome camera manufacturers inertia in this area.  I am making an assumption that currently no manufacturer has the option to view a histogram giving useful raw information.

As a matter of interest what software produced your posted image?
Title: Re: Optimize In Camera Jpeg White Balance RGB Histogram For Better ETTR.
Post by: Guillermo Luijk on December 21, 2018, 03:35:28 am
Oh yes, I like that and would welcome the option to choose this type of display during capture.  If only there was a strong enough photographic lobby to overcome camera manufacturers inertia in this area.  I am making an assumption that currently no manufacturer has the option to view a histogram giving useful raw information.

As a matter of interest what software produced your posted image?

Canon cameras with Magic Lantern do have RAW histograms as Andrew showed. I also believe Leica Monochrome can be considered to have RAW histograms in terms of exposure, since their monochrome histogram clipping reflects that of the RAW. Not 100% sure of that though.

Obtaining a real RAW histogram is trivial, camera manufacturers just don't implement it:

http://www.elmomentodecisivo.com/2018/05/histogramas-raw-en-camaras-digitales.html

That histogram was created with this piece of software I wrote time ago:

http://www.guillermoluijk.com/tutorial/histogrammar/index.htm

It can input DCRAW decoded RAW files (-D and -d options) and plot them in different styles.

Regards
Title: Re: Optimize In Camera Jpeg White Balance RGB Histogram For Better ETTR.
Post by: TonyW on December 21, 2018, 03:50:54 am
Canon cameras with Magic Lantern do have RAW histograms as Andrew showed. I also believe Leica Monochrome can be considered to have RAW histograms in terms of exposure, since their monochrome histogram clipping reflects that of the RAW. Not 100% sure of that though.

Obtaining a real RAW histogram is trivial, camera manufacturers just don't implement it:

http://www.elmomentodecisivo.com/2018/05/histogramas-raw-en-camaras-digitales.html

That histogram was created with this piece of software I wrote time ago:

http://www.guillermoluijk.com/tutorial/histogrammar/index.htm

It can input DCRAW decoded RAW files (-D and -d options) and plot them in different styles.

Regards
Thank you.  I will look at your links in detail.

Lamento decir que mi español es muy limitado. Pero con la ayuda de Google, estoy deseando leer tu artículo.
Title: Re: Optimize In Camera Jpeg White Balance RGB Histogram For Better ETTR.
Post by: bjanes on December 21, 2018, 09:20:27 am
A flaw of any technique trying to use the JPEG histogram as a proxy to mimic the genuine RAW histogram (of which UniWB is probably the most effective generally speaking), is that in the end of the camera RAW processing pipeline, a colour conversion from camera space (or whatever we like to call RAW data) to an output colour space (sRGB or Adobe RGB) takes place.

This 3D matrix conversion makes all three RGB input channels to individually contribute to each of the three output channels. Being G the most interesting (since it predominantly encodes higher RAW values), we could specify each output G value as a linear combination:

G_out = k1*R_in + k2*G_in + k3*B_in

G_out=1 could correspond to a G_in<1 (false positive RAW clipping)

G_out<1 could correspond to a G_in=1 (false negative RAW clipping)

In other words, no matter what we do there is never a guarantee of having a reliable RAW clipping warning through JPEG histograms.

Regards

Guillermo,

An excellent post. The WB multipliers and matrix coefficients for the Nikon D850 according to DXO are shown below. It would be an interesting exercise to take some shots of varying subjects with the D850 set to record both raw and sRBG, read the raw values in Rawdigger, apply the white balance, apply the matrix for sRGB, and then convert from linear gamma to sRGB gamma. Would this be technically sound? I presume one would apply the WB before the matrix coefficients--is this correct?



Best regards,

Bill
Title: Re: Optimize In Camera Jpeg White Balance RGB Histogram For Better ETTR.
Post by: digitaldog on December 21, 2018, 09:39:16 am
Guillermo,

An excellent post. The WB multipliers and matrix coefficients for the Nikon D850 according to DXO are shown below. It would be an interesting exercise to take some shots of varying subjects with the D850 set to record both raw and sRBG, read the raw values in Rawdigger, apply the white balance, apply the matrix for sRGB, and then convert from linear gamma to sRGB gamma. Would this be technically sound? I presume one would apply the WB before the matrix coefficients--is this correct?



Best regards,

Bill
When people say optimal exposure is hard, I would ask "Compared to what?" Now I know. ;D
Title: Re: Optimize In Camera Jpeg White Balance RGB Histogram For Better ETTR.
Post by: faberryman on December 21, 2018, 07:20:08 pm
An excellent post. The WB multipliers and matrix coefficients for the Nikon D850 according to DXO are shown below. It would be an interesting exercise to take some shots of varying subjects with the D850 set to record both raw and sRBG, read the raw values in Rawdigger, apply the white balance, apply the matrix for sRGB, and then convert from linear gamma to sRGB gamma. Would this be technically sound? I presume one would apply the WB before the matrix coefficients--is this correct?
The first question I would ask is whether the image itself is really worth all that rigamarole.
Title: Re: Optimize In Camera Jpeg White Balance RGB Histogram For Better ETTR.
Post by: DP on December 21, 2018, 08:25:12 pm
In other words, no matter what we do there is never a guarantee of having a reliable RAW clipping warning through JPEG histograms.
in theory no, in practice though UniWB + flattest contrast curve + widest gamut output colorspace (AdobeRGB vs sRGB, but even with sRGB) + blinkies/zebra reliably show clipping in raw within < 1/3 EV precision as checked with RawDigger under tungsten to sun to blue skies/cloudy daylight illumination (and consumer type LED bulbs & fluorescents) and that I use on my Olympus and later Sony dSLMs for years...
Title: Re: Optimize In Camera Jpeg White Balance RGB Histogram For Better ETTR.
Post by: DP on December 21, 2018, 08:26:16 pm
The quantum efficiency of modern sensors is something like 70%.
I am not talking about "modern" sensors (as used in current cameras) if you did not notice
Title: Re: Optimize In Camera Jpeg White Balance RGB Histogram For Better ETTR.
Post by: Aram Hăvărneanu on December 21, 2018, 08:29:48 pm
The point is that sensors can't significantly improve in sensitivity anymore, making such a feature mostly pointless.
Title: Re: Optimize In Camera Jpeg White Balance RGB Histogram For Better ETTR.
Post by: Osprey on December 21, 2018, 09:31:06 pm
Is UniWB still used?  I remember that green tinted look on my Nikon D200.
Title: Re: Optimize In Camera Jpeg White Balance RGB Histogram For Better ETTR.
Post by: TonyW on December 22, 2018, 09:28:47 am
The point is that sensors can't significantly improve in sensitivity anymore, making such a feature mostly pointless.
I think I lost track somewhere along the way so can I ask in your opinion which feature would be mostly pointless?
Title: Re: Optimize In Camera Jpeg White Balance RGB Histogram For Better ETTR.
Post by: Aram Hăvărneanu on December 22, 2018, 09:57:56 am
If there were a way for the ISO setting to somehow physically change the sensor sensitivity to light.
Title: Re: Optimize In Camera Jpeg White Balance RGB Histogram For Better ETTR.
Post by: Alan Klein on December 22, 2018, 02:52:41 pm
How much more do you get or need by maximizing ETTR?  I shoot Velvia 50 film and I'm just happy when I don't clip the whites.  I usually bracket to make sure.  I really don;t care about the shadows.  Whatever details are there are usually OK.  Normal people don;t look there anyway.  They're looking at the brighter areas where the subject is.  Maximizing details and measuring DR in the shadows is a form of pixel peeping. 
Title: Re: Optimize In Camera Jpeg White Balance RGB Histogram For Better ETTR.
Post by: Guillermo Luijk on December 22, 2018, 03:52:04 pm
How much more do you get or need by maximizing ETTR?

In the shadows, where electronic noise is the most relevant, SNR is doubled for every extra stop of exposure. Four extra stops means increasing SNR by 2^4=16 like here:

(http://www.guillermoluijk.com/article/nonoise/sillon.jpg)

Regards
Title: Re: Optimize In Camera Jpeg White Balance RGB Histogram For Better ETTR.
Post by: Alan Klein on December 22, 2018, 04:16:21 pm
If the picture on the left represents the whole picture without a crop, then it wasn't exposed properly in the first place.
Title: Re: Optimize In Camera Jpeg White Balance RGB Histogram For Better ETTR.
Post by: Guillermo Luijk on December 22, 2018, 04:33:49 pm
If the picture on the left represents the whole picture without a crop, then it wasn't exposed properly in the first place.
Both are 100% crops of the shadows of an entire scene. Their only purpose was to answer to your question "How much more do you get or need by maximizing ETTR?". You get a 2x factor in SNR in the shadows for every extra stop of exposure. If ETTR helps you to get more exposure, ETTR helps you to increase SNR.

If a scene has 12 stops of DR and your camera too (e.g. Nikon D850), you need ETTR to capture it in a single shot.

12-stops scene before processing: in your Velvia images you probably clipped the shadows to black, but this is not an option in interiors photography.

(https://2.bp.blogspot.com/-W0bsz96xglM/W1EJIwAZZZI/AAAAAAAAEZU/xT9dM5gCPt070lIV-S4yTGv9L1yih8LewCLcBGAs/s1600/raw1.jpg)

After processing:

(https://1.bp.blogspot.com/-Mo04zTXBquQ/W1EHkXw4BRI/AAAAAAAAEZI/qfkfxb9W7fMERnQUwmqF-fMn-6QLxLjvwCLcBGAs/s1600/mapeotonos512.jpg)


Regards
Title: Re: Optimize In Camera Jpeg White Balance RGB Histogram For Better ETTR.
Post by: Alan Klein on December 23, 2018, 05:26:05 am
OK.  For an interior pro photographer, squeezing out every last bit of shadow area may be important.  But for the non-interior photographer, the difference between a great shot and an average shot isn't going to be if he gets let's say 11 or 12 stops DR rather than 9 or 10 stops.  If the shadow areas go dark a little sooner, no one is going to care.  The great shots will be better lighting overall, content, etc. as long as they're not clipping the highlights.  After all, 12 stops DR is a rather new capability.  How did photographers create all the great shots of the past with films like Velvia with its 5 stops or even even digital cameras before the 750 with much lower DR ranges?  We don't want to substitute technical prowess for aesthetic feeling and content.
Title: Re: Optimize In Camera Jpeg White Balance RGB Histogram For Better ETTR.
Post by: E. Dinur on December 23, 2018, 06:33:27 am
But it is surely better when the artist has the freedom to decide how much of the DR he wants to use and how much he will relinquish for the sake of aesthetics, rather than have the decision forced on him by the the restraints of inferior tools or less than optimum utilization of better tools.
Title: Re: Optimize In Camera Jpeg White Balance RGB Histogram For Better ETTR.
Post by: digitaldog on December 23, 2018, 08:24:24 am
OK.  For an interior pro photographer, squeezing out every last bit of shadow area may be important.  But for the non-interior photographer, the difference between a great shot and an average shot isn't going to be if he gets let's say 11 or 12 stops DR rather than 9 or 10 stops.  If the shadow areas go dark a little sooner, no one is going to care.  The great shots will be better lighting overall, content, etc. as long as they're not clipping the highlights.  After all, 12 stops DR is a rather new capability.  How did photographers create all the great shots of the past with films like Velvia with its 5 stops or even even digital cameras before the 750 with much lower DR ranges?  We don't want to substitute technical prowess for aesthetic feeling and content.
No, it's important for any photographer who wishes control over the quality of how they render their images.
Title: Re: Optimize In Camera Jpeg White Balance RGB Histogram For Better ETTR.
Post by: Aram Hăvărneanu on December 23, 2018, 08:43:15 am
After all, 12 stops DR is a rather new capability.  How did photographers create all the great shots of the past with films like Velvia with its 5 stops

Photographers knew to use color negative film when the dynamic range of the scene exceeded the capabilities of Velvia. Color negative film has as much dynamic range as today's cameras (even more, if you shoot straight into the sun).
Title: Re: Optimize In Camera Jpeg White Balance RGB Histogram For Better ETTR.
Post by: Guillermo Luijk on December 23, 2018, 09:02:59 am
Color negative film has as much dynamic range as today's cameras (even more, if you shoot straight into the sun).

I was surprised not long ago to find out about this. Indeed film can have much more DR than digital sensors, specially towards the highlights. What film lacks by far vs digital is image quality in terms of SNR. The peak of SNR in a digital sensor is many dB's above film:

(http://www.sinespejo.com/misc/drfilm.png)

The sensor reaches +25dB over film. Every 6dB means doubling SNR, so peak SNR in digital is about 16 times that of film. That's a huge gap, the same as in the crops I showed to Alan.

In addition to that, generally speaking SNR below 12dB is insufficient by today's standards. This reduces quite a lot the theoretical DR of film messured at SNR=0dB.

The conclusion is that film is no rival today for digital: sensors have similar practical DR and much better SNR. Not to speak about processing possibilities.

Regards
Title: Re: Optimize In Camera Jpeg White Balance RGB Histogram For Better ETTR.
Post by: Alan Klein on December 23, 2018, 12:12:58 pm
I'm not saying that it's not good to have widest DR range possible. But you don't want to overly spend time to squeeze another half a stop out of an unimportant shadow detail rather than finding a better place to aim the camera or waiting for better light.  Sometimes we get caught up on the technical rather than focusing on the aesthetic.
Title: Re: Optimize In Camera Jpeg White Balance RGB Histogram For Better ETTR.
Post by: Alan Klein on December 23, 2018, 12:28:42 pm
Photographers knew to use color negative film when the dynamic range of the scene exceeded the capabilities of Velvia. Color negative film has as much dynamic range as today's cameras (even more, if you shoot straight into the sun).



Most of the best magazine covers for Outdoor Photographer and other magazines of the era were shot with velvia or other chrome film.    Somehow they lived with the five to seven stops that chome film gave them. Another one or two stops isn't going to make the picture. Let's not get carried away with the power of ETTR.
Title: Re: Optimize In Camera Jpeg White Balance RGB Histogram For Better ETTR.
Post by: Guillermo Luijk on December 23, 2018, 12:34:54 pm
Somehow they lived with the five to seven stops that chome film gave them.

One day I went to Ikea and found a very nice exhibition of the covers of all their famous (at least in Europe) annual brochures. There were over 40-50 framed images.

The evolution on technical quality was amazing, from unsharp pictures with totally blown up lamps and windows to the recent ones, sharp and clean images with all lights under control. I doubt anyone complaint about the 60's and 70's images in the time, but they would never be accepted and published today.

Regards
Title: Re: Optimize In Camera Jpeg White Balance RGB Histogram For Better ETTR.
Post by: Alan Klein on December 23, 2018, 12:47:31 pm
One day I went to Ikea and found a very nice exhibition of the covers of all their famous (at least in Europe) annual brochures. There were over 40-50 framed images.

The evolution on technical quality was amazing, from unsharp pictures with totally blown up lamps and windows to the recent ones, sharp and clean images with all lights under control. I doubt anyone complaint about the 60's and 70's images in the time, but they would never be accepted and published today.

Regards

I agree that photography today is much better than it was before. But that's the technology of the camera. But squeezing another half a stop out of a shadow area is not going to make a photograph compelling. I think many photographers, and I'm included, get sidetracked with technical aspects thinking that if I only had that better camera or that better technology that somehow my photos are going to be better. Only to be disappointed that when I upgrade, I'm still shooting snapshots. Oh sure, they are sharper and have more dynamic range. But they're still crummy snapshots.

I'd be better off learning about lighting,  content,  and making a more compelling photograph.

Title: Re: Optimize In Camera Jpeg White Balance RGB Histogram For Better ETTR.
Post by: Frans Waterlander on December 23, 2018, 01:31:47 pm
Now, come on Alan! Technical and artistic issues are different. The OP brought up a technical issue, so let's stick with that. Whether or not this issue is important to you is for you to decide.

It's not only about blocked up shadows/more shadow detail, but more importantly, shadow detail with less noise/grain. This is even more of an issue with HDR photography.
Title: Re: Optimize In Camera Jpeg White Balance RGB Histogram For Better ETTR.
Post by: Aram Hăvărneanu on December 23, 2018, 01:48:11 pm
Somehow they lived with the five to seven stops that chome film gave them.

Photographers lived for more than 100 years only with black and white photography, and great art was created during this time. Does that mean that photographers don't need color?

Should we abandon panchromatic film as well?
Title: Re: Optimize In Camera Jpeg White Balance RGB Histogram For Better ETTR.
Post by: digitaldog on December 23, 2018, 01:50:00 pm
I'm not saying that it's not good to have widest DR range possible. But you don't want to overly spend time to squeeze another half a stop out of an unimportant shadow detail rather than finding a better place to aim the camera or waiting for better light.  Sometimes we get caught up on the technical rather than focusing on the aesthetic.
Optimal exposure doesn't taken more time, just a little knowledge!
Title: Re: Optimize In Camera Jpeg White Balance RGB Histogram For Better ETTR.
Post by: TonyW on December 23, 2018, 02:02:06 pm

Most of the best magazine covers for Outdoor Photographer and other magazines of the era were shot with velvia or other chrome film.    Somehow they lived with the five to seven stops that chome film gave them. Another one or two stops isn't going to make the picture. Let's not get carried away with the power of ETTR.
It really is not a question of how many stops DR you have at your disposal it is really a matter of exposing correctly to get the best out of the medium taking into account its limitations - in this case digital. 

You shot transparency or negative stock and you must have tried to expose correctly e.g. exposure to make sure you did not loose highlight detail (IF highlight detail was the most important factor of a particular shot) in transparency and maybe you exposed for the shadows developed for the highlights with B&W stock. 

You are doing no more when trying for an optimal exposure with digital to make the most of the data you have available.  You can make it as complex as you want or make it very simple. 

Simple maybe to find out how many stops you have before sensor saturation - practical testing - meter an important highlight (not specular - that should be blown) and see how many stops you have before that particular highlight gone.  I suspect most systems will not allow you more than 3 stops before clipping equating to your system being calibrated to expect 12.5% grey under the metered ROI.  It may be more or less but needs testing, again either very simple or in more detail with something like Rawdigger

DR is only part of the story and will not really change clipping limits of around 3 stops for highlights (a generalization for most systems!) so 6 stops or 14 stops DR does not change.   
Title: Re: Optimize In Camera Jpeg White Balance RGB Histogram For Better ETTR.
Post by: Guillermo Luijk on December 23, 2018, 02:12:01 pm
I suspect most systems will not allow you more than 3 stops before clipping equating to your system being calibrated to expect 12.5% grey under the metered ROI.  It may be more or less but needs testing, again either very simple or in more detail with something like Rawdigger

Spot metering against a flat surface under tungsten lighting:

Canon 350D:
(http://www.guillermoluijk.com/article/spot/histomax350d.gif)

Canon 5D:
(http://www.guillermoluijk.com/article/spot/histomax5d.gif)

Regards
Title: Re: Optimize In Camera Jpeg White Balance RGB Histogram For Better ETTR.
Post by: Alan Klein on December 23, 2018, 03:07:29 pm
Now, come on Alan! Technical and artistic issues are different. The OP brought up a technical issue, so let's stick with that. Whether or not this issue is important to you is for you to decide....

...

Fair enough,  Frans.  I suppose I'm trying to remind myself that my issue with taking better shots is not technical on the margins,  but rather in the area of content,  overall lighting and other aesthetic areas.
Title: Re: Optimize In Camera Jpeg White Balance RGB Histogram For Better ETTR.
Post by: Aram Hăvărneanu on January 31, 2019, 06:16:49 am
Most of the best magazine covers for Outdoor Photographer and other magazines of the era were shot with velvia or other chrome film.    Somehow they lived with the five to seven stops that chome film gave them. Another one or two stops isn't going to make the picture. Let's not get carried away with the power of ETTR.

Totally bad example, ETTR is far more critical with slide film (and other low dynamic range mediums) than it is with digital. All those Outdoor Photographer shots were properly exposed (ETTR).
Title: Re: Optimize In Camera Jpeg White Balance RGB Histogram For Better ETTR.
Post by: Dave Rosser on January 31, 2019, 08:20:38 am
To repeat what has already been said some people do like to make things difficult (and this is someone who has those green NEFs from experimenting with UniWB speaking  :( ). I just went back to square one and took a shot using the device shown in the first picture to estimate required exposure (its a simple calculator to make using B.S.935/1957 exposure tables a bit easier).
The second picture is the resulting shot of the road outside my house.  The only difficulty is deciding on weather conditions - I chose cloudy dull and the histogram in Capture One shows the exposure was spot on.  ;D

Dave




Title: Re: Optimize In Camera Jpeg White Balance RGB Histogram For Better ETTR.
Post by: Alan Klein on January 31, 2019, 08:37:11 am
Totally bad example, ETTR is far more critical with slide film (and other low dynamic range mediums) than it is with digital. All those Outdoor Photographer shots were properly exposed (ETTR).

Actually when shooting chromes, pros often underexposed 1/3 of a stop to avoid clipping the highlights.  ETTR, even if one could make a film comparison to digital, would have caused more throwaways due to blown highlights. Underexposing also tends to saturate the colors adding even more "pop" to the already saturated Velvia. 
Title: Re: Optimize In Camera Jpeg White Balance RGB Histogram For Better ETTR.
Post by: digitaldog on January 31, 2019, 09:25:11 am
Actually when shooting chromes, pros often underexposed 1/3 of a stop to avoid clipping the highlights.  ETTR, even if one could make a film comparison to digital, would have caused more throwaways due to blown highlights. Underexposing also tends to saturate the colors adding even more "pop" to the already saturated Velvia.
No, we pros didn't because clipping highlights we didn't want to clip is over exposing!
Title: Re: Optimize In Camera Jpeg White Balance RGB Histogram For Better ETTR.
Post by: Alan Klein on January 31, 2019, 10:01:25 am
Of course you want to get exposure exactly right.  But metering is not always an exact science.  If the photographer is concerned that he might blow the highlights and ruin the shot, he'd play it cautious and shoot 1/3 or 1/2 of a stop slower.  There were no histograms back then.  Certainly nothing where you could ETTR which has no point with film in any case.
Title: Re: Optimize In Camera Jpeg White Balance RGB Histogram For Better ETTR.
Post by: Alan Klein on January 31, 2019, 10:01:58 am
Sorry faster.
Title: Re: Optimize In Camera Jpeg White Balance RGB Histogram For Better ETTR.
Post by: Aram Hăvărneanu on January 31, 2019, 10:10:57 am
ETTR, even if one could make a film comparison to digital, would have caused more throwaways due to blown highlights.

ETTR by definition can't blow highlights. If you have (unintentional) blown highlights, you didn't do ETTR. Perhaps you attempted to do ETTR and failed.

Of course you want to get exposure exactly right.  But metering is not always an exact science.  If the photographer is concerned that he might blow the highlights and ruin the shot, he'd play it cautious and shoot 1/3 or 1/2 of a stop slower. 

Yes, but that's the same for chromes and digital.

There were no histograms back then.  Certainly nothing where you could ETTR which has no point with film in any case.

You don't need histograms to do ETTR. You only need a reflective spot meter and the film characteristic curve.

When I shoot velvia I meter the important highlights and place them at +2, knowing that +2 will still retain detail. +2.5 is pure white. I get proper exposure every time. Other slide film might have different limits.

When I shoot negative film I put important shadows at -1.5. Ansel Adams used zone III, which is -2, but I think -1.5 or even -1 is better today. Modern film has plenty dynamic range even if you put shadows at zone IV, and it's easier to control contrast digitally (Ansel benefited from a contrast boost using the toe of the transfer curve, no such need today, using the linear part of the curve is better).
Title: Re: Optimize In Camera Jpeg White Balance RGB Histogram For Better ETTR.
Post by: digitaldog on January 31, 2019, 10:58:24 am
Of course you want to get exposure exactly right.  But metering is not always an exact science.  If the photographer is concerned that he might blow the highlights and ruin the shot, he'd play it cautious and shoot 1/3 or 1/2 of a stop slower.  There were no histograms back then.  Certainly nothing where you could ETTR which has no point with film in any case.
Alan, your comments indicate to me you are not familiar how we professional photographers exposed E6 film. In my case, shooting national ads, magazines, annual reports etc, on E6 film for a living.
IF you haven't experienced it, you've only imagined it. Or misunderstood it.  ;)


We professional photographers purchased very large quantities of the same film emulsion (bricks and bricks), stored them in dedicated refrigerators. We'd run exposure and processing tests on the first roll. Call out and thanks to the legendary Ish of A&I color for assisting in advising us on CC filters for color balance under our strobes when testing the film and processing for that task. Every time we started a new batch of film.
We understood how our tools worked, including light meters: incident, spot or otherwise and when the meters were fooled (the old white dog on snow/black cat on coal analogy AGAIN).
Old saying from actual pro's: Don't learn the tricks of the trade, learn the trade!

We pros cloud therefore more often than not, NAIL exposure within a 1/4 a stop. That wasn't good enough! Shooting Med format, we'd dedicate an A12 back as a test role for each setup, run (process at the same lab) that then again, show it to Ish and there's the ability to finely control push/pull processing FOR EACH subsequent labelled roll in a setup for dead nuts results for reproduction! Thats what we pro's did to make the bucks shooting on E6 film. Of run what's called a snip test again to nail the processed film that was exposed within 1/4 stop or less in the first place (indeed, Ish could process film 1/8 a stop +/- and do so, day in and day out).


We know where we wanted highlights to fall. NO! We didn't under expose 1/3 a stop. We exposed OPTIMALLY for those highlights. Despite what a pretty dumb device (a light meter) suggests.
That's at least how I was professionally trained to expose E6 film and how I exposed E6 film (among other kinds of film) professionally; my day job, how I fed my family.
History lesson over.  :o
Title: Re: Optimize In Camera Jpeg White Balance RGB Histogram For Better ETTR.
Post by: Alan Klein on January 31, 2019, 11:57:32 am
Aram and Andrew: I bow to your experience. But I'm wondering if all that was mainly for studio control? How did you handle landscape photography with constantly changing lighting conditions?  That's when I was referring to shooting with caution when you were using the 35mm camera's meter to set exposure.  You were using center weighted or overall metering depending on the camera and era.  Spots weren't available.  And shooting a little faster not to blow the highlights.
Title: Re: Optimize In Camera Jpeg White Balance RGB Histogram For Better ETTR.
Post by: digitaldog on January 31, 2019, 12:10:16 pm
Aram and Andrew: I bow to your experience. But I'm wondering if all that was mainly for studio control? How did you handle landscape photography with constantly changing lighting conditions?  That's when I was referring to shooting with caution when you were using the 35mm camera's meter to set exposure.  You were using center weighted or overall metering depending on the camera and era.  Spots weren't available.  And shooting a little faster not to blow the highlights.
IF you can bracket (based on sound and fundamental understanding of exposure as a basis) DO SO.
Title: Re: Optimize In Camera Jpeg White Balance RGB Histogram For Better ETTR.
Post by: smthopr on January 31, 2019, 12:36:34 pm
The first is 'pushed' as it isn't an ideal exposure for raw as 'recommended' by the meter. The 2nd appears far too bright until (using Michaels term from the original ETTR article) normalized. Had the first been optimally exposed, it too would need to be normalized and wouldn't be as noisy. But the main take away is the incorrect concept that higher ISO always produces more noise as the OP implied.

Maybe. But, since the exposures here are the same, the difference must be in the gain applied to the Sensor capture, before the recording of the data, plus any image processing also applied by the camera software.  And so this is the result.

I work mostly with professional motion picture digital cameras.  Many of these do not process the image before the RAW capture and simply record an image exposed at higher ISO settings/exposure as "darker" or images with less amplitude.  It's only in post processing that the exposure is normalized.  So, in my case of movie work, the higher ISO image always has more noise. (The gain is applied in post processing).  I do wish that my Canon 5Ds would work the same way sometimes :), but it doesn't.

With the movie camera I always know that using a higher ISO will protect more highlights, at the expense of shadows.  And vice versa.

And as for exposure strategy, I aim for consistency, and not for "lowest noise".  So, generally, I know that if I'm working at ISO 800 (depending on the camera, this is the ISO that provides near equal dynamic range above and below the middle value), I have plenty of data to work with without clipping any wanted highlights.  And so, there is little reason for ETTR.  Once in a while, on a very high contrast image (typically a day exterior) I'll set the camera to show me the entire range of tones, as opposed to a "preview" that is like a jpg on a still camera.  And adjust the exposure accordingly.  Or, I'll just use my spot meter.  And sometimes, when shooting a very dark scene, I'll shoot at a lower ISO, so that the on set preview is good and dark, but I'lll know that my exposure now will allow me to lighten it a little in post production, without excessive noise, when necessary.

When shooting stills, because my Canon 5Ds adds gain before the RAW recording, when using higher ISO settings, I need to adjust the "exposure compensation" instead of the ISO setting to get the same results or protect the highlights.  It would be nice, therefore, if the 5D had a jpg setting for LOG gamma, or maybe "full dynamic range" in order to preview the clipping points.  And then, the histograms would be accurate for adjusting exposure in extreme conditions.  I think there might be some Sony cameras that allow this in still photography mode, but I'm not sure.  These images wouldn't be so good for color correction as the compressed jpgs would fall apart when adding in the missing contrast.  This is certainly the case when shooting movies on a Sony A7RII camera in LOG mode and recording the movie in the camera.  But if one records to an outboard recorder, this situation improves quite a bit, but is still limited to 8 bits which can cause some banding.  But for a purely "exposure preview" plus a RAW capture, for still photography,  I'd like to see this feature!
Title: Re: Optimize In Camera Jpeg White Balance RGB Histogram For Better ETTR.
Post by: bjanes on January 31, 2019, 06:47:45 pm
No, we pros didn't because clipping highlights we didn't want to clip is over exposing!

Of course you want to get exposure exactly right.  But metering is not always an exact science.  If the photographer is concerned that he might blow the highlights and ruin the shot, he'd play it cautious and shoot 1/3 or 1/2 of a stop slower.  There were no histograms back then.  Certainly nothing where you could ETTR which has no point with film in any case.

Andrew,

Out of the many thousands of exposures of E6 film in your professional career, did you ever have a less than optimally exposed image? I would think that the answer would be yes, since metering is not an exact science as Alan points out in his post. In these cases, it is helpful to have tools to improve a less than optimally exposed image. Like Jim Kasson, I still use the term ETTR for shots at base ISO and use the histogram and blinkies in evaluating exposure and think the term ETTR is appropriate, since it denotes that the histogram was used in the evaluation. Proper exposure is a less than helpful term IMO.

Even when shooting with ISOs above base, the histogram is helpful in that a shot with a histogram to the left is leaving image quality on the table when one is shooting in raw. Raising ISO to move the histogram to the right will not improve the situation if one uses the same exposure (f/stop and shutter speed), but will only decrease highlight headroom.

As I recall from film days, many photographers would underexpose (if you forgive the term) by 1/3 EV with reversal film mainly to improve color and saturation more than to protect the highlights. One third EV under will provide minimal highlight protection.

Regards,

Bill
Title: Re: Optimize In Camera Jpeg White Balance RGB Histogram For Better ETTR.
Post by: digitaldog on January 31, 2019, 06:55:24 pm
Andrew,
Out of the many thousands of exposures of E6 film in your professional career, did you ever have a less than optimally exposed image?
I'm sure I did. When I shot the 1984 Olympic games, about 100 rolls a day, in so many different venues, pretty sure I didn't nail them all. But those brain cells are long gone.
Metering isn't an exact science but I know how my tools behave and when they are dead wrong; examples provided.

Quote
As I recall from film days, many photographers would underexpose (if you forgive the term) by 1/3 EV with reversal film mainly to improve color and saturation more than to protect the highlights. One third EV under will provide minimal highlight protection.
I can only speak for the professionals I've shot with and myself. We might COMPENSATE 1/3 over or under what a dumb meter suggested because we understood how our tools worked. What Alan stated was simply wrong: we didn't UNDER EXPOSE to avoid clipping highlights. We exposed FOR the highlights. We understood based on our tools when and how to do so. That's absolutely NOT under exposing. That's simply a misstatement based on misunderstanding. If you or anyone else here thinks the meter knows what we want and should blindly follow it, otherwise an adjustment based on a through understanding of the tools is over or under exposing, doesn't understand what exposure is. I can assure you, it's not what the meter demands. It is what the knowledgeable photographer sets.
Title: Re: Optimize In Camera Jpeg White Balance RGB Histogram For Better ETTR.
Post by: Alan Klein on January 31, 2019, 10:28:51 pm
Andrew, We actually agree.  It's just that we're using different words to say the same thing.