Optimize In Camera Jpeg White Balance RGB Histogram For Better ETTR.

[Aram Hăvărneanu]

with the fine print - with the current technology ... you can't be sure what comes next or may be already lurking in the labs ...

The quantum efficiency of modern sensors is something like 70%. If you could control the sensitivity to light, you could only make it less sensitive.

[Guillermo Luijk]

A flaw of any technique trying to use the JPEG histogram as a proxy to mimic the genuine RAW histogram (of which UniWB is probably the most effective generally speaking), is that in the end of the camera RAW processing pipeline, a colour conversion from camera space (or whatever we like to call RAW data) to an output colour space (sRGB or Adobe RGB) takes place.

This 3D matrix conversion makes all three RGB input channels to individually contribute to each of the three output channels. Being G the most interesting (since it predominantly encodes higher RAW values), we could specify each output G value as a linear combination:

G_out = k1*R_in + k2*G_in + k3*B_in

G_out=1 could correspond to a G_in<1 (false positive RAW clipping)

G_out<1 could correspond to a G_in=1 (false negative RAW clipping)

In other words, no matter what we do there is never a guarantee of having a reliable RAW clipping warning through JPEG histograms.

Regards

[faberryman]

In other words, no matter what we do there is never a guarantee of having a reliable RAW clipping warning through JPEG histograms.

Let's remember there are no guarantees in life about anything, and we just do the best job we can. I don't have a problem with blown highlights very often so I can't get too excited about all this. I bet most members don't routinely have these problems either. Maybe if you have a ten year old camera with limited dynamic you have to be more careful. I wish people would worry more about their composition and less about their histogram.

[digitaldog]

Let's remember there are no guarantees in life about anything, and we just do the best job we can. I don't have a problem with blown highlights very often so I can't get too excited about all this. I bet most members don't routinely have these problems either. Maybe if you have a ten year old camera with limited dynamic you have to be more careful. I wish people would worry more about their composition and less about their histogram.

I believe his point is that one does not need to rely on a JPEG histogram to optimally expose raw. The history of photography seems to clearly back up that concept concerning exposure.

[TonyW]

Let's remember there are no guarantees in life about anything, and we just do the best job we can. I don't have a problem with blown highlights very often so I can't get too excited about all this. I bet most members don't routinely have these problems either. Maybe if you have a ten year old camera with limited dynamic you have to be more careful. I wish people would worry more about their composition and less about their histogram.

I do not think this is solely or particularly about blown highlights, rather more about optimal exposure for digital capture including that of the scene DR fitting well inside the cameras DR capacity where you really have a chance to optimise exposure moving as far right as possible prior to clipping.  How important this is for noise reduction with modern sensors and processing algorithms I do not know but suspect not quite as much as in the past

My 10 year old Nikon had limited DR capability maybe 3-4 EV less than sensors of the last few years, but still more range than most transparency film IIRC and probably a little less than colour neg material and almost certainly less than the best B&W film/print combination

[nirpat89]

A practical question about the in-camera raw histogram if there was such a thing, how would one use it?  It would still have to be displayed on a scale so in order to know what is going on beyond this scale, one would need some way to handle the histogram as you would in CameraRaw or Lr.  Kind of awkward fidgeting with the camera display trying to figure out if you nailed the ETTR while shooting on location.  No?

[digitaldog]

A practical question about the in-camera raw histogram if there was such a thing, how would one use it?  It would still have to be displayed on a scale so in order to know what is going on beyond this scale, one would need some way to handle the histogram as you would in CameraRaw or Lr.  Kind of awkward fidgeting with the camera display trying to figure out if you nailed the ETTR while shooting on location.  No?

Some examples from the way done on Canon with Magic Lantern:

http://hdr-photographer.com/2014/10/why-i-love-magic-lantern-several-years-later/
https://www.magiclantern.fm/forum/index.php?topic=12096.0


Again, assuming you MUST have a Histogram to expose. 

[Guillermo Luijk]

A practical question about the in-camera raw histogram if there was such a thing, how would one use it?  It would still have to be displayed on a scale so in order to know what is going on beyond this scale, one would need some way to handle the histogram as you would in CameraRaw or Lr.  Kind of awkward fidgeting with the camera display trying to figure out if you nailed the ETTR while shooting on location.  No?

My preferred format would be log X axis (with EV divisions) + linear Y axis. Histograms like this would be tremendously intuitive giving precise information about the adjustments needed for accurate optimum exposure (ETTR):



Just a glance to find out that adding +1,5EV means perfect ETTR here. Or just +1EV if you don't even want to sacrifice those very few G photosites.

Such histogram also provides a precise DR measurement of the scene; 8 stops in this case (number of vertical divisions with relevant content). A camera with N-bit RAW files encoding can measure DR of up to N stops, no matter if the sensor's capture DR is lower than N stops.

Regards

[nirpat89]

My preferred format would be log X axis (with EV divisions) + linear Y axis. Histograms like this would be tremendously intuitive giving precise information about the adjustments needed for accurate optimum exposure (ETTR):



Just a glance to find out that adding +1,5EV means perfect ETTR here. Or just +1EV if you don't even want to sacrifice those very few G photosites.

Such histogram also provides a precise DR measurement of the scene; 8 stops in this case (number of vertical divisions with relevant content). A camera with N-bit RAW files encoding can measure DR of up to N stops, no matter if the sensor's capture DR is lower than N stops.

Regards

That makes perfect sense.  Thanks.  I hope Nikon is consulting you...

[nirpat89]

Some examples from the way done on Canon with Magic Lantern:

http://hdr-photographer.com/2014/10/why-i-love-magic-lantern-several-years-later/
https://www.magiclantern.fm/forum/index.php?topic=12096.0


Again, assuming you MUST have a Histogram to expose. 

Thanks, Andrew.  Had never heard of Magic Lantern.  It is not for Nikons, unfortunately, as yet so not useful for me, though.   May be some day.

[digitaldog]

Thanks, Andrew.  Had never heard of Magic Lantern.  It is not for Nikons, unfortunately, as yet so not useful for me, though.   May be some day.

Another reason to simply learn to expose optimally without a Histogram. A raw Histogram could be useful. It's absolutely not required. Kodachrome, Velvia etc, didn't have one and somehow, many of us managed to expose that stuff.

[TonyW]

.....
Again, assuming you MUST have a Histogram to expose. 

Agreed, although a nice to have feature it may be viewed as the post-mortem/inquest after the decisive moment has long passed

[TonyW]

My preferred format would be log X axis (with EV divisions) + linear Y axis. Histograms like this would be tremendously intuitive giving precise information about the adjustments needed for accurate optimum exposure (ETTR):



Just a glance to find out that adding +1,5EV means perfect ETTR here. Or just +1EV if you don't even want to sacrifice those very few G photosites.

Such histogram also provides a precise DR measurement of the scene; 8 stops in this case (number of vertical divisions with relevant content). A camera with N-bit RAW files encoding can measure DR of up to N stops, no matter if the sensor's capture DR is lower than N stops.

Regards

Oh yes, I like that and would welcome the option to choose this type of display during capture.  If only there was a strong enough photographic lobby to overcome camera manufacturers inertia in this area.  I am making an assumption that currently no manufacturer has the option to view a histogram giving useful raw information.

As a matter of interest what software produced your posted image?

[Guillermo Luijk]

Oh yes, I like that and would welcome the option to choose this type of display during capture.  If only there was a strong enough photographic lobby to overcome camera manufacturers inertia in this area.  I am making an assumption that currently no manufacturer has the option to view a histogram giving useful raw information.

As a matter of interest what software produced your posted image?

Canon cameras with Magic Lantern do have RAW histograms as Andrew showed. I also believe Leica Monochrome can be considered to have RAW histograms in terms of exposure, since their monochrome histogram clipping reflects that of the RAW. Not 100% sure of that though.

Obtaining a real RAW histogram is trivial, camera manufacturers just don't implement it:

http://www.elmomentodecisivo.com/2018/05/histogramas-raw-en-camaras-digitales.html

That histogram was created with this piece of software I wrote time ago:

http://www.guillermoluijk.com/tutorial/histogrammar/index.htm

It can input DCRAW decoded RAW files (-D and -d options) and plot them in different styles.

Regards

[TonyW]

Canon cameras with Magic Lantern do have RAW histograms as Andrew showed. I also believe Leica Monochrome can be considered to have RAW histograms in terms of exposure, since their monochrome histogram clipping reflects that of the RAW. Not 100% sure of that though.

Obtaining a real RAW histogram is trivial, camera manufacturers just don't implement it:

http://www.elmomentodecisivo.com/2018/05/histogramas-raw-en-camaras-digitales.html

That histogram was created with this piece of software I wrote time ago:

http://www.guillermoluijk.com/tutorial/histogrammar/index.htm

It can input DCRAW decoded RAW files (-D and -d options) and plot them in different styles.

Regards

Thank you.  I will look at your links in detail.

Lamento decir que mi español es muy limitado. Pero con la ayuda de Google, estoy deseando leer tu artículo.

[bjanes]

A flaw of any technique trying to use the JPEG histogram as a proxy to mimic the genuine RAW histogram (of which UniWB is probably the most effective generally speaking), is that in the end of the camera RAW processing pipeline, a colour conversion from camera space (or whatever we like to call RAW data) to an output colour space (sRGB or Adobe RGB) takes place.

This 3D matrix conversion makes all three RGB input channels to individually contribute to each of the three output channels. Being G the most interesting (since it predominantly encodes higher RAW values), we could specify each output G value as a linear combination:

G_out = k1*R_in + k2*G_in + k3*B_in

G_out=1 could correspond to a G_in<1 (false positive RAW clipping)

G_out<1 could correspond to a G_in=1 (false negative RAW clipping)

In other words, no matter what we do there is never a guarantee of having a reliable RAW clipping warning through JPEG histograms.

Regards

Guillermo,

An excellent post. The WB multipliers and matrix coefficients for the Nikon D850 according to DXO are shown below. It would be an interesting exercise to take some shots of varying subjects with the D850 set to record both raw and sRBG, read the raw values in Rawdigger, apply the white balance, apply the matrix for sRGB, and then convert from linear gamma to sRGB gamma. Would this be technically sound? I presume one would apply the WB before the matrix coefficients--is this correct?



Best regards,

Bill

[digitaldog]

Guillermo,

An excellent post. The WB multipliers and matrix coefficients for the Nikon D850 according to DXO are shown below. It would be an interesting exercise to take some shots of varying subjects with the D850 set to record both raw and sRBG, read the raw values in Rawdigger, apply the white balance, apply the matrix for sRGB, and then convert from linear gamma to sRGB gamma. Would this be technically sound? I presume one would apply the WB before the matrix coefficients--is this correct?



Best regards,

Bill

When people say optimal exposure is hard, I would ask "Compared to what?" Now I know.

[faberryman]

An excellent post. The WB multipliers and matrix coefficients for the Nikon D850 according to DXO are shown below. It would be an interesting exercise to take some shots of varying subjects with the D850 set to record both raw and sRBG, read the raw values in Rawdigger, apply the white balance, apply the matrix for sRGB, and then convert from linear gamma to sRGB gamma. Would this be technically sound? I presume one would apply the WB before the matrix coefficients--is this correct?

The first question I would ask is whether the image itself is really worth all that rigamarole.

[DP]

In other words, no matter what we do there is never a guarantee of having a reliable RAW clipping warning through JPEG histograms.

in theory no, in practice though UniWB + flattest contrast curve + widest gamut output colorspace (AdobeRGB vs sRGB, but even with sRGB) + blinkies/zebra reliably show clipping in raw within < 1/3 EV precision as checked with RawDigger under tungsten to sun to blue skies/cloudy daylight illumination (and consumer type LED bulbs & fluorescents) and that I use on my Olympus and later Sony dSLMs for years...

[DP]

The quantum efficiency of modern sensors is something like 70%.

I am not talking about "modern" sensors (as used in current cameras) if you did not notice