Leica and histograms

[larsrc]

Nice article about the Leica factory, pretty cool that they can be so successful with so many steps done by hand.

I get my hackles up over the comment on why histograms are based on the JPG rather than raw data. While raw data is not an image per se, a histogram does not need an image, just some data on a scale. Yes, the white balance settings can alter where the histogram ends up, but for many RAW shooters, the most important piece of information in the histogram is whether any channel is clipped. Clipping can easily be shown in a histogram. Lack of raw histogram shows a lack of acceptance (if no understanding) of ETTR.

[NikoJorj]

While raw data is not an image per se, a histogram does not need an image, just some data on a scale.

Ditto ; this response is quite puzzling, coming from a senior manager in a reputed manufacturer.

We're waiting since some time now (http://www.luminous-landscape.com/forum/index.php?topic=33267 if not http://www.luminous-landscape.com/tutorials/expose-right.shtml for the metering mode) and still nothing in sight...
Is that a giant conspiracy of aliens not wanting us to have the best tools available to take embarassing photographs of them?

[dreed]

Nice article about the Leica factory, pretty cool that they can be so successful with so many steps done by hand.

I get my hackles up over the comment on why histograms are based on the JPG rather than raw data. While raw data is not an image per se, a histogram does not need an image, just some data on a scale. Yes, the white balance settings can alter where the histogram ends up, but for many RAW shooters, the most important piece of information in the histogram is whether any channel is clipped. Clipping can easily be shown in a histogram. Lack of raw histogram shows a lack of acceptance (if no understanding) of ETTR.

I wonder, do you really need a histogram or just something that flashes saying "maximum value reach in colour channel <X> reached"?

For example, if the red channel reached its maximum value (16383 for a 14bit sensor) in any pixel and a red circle or square flashed on the camera's review screen next to the jpeg based histogram, doesn't that suffice?

[NikoJorj]

I wonder, do you really need a histogram or just something that flashes saying "maximum value reach in colour channel <X> reached"?

Both :
1 the histogram to check there is no underexposure and the DR of the camera is fully used,
2 the highlight clipping indicator to check that there is no significant overexposure, ie nothing significant in the image is blown.

[dreed]

Both :
1 the histogram to check there is no underexposure and the DR of the camera is fully used,
2 the highlight clipping indicator to check that there is no significant overexposure, ie nothing significant in the image is blown.

Doing (1) on a raw file is perhaps not quite what you think. To get a feel for what it would look like, apply a Linear tone curve mapping to a raw file and see what results. That is the closest example of what I believe a histogram would look like for a raw file.

Further, I think that (1) is unnecessary. What we really want is to find the setting at which clipping occurs and then go one step under that.

[JimGoshorn]

From what I have read, the S2 offers the ability to set the clipping values in their last firmware update:

http://www.luminous-landscape.com/reviews/cameras/leica_s2_firmware_update.shtml

Now if that could only be universally adopted...

Jim

[Hans Kruse]

And the note from Mark Dubovoy says that Leica claims that the histogram is based on RAW data and not a JPG. Seems a bit contradicting the Leica executive.

[Nick Rains]

And the note from Mark Dubovoy says that Leica claims that the histogram is based on RAW data and not a JPG. Seems a bit contradicting the Leica executive.

This was a mistake.

Mark and I clarified this with the Leica guys and the comments I reported from Stephan Schulz at Leica can be taken as the definitive word on the subject. That's not to say this situation might change in the future, but for now, the S2 histograms are jpeg based.

[JeanMichel]

Hi,

Well, when I went 'digital' with a Canon 5d, later 5d2, I found the histograms useful to check my exposures. My photography became very much a shoot, glance at screen; shoot, glance at screen; perhaps double check the histogram. I am very happy with my prints from this camera. ...and LiveView is amazing when on a tripod...

But, going back to Leica, a few weeks ago I received my Leica M9. I am using my old but excellent M lenses with it (vintage 1962, 1968 and 1973 and a more recent 21 mm VC). Almost immediately I set the image review to 'none'. There really is no absolute necessity to always check the image or histogram that you have just taken. In film days, I used an M4 and M3, and later an M6, together with a Lunasix meter. I metered whenever I felt I needed, but definitely not before every shot. The meters in the M6 and M9 are not the same but both are more or less centre-weighted. If the area being metered is likely giving me a longer or shorter exposure than I think i need I simply open or close the aperture or dial in an exposure compensation on the M9.

I'm all for progress and technology, but spending some time with a light meter and a graycard will eventually give you an intuitive knowledge of what exposure to use. Maybe one can learn that from looking at histograms, but I doubt it.

Jean-Michel 

[ErikKaffehr]

Hi,

As pointed out in Nick's article that feature is of very limited use. Not useful at all if shooting raw?!

Best regards
Erik

From what I have read, the S2 offers the ability to set the clipping values in their last firmware update:

http://www.luminous-landscape.com/reviews/cameras/leica_s2_firmware_update.shtml

Now if that could only be universally adopted...

Jim

[Hans Kruse]

This was a mistake.

Mark and I clarified this with the Leica guys and the comments I reported from Stephan Schulz at Leica can be taken as the definitive word on the subject. That's not to say this situation might change in the future, but for now, the S2 histograms are jpeg based.

Thanks for the clarification. It would be good for an update on the LuLa article from Mark to avoid the contradiction. Your comment on the histogram being RAW based as a rumour on the internet could come from this, maybe.

[NikoJorj]

I'm all for progress and technology, but spending some time with a light meter and a graycard will eventually give you an intuitive knowledge of what exposure to use.

If you shoot film, or maybe use jpegs, then gray-based metering helps two things :
- optimize exposure while placing most of the information on the linear, central part of the film's characteristic curve,
- render tone as you want them straight out of the camera.

That was then, this is now. On n'est plus au millénaire précédent, with due respect.
If you shoot raw, then the optimal place to put your information is on the right side of the histogram, ie base your metering on highlights and not midtones : the farther at the left the more noise, but don't clip anything in the HL.
See http://www.luminous-landscape.com/tutorials/optimizing_exposure.shtml or even better if you can read spanish (or can stand an automated translation) http://www.guillermoluijk.com/article/ettr3/index.htm

[JeanMichel]

That was then, this is now. On n'est plus au millénaire précédent, with due respect.
If you shoot raw, then the optimal place to put your information is on the right side of the histogram, ie base your metering on highlights and not midtones : the farther at the left the more noise, but don't clip anything in the HL.

Yes, I too only shoot RAW and do expose to the right. I took the time to read, experiment and learn to expose in the digital era, including using ETTR - it all makes sense. All I am saying is that learning to expose for digital -- just as learning to expose for film when you had to change your thinking between using b&w film, colour negs, or transparencies -- is a matter of learning to place the exposure at the right place, and that this can and should become intuitive. Just as with the zone system, one learns what values are and where to place them for optimum digital era rendition.

Jean-Michel

[bjanes]

This was a mistake.

Mark and I clarified this with the Leica guys and the comments I reported from Stephan Schulz at Leica can be taken as the definitive word on the subject. That's not to say this situation might change in the future, but for now, the S2 histograms are jpeg based.

The histograms may be JPEG based, but what is the color space from which they are derived? Many dSLRs have a selectable color space no wider than Adobe RGB and this is not sufficient to cover what is captured in the raw image and may result in clipping of the histogram when no clipped raw channels are present. One could also ask if the histograms are white balanced; if so, clipping of the blue and red channels may occur when the raw file is intact.

Regards,

Bill

[dng88]

No longer have M8 but my experience is that the camera is quite unique in its exposure calculation.  Its historgram actually change when you zoom in.  That is good for basic checking of point of internest exposure situation.  Jpeg but at least better than quite a number of camera I used.

[32BT]

If you shoot film, or maybe use jpegs, then gray-based metering helps two things :
- optimize exposure while placing most of the information on the linear, central part of the film's characteristic curve,
- render tone as you want them straight out of the camera.

That was then, this is now.

Perhaps things haven't changed as much as you think.

1. As I understand it, digital sensors aren't as linear as most of those ETTR texts want them to be. There are generally a non-linear minimum and maximum, with a more-or-less linear response in the middle. How much data can be derived from the linear part of that response curve is obviously limited by a lot of factors, including exposure time and iso, and for example the base voltage applied to the sensor prior and during exposure. Which sensors have what response, and how camera manufacturers circumvent non-linearities, isn't exactly public knowledge.

2. How do you know that camera manufacturers aren't already using an ETTR with automatic re-adjusment? If at all possible, with regards to some reasonable exposure+iso setting, then perhaps they are already applying some form of ETTR to capture the most linear part of the response curve.

3. Perhaps you want to increase exposure at the expense of linearity. Perhaps less noise is more important to you than a linear response. I believe that Canon are doing something to that effect with high-light priority mode. You get some extra dynamic range, but possibly less accurate and harder to control color.

4. And additionally: I don't believe any of us really want a "linear" histogram. What you may want is a histogram with RAW data, but with some form of gamma applied. Otherwise you will just be looking at a clump of data on a tiny display which means precious little to most of us. In that respect, I doubt that it will be at all useful for determining clipped data. As mentioned previously here, it will be much more useful to just read: channel R, G, or B has x% clipped data.

5. Plus one of my current issues: what would you consider a correct exposure reference?
Averaged Linear data?
Averaged gamma 2.2 data?
Averaged Lab data?

What reference values do you think are correct?

You see, if you want 21st century exposure methods, then you have to define exactly what you want. Otherwise it is obviously more useful to most Photographers to stick to references they already know.



 

[NikoJorj]

1. As I understand it, digital sensors aren't as linear as most of those ETTR texts want them to be. There are generally a non-linear minimum and maximum, with a more-or-less linear response in the middle.

The bottom (shadows) non-linearity is noise, but for the highlight non-linearity, it's generally very tiny (it seems however more common that once one channel clip, others becom non-linear).
The main point of ETTR is there, that there is one side of the linear curve where there is almost no non-linearity before clipping, hence the need to take the clipping as upper limit of exposure.

2. How do you know that camera manufacturers aren't already using an ETTR with automatic re-adjusment?

Even if this is a bit a "how do you know we aren't butterflies dreaming we are human" question, I'd say that the mere fact that there is a benefit from exposing to the right in shadow noise, especially for high contrast scenes, proves that we aren't butterflies. And moreover, the ISO12232 norm provides an exposure reference, and does also hints that this hidden ETTR doesn't apply (see dxomark, some other "hidden" features such as fake lowest ISO, or a bias in ISO values, are uncovered using this reference).

3. Perhaps you want to increase exposure at the expense of linearity. Perhaps less noise is more important to you than a linear response. I believe that Canon are doing something to that effect with high-light priority mode. You get some extra dynamic range, but possibly less accurate and harder to control color.

Did you see any changes of colorimetry using the canon HTP mode, outside of DPP (which seems to use a slightly different color profile for HTP captures), outside of clipping problems of course?
Keep in mind that HTP is basically only a slight underexposure relative to the middle values, giving a bit more headroom for the highlights - having the exposure nailed to the highlights as in ETTR just makes that unnecessary.

4. And additionally: I don't believe any of us really want a "linear" histogram.

Oh, yes, I don't want it too, I want it to be logarithmic so that it can be graduated in Fstops of course.

I think what we want is rather clear : an histogram just showing the numeric values present in the raw file, with no further processing applied. Guillermo Luijk said it better than me a few years ago anyway...

[madmanchan]

I've suggested it in another thread but I think it's worth mentioning again: I don't think the histogram (linear, logarithmic, or otherwise) is very useful for determining ETTR or clipping, because it doesn't show you WHERE in your image the clipping happened. For this purpose, I think an over-the-image clipping visualization (I.e., blinking warning or similar) is much more useful because it can show exactly where the clipping occurred. And this can be meaningfully done per raw channel. (Live preview images are subsampled anyways so the fact that Bayer planes are subsampled is not an issue.).

[32BT]

The bottom (shadows) non-linearity is noise, but for the highlight non-linearity, it's generally very tiny (it seems however more common that once one channel clip, others becom non-linear).
The main point of ETTR is there, that there is one side of the linear curve where there is almost no non-linearity before clipping, hence the need to take the clipping as upper limit of exposure.

I was referring to the sensor as transistors. All transistors, including the light sensitive ones, are non-linear. It may take a lot of very meticulous and careful electronic balancing to get the data from the linear part of the curve. The RAW data we usually see in files, is obviously not related to the response curve of the sensor considered as transistors. Probably far from it.

Plus another issue: once you have an exact linear response, how do you (we) propose to benefit from ETTR? If the response is linear, then shifting the data makes no difference. The reason that ETTR works is because it is a multiplication, not a transition.

So you can only get ETTR plus its benefits when you specifically dial in a different (longer) exposure, not by dialing in a standard exposure where the camera shifts the data to the right if at all possible. (Note that for most landscape situations changing exposure time automagically may be possible, but for the majority of photography, this is obviously not what we want).

Oh, yes, I don't want it too, I want it to be logarithmic so that it can be graduated in Fstops of course.

I understand, but my other question remains:

- do you want Fstops? (= gamma 2.0)
- do you want gamma 2.2 / sRGB / Lab?
- or user selectable?

And more importantly: what will be the our reference in bringing the data back to some reasonable reproduction, considering that the data usually represents overexposure?


ps. Yes, I did experience rather annoying problems with HP mode in Canon. Specifically, blue skies with gray/white "fluffy" clouds. Both in DPP as well as Lightroom. Only after the custom profile option back then, things became somewhat more controllable. But that is too long ago to really know for sure where the real problems came from.

I did once decode Canon RAW files and one particularly interesting anomaly I noticed was this: there is data available below and above a certain threshold, but in the histogram it is non-continuous data. You have to specifically "compress" that data back to the continuous part. Which by the way is an interesting problem, because it somehow requires a mid-point reference to determine what needs to be compressed toward light, and what toward dark…

While i realize this is somewhat technical, it may also be important because of the logarithmic RAW histogram we propose. For Canon RAW files for example, the gamma curve is not applied to the exact zero origin, but rather to a specific lower threshold level. This may not be true for other manufacturers, but obviously has significant impact on what we will be looking at…

[32BT]

I've suggested it in another thread but I think it's worth mentioning again: I don't think the histogram (linear, logarithmic, or otherwise) is very useful for determining ETTR or clipping, because it doesn't show you WHERE in your image the clipping happened. For this purpose, I think an over-the-image clipping visualization (I.e., blinking warning or similar) is much more useful because it can show exactly where the clipping occurred. And this can be meaningfully done per raw channel. (Live preview images are subsampled anyways so the fact that Bayer planes are subsampled is not an issue.).

Per channel clipping visualization would seem useful indeed. With the caveat though that we would need to know what exactly constitutes "clipped data". For Canon files for example, there is an upper limit above which there is data, but the Canon processing may not actually use it. So the data is available, but a Canon workflow would clip that data?