Camera manufacturers PLEASE: when RAW histograms and an ETTR mode?

[Guillermo Luijk]

Sometimes I wonder if I am such a strange user to be praying for RAW histograms and automatic ETTR (Expose to the right) mode on my camera.
If many users shoot RAW, why after 4 years of DSLR we are still looking at JPEG histograms and clipped info? why are we doing endless trial/error shots to _try to achieve_ an optimum ETTR of the RAW channels instead of having the tools to easily _achieve_ it? why cameras measure light and calculate exposure to obtain a pleasant JPEG instead of optimised RAW data? why not allow both modes of operation? JPEG-oriented and RAW-oriented.


RAW HISTOGRAMS

All the mess of UniWB wouldn't make sense if just cameras would allow to display the undemosaiced RAW histograms, which are simpler to calculate and display than the JPEG histograms. Why not allow the user to choose?
A RAW histogram (it could even be logarithmic arranging the info in real EV divisions) of the 3 channels would allow at a glance to find out how well your RAW was exposed.

This is a perfectly exposed (but just because of a lucky shooter) RAW on an Oly camera, easy to see and clear to understand:



Also the DR of the scene is easily evaluated thanks to it, and hence you can quickly know the amount of noise you can expect in the shadows without even looking at the image, and find out how many extra shots are needed to capture all DR. Is it so difficult to add that to your cameras?

And the same would apply to the clipped blinking information, it should be possible to refer it to the RAW data.


AUTOMATIC ETTR MODE

With Live View cameras can evaluate the histogram of the scene in front of you in real time. Why don't use that valuable source of information to set up a mode for automatic ETTR? the camera would calculate exposure (aperture/shutter/ISO) to obtain a properly ETTR RAW file. A user setting could be % of blown pixels allowed in the RAW data.

If the Live View histogram is not precise enough (maybe it is calculated from an auxiliar sensor), why don't make then a quick preview shot (it can be very high ISO, no problem, and it _must_ be underexposed in order not to clip any channel), it could even be transparent to the user. With that information, the corrected exposure values can be calculated in a fraction of second to achieve the perfect ETTR in the real shot right afterwards.

I know Sony cameras can simulate the histograms you would get after you shot; if you move the exposure wheel histograms change in front of your eyes. Why don't use that in a real time exposure calculation?

I did it on my software and it's trivial to calculate the needed exposure correction to automatically achieve ETTR on a given set of data:




Someone would sign for this wish list or add new ideas?

BR

[douglasf13]

I couldn't agree more.  Bring on the RAW histograms!

[JBerardi]

Right? For all the gnashing of teeth over tiny differences in top-end DSLR image quality, huge deficiencies such as the lack of RAW-based histograms go almost completely unnoticed. It's kind of like buying a Porsche and finding out they didn't bother to put a tachometer on the dash.

[Hoang]

I wouldn't hold my breath. Manufacturers seem to be more interested in features that attract attention, but ignore those that would actually be useful.

[Vivec]

Both would be great features -- I have been wondering about this from the day I started using raw. The histogram would be really easy. The ETTR exposure may prove harder in practice but could be done in theory for sure, especially with a widely spaced grid of light meters (like in most modern DSLR's).

[douglasf13]

Right? For all the gnashing of teeth over tiny differences in top-end DSLR image quality, huge deficiencies such as the lack of RAW-based histograms go almost completely unnoticed. It's kind of like buying a Porsche and finding out they didn't bother to put a tachometer on the dash.

  Yeah, no kidding.  As an A900 user, I always crack up when fellow A900 users comment on how good/bad their camera is compared to others, and then underexpose and use ACR to convert.  Too many drunk drivers

[Guillermo Luijk]

The ETTR exposure may prove harder in practice but could be done in theory for sure, especially with a widely spaced grid of light meters (like in most modern DSLR's).

I don't think multipoint light metering would be the way. After all you have 20 million light meters in your sensor! Why not perform a super-quick underexposed shot at high ISO prior to the real shot, analysing the RAW file obtained in that preliminary shot, and then adjust exposure settings to obtain the perfect ETTR in the final shot? even some intelligence could be applied in the algorithm, such as spatial clipped pattern recognition (e.g. the circle of the sun or artificial lamps on indoor and night shooting, specular reflections,...).

In fact the Sony A900 performs histogram simulation. You can move the exposure wheel after a quick shot and find out how the histogram would have been with any arbitrary exposure setting. You just need that to be done automatically in-camera to have an ETTR mode.

[Daniel Browning]

If many users shoot RAW, why after 4 years of DSLR we are still looking at JPEG histograms and clipped info?

Because they don't care. Photography is not important to the corporate giants. All they care about is making money. Their market research must be telling them that they can make more money by not implementing these highly valuable features, despite how easy they would be to complete.

Perhaps informed photographers are so few and far between that only a very tiny minority of people realize the void left by these missing features. Perhaps our lamentations don't even make a blip on their radar. I would like to think their market research is flawed, but either way the answer is because they don't know or don't care.

What would it be like if there was a manufacturer who actually *did* listen to raw photographers? Where lowly "customers" can talk directly to the billionaire owner and the rest of the employees? And implemented their requests for raw histograms, eradication of variable pattern noise, ISO-as-metadata, black bias, no color matrix, etc.?

That would be RED Digital Cinema. The camera costs a cool $20K, but they implemented all the features that we've been begging for. On their first try. Canikon has had almost a decade and still hasn't bothered to implement these simple requests. RED even has a false color mode that shows the raw exposure level of everything in the scene (like the infrared sight of "Predator" as in "Predator-vs-Aliens").

I'll keep dreaming that some day one of the manufacturers will bother with features that are vital to my photography. In the mean time, I'll keep using the tiresome workarounds (uniWB, -5 EC instead of ISO 25600, etc.).

[cmi]

Sometimes all it takes is having the idea. Representatives of the various companies are reading here I would guess, and I wouldnt be too surprised if someone just implements it. Seriously.

On a second throught: How would one go about implementing it? For me it appears as an option wich would correct exposure time, aperture or iso - depending on wich shooting mode I am in. (E.g. in aperture mode it corrects the time, in time mode it works on aperture.) This would be all under the hood, just an on/off switch or menu item wich works shooting-mode dependent.

Or in other words, it would be just another metering option. (ETTR metering.)


Regards

Christian

[thierrylegros396]

I wouldn't hold my breath. Manufacturers seem to be more interested in features that attract attention, but ignore those that would actually be useful.

On the G10 for example, they prefer to add "my sound" menu and other "non necessary" things  

[thierrylegros396]

A user setting could be % of blown pixels allowed in the RAW data.

Exactly what I thought yesterday      

YES, RAW histograms are necessary !!!

Thierry

[NikoJorj]

Sometimes all it takes is having the idea.

Then, manufacturers really do need some imagination, or better said they need to put some drug controls in their factories to have their ingeneers design useful features instead of overcharging the main dials with funky "cat whiskers priority" modes...  
RAW-based histograms and ETTR and talked about regularly since at least 2003.   (The LL article that put the spots on the subject is of 2003, and here is a random example of 2007).
I just can't but second Guillermo's request, of course, with a few caveats.

As with last-millenia 18%-based matrix exposure, any automated exposure is prone to errors, and in the present case knowing the limit between what not to blow (important texture) and what to blow
(sun or light sources, specular reflections...) is not that easy for a machine - it's just an artistic choice in some cases.

The ability to choose the percentage of blown pixels would then come quite handy, and it would then be very useful to display a broader histogram, showing what stands in the blown area (at least 1 or 2EC past saturation), to give a hint on the eventual -EC needed.
Another solution would be not to allow anything to blow, but having at hand a greater amount of +EC (for the case where the sun is in the frame) - not as practical, I'd think.

For this specular reflection problem, it might also be interesting to keep those oldthink matrix meters, because they effectively offset little specular reflections (at the expense of accuracy) - and besides that, they don't need the shortcomings of live view mode.

[cmi]

... ETTR ... talked about regularly since at least 2003. ...

Ok thats a different story, one indeed would think that the engineers catched up the idea since then!

[Panopeeper]

I would leap for joy if my camera had only half of it: showing the raw histogram before shooting, so that I can save the multiple exposures. If one thinks about live view, one sees that this should not be difficult at all; even more, it would be easy to "predict" the clipping and indicate it with flashing, based on the selected ISO and exposure.

I don't think manufacturers see this as an important point, and probably they are right. I have been spreading the idea and the technique of ETTR on several forums, repeatedly for newcomers. My experience is, that people are excited about it first, but only very few are actually doing it. Of course, one could say more photogs would be using it if it was much simpler, but I am not sure.

The other side of it is the amount of complaint the manufacturers could expect by the masses of people, who would not understand what they are doing and why the raw histogram shows everything in scope, while the preview is horrendeously clipped, etc.

[terence_patrick]

I ETTR myself, but just to play devil's advocate, ask yourself how many people with DSLRs out there in the world even know what that concept means? Now imagine you're Canon or Nikon and try from a business standpoint to explain to the millions of people who've purchased your supposedly top-shelf camera that your metering has been wrong this entire time and now you should do it a new way and overexpose by X-amount. Or heck, what about those people who still shoot JPG? I imagine there are more people in the world who shoot JPG on a DSLR than those who shoot RAW.

It's kind of like when film was around and everyone knew that Portra 160nc could be shot at 80iso to open up the shadows. Or that Tri-X 400 could be exposed at 200. The ETTR concept has been around for a pretty long time, but I don't think that camera manufacturers should be changing things now.

[Dale_Cotton2]

The other side of it is the amount of complaint the manufacturers could expect by the masses of people, who would not understand what they are doing and why the raw histogram shows everything in scope, while the preview is horrendeously clipped, etc.

To eliminate quite a bit of this problem all that would be needed is to hide or disable the options for ETTR metering and raw histograms based on whether the camera is in raw mode or not. So the standard matrix metering position on the dial would do ETTR if both the camera is set for raw output AND the appropriate custom function is enabled.

Terence Patrick wrote:

ask yourself how many people with DSLRs out there in the world even know what that concept [ETTR] means?

If ETTR metering were called highlight protection that might make it a bit more comprehensible to more photographers. The manual would then explain that having highlight protection enabled in high contrast light will likely result in images that appear too dark overall, and therefore would require post processing (this for JPEG shooters). And that of course leads to the idea of having the camera automatically modify the tone curve to open the shadows or of having this as one of those post-exposure JPEG adjustment menu options, like sepia toning or Velvia saturation.

Since camera marketing is highly competitive, and manufacturers are always on the look-out for new features to add to the laundry list for a given camera, both highlight protection metering and raw histograms would actually benefit whichever company got them out first. All that's needed to make this a plus instead of a minus is to use accessible terminology.

[Panopeeper]

If ETTR metering were called highlight protection that might make it a bit more comprehensible to more photographers

That would be sad, as ETTR is just the opposite of highlight protection: it is shadow protection. Canon had a good reason to name "highlight tone priority" a function, which underexposes the shot by one stop.

[Dale_Cotton2]

That would be sad, as ETTR is just the opposite of highlight protection: it is shadow protection. Canon had a good reason to name "highlight tone priority" a function, which underexposes the shot by one stop.

If the DR of the scene covers more range than the camera can record, and we expose so the brightest regions butt up against the right edge of the histogram, aren't the losses going to be in the shadows? That's what I had in mind, at any rate. But I don't care what the heck they call it as long as they do it. ;)

[Panopeeper]

If the DR of the scene covers more range than the camera can record, and we expose so the brightest regions butt up against the right edge of the histogram, aren't the losses going to be in the shadows? That's what I had in mind, at any rate

This's not what ETTR is about; this is preventing clipping (one could say it is prudent exposure).

ETTR is about exposing higher than metering suggests, if this can be done without causing clipping. When creating in-camera JPEG, this method leads to overexposed result, even if actual clipping did not occur. The raw conversion with default parameters too looks like overexposed; normally one has to counter this with adjustments, for example by "Recovery" or by reducing the "Exposure" or "Brightness" in ACR. The goal of ETTR is to reduce the noise in the very dark regions.

[Guillermo Luijk]

I think a correct introduction of an ETTR mode would prevent any misunderstanding from users. It's clearly that camera vendors are not (yet) interested in this.

ETTR should be clearly introduced as:

- A mode only for RAW shooting since the concept itself (exposure correction required in postprocessing) doesn't make sense in JPEG.
- An advanced mode for users who know well what it means and what they want to achieve. It's a 'use at your own risk' philosophy.
- ETTR doesn't necessarily mean overexposing. I am in favour to consider it a method to prevent clipping with maximum exposure (I am not with Gabor here).
- ETTR shouldn't actually be an extra metering mode, but a new exposure calculation mode. So it could work in parallel to preexisting metering philosophies. For example ETTR + spot metering would mean you want to expose the metered point as much as possible right before clipping it. This spot doesn't need to be the most luminous point in the scene and this would be very useful to quickly choose which areas are to be ETTR'ed allowing any higher area to clip.

However for a precise calculation, it is a must here to have detailed light data gathered via Liveview or a preview shot. Just light metering wouldn't be precise enough.


Regarding the algorithm to use when evaluating the entire scene lightness, so that ETTR mode doesn't make wrong decisions ending in heavily underexposed shots because of the presence of high luminance elements (sun, lamps, speculars,...), I don't think it's difficult at all to detect shapes (continuous areas of adjacent pixels) in the scene with a high degree of light, several stops over any other part of the image, and ignore them for the calculation.

This can be easily seen in any histogram in the form or a high peak at saturation with a lot of empty levels right before:



This is not the best example, but good enough to show that little peak on the right end preceeded by empty levels. That was a small specular reflection in the car made of connected pixels easy to identify and ignore for the ETTR exposure calculation: