A77 raw files - ETTR not applicable?

[maxmelvin19]

I am a relative beginner and I am thinking of getting the Nex 7 as my main digital camera. I have read that the A77's raw files do not benefit from the ETTR method because Sony are doing something strange to the raw files. The claim is as follows:

'...If you are shooting in good light, don’t bother exposing to the right, as it seems that the RAW file stores almost the same number of levels per stop, so there is no nett(sic.) gain by exposing to the right...[T]here is no way to buy yourself any more processing headroom by ETTR (stuff like increasing clarity and contrast will increase noise, and if you tried to boost a properly exposed, low contrast subject, you will get more noise. Traditionally, if you ETTR, reduced exposure in ACR and then boosted contrast, you’d get better results).'

Source: http://www.sonyalpharumors.com/a77-raws-are-not-real-raw/

I don't mean to troll whatsoever. I just would like someone who knows about the A77 and this purported issued to fill me in with the correct answer - or at least clarify. Because if it is true and the Nex 7's raws are processed the same way then maybe I have to learn a different exposure technique to get the best low ISO image quality for landscapes and the like...

Thank you,
Max

[maxmelvin19]

http://forums.dpreview.com/forums/readflat.asp?forum=1037&message=39655557

This link shows the above claim's alleged evidence.

Thanks again,
Max

[bjanes]

I am a relative beginner and I am thinking of getting the Nex 7 as my main digital camera. I have read that the A77's raw files do not benefit from the ETTR method because Sony are doing something strange to the raw files. The claim is as follows:

'...If you are shooting in good light, don’t bother exposing to the right, as it seems that the RAW file stores almost the same number of levels per stop, so there is no nett(sic.) gain by exposing to the right...[T]here is no way to buy yourself any more processing headroom by ETTR (stuff like increasing clarity and contrast will increase noise, and if you tried to boost a properly exposed, low contrast subject, you will get more noise. Traditionally, if you ETTR, reduced exposure in ACR and then boosted contrast, you’d get better results).'

Source: http://www.sonyalpharumors.com/a77-raws-are-not-real-raw/

I don't mean to troll whatsoever. I just would like someone who knows about the A77 and this purported issued to fill me in with the correct answer - or at least clarify. Because if it is true and the Nex 7's raws are processed the same way then maybe I have to learn a different exposure technique to get the best low ISO image quality for landscapes and the like...

Thank you,
Max

I don't know about this camera, but the theory for raw files and ETTR is the same for any camera. The rationale for ETTR is to maximize the exposure to increase the signal to noise ratio. It has nothing to do with the number of levels. The newer Sony sensors do have very low read noise levels and the shadows can be brought up with good effect. However, you will get better IQ with proper ETTR, but you can be a bit conservative to avoid blowing highlights.

See the analysis of ETTR by Emil Martinec. It applies to all cameras.

Regards,

Bill

[ErikKaffehr]

Hi,

Based on my experience with Sony Alpha 900 and Alpha 55 SLT I would be a bit careful with ETTR. I would use ETTR but base exposure on blinking highlights or the unclipped histogram. Conventional knowledge says that the histogram is conservative, but increasing exposure beyond clipped histogram or blinking highlights usually ends i clipped highlights. So my suggestion would be use ETTR but don't assume that you can expose more than what histogram and blinking highlights allow.

Best regards
Erik

I don't know about this camera, but the theory for raw files and ETTR is the same for any camera. The rationale for ETTR is to maximize the exposure to increase the signal to noise ratio. It has nothing to do with the number of levels. The newer Sony sensors do have very low read noise levels and the shadows can be brought up with good effect. However, you will get better IQ with proper ETTR, but you can be a bit conservative to avoid blowing highlights.

See the analysis of ETTR by Emil Martinec. It applies to all cameras.

Regards,

Bill

[jonathanlung]

As Bill said, ETTR theoretically applies to any (digital) camera. The conclusion about ETTR in the dpreview post, based on the poster's observations, are erroneous. Instead, the practical implications of the data are that one should ETTR, but not use any ISO past 1000 on the NEX 7. That is, for a given exposure value, if the slowest useable shutter speed has been reached, ISO should be raised, but only until ISO 1000; after this, the image should be pushed in post since, according to the dpreview post, continuing to raise ISO will buy you nothing for shadows and, therefore, serves only to reduce recorded DR. This thread is relevant.

[allegretto]

I am a relative beginner and I am thinking of getting the Nex 7 as my main digital camera. I have read that the A77's raw files do not benefit from the ETTR method because Sony are doing something strange to the raw files. The claim is as follows:

'...If you are shooting in good light, don’t bother exposing to the right, as it seems that the RAW file stores almost the same number of levels per stop, so there is no nett(sic.) gain by exposing to the right...[T]here is no way to buy yourself any more processing headroom by ETTR (stuff like increasing clarity and contrast will increase noise, and if you tried to boost a properly exposed, low contrast subject, you will get more noise. Traditionally, if you ETTR, reduced exposure in ACR and then boosted contrast, you’d get better results).'

Source: http://www.sonyalpharumors.com/a77-raws-are-not-real-raw/

I don't mean to troll whatsoever. I just would like someone who knows about the A77 and this purported issued to fill me in with the correct answer - or at least clarify. Because if it is true and the Nex 7's raws are processed the same way then maybe I have to learn a different exposure technique to get the best low ISO image quality for landscapes and the like...

Thank you,
Max

Have an A77 and I can tell you from my 2 weeks experience that ETTR is of limited value. As with any camera it's best to paint-by-the-numbers and get adventuresome in steps. Deciding to ETTR out of the box does a disservice to your learning about the capabilities of the unit.

I didn't mind the gentleman's article you referenced, but he's sure got no shortage of humility deciding that Sony's engineers acted "stupidly" in their decisions about trade-offs. When I read things such as this it immediately makes me suspect of the writer (not that I have any knowledge of this fellow however, just that his Ego outshines his understanding).

In any case, shoot your NEX-7 "as written" first, then start blowing out ISO's if needed. However, since PP tools like LR are available I wonder why anyone would want to clip highlights in the first place...

[NikoJorj]

This link shows the above claim's alleged evidence.

Isn't such a mapping already implemented in compressed leica and/or nikon raws?

That makes sense, because, let us repeat that altogether, brothers and sisters :
"Expose to the Right is not about levels and quantification, it's about noise and clipping".

Instead, the practical implications of the data are that one should ETTR, but not use any ISO past 1000 on the NEX 7.

Amen!


As a side and off-topic note, sorry but I couldn't resist :

I can tell you from my 2 weeks experience that ETTR is of limited value [...] he's sure got no shortage of humility [...] his Ego outshines his understanding [...]

Mmmm, what was that old tale of motes and beams?

[bjanes]

I didn't mind the gentleman's article you referenced, but he's sure got no shortage of humility deciding that Sony's engineers acted "stupidly" in their decisions about trade-offs. When I read things such as this it immediately makes me suspect of the writer (not that I have any knowledge of this fellow however, just that his Ego outshines his understanding).

In any case, shoot your NEX-7 "as written" first, then start blowing out ISO's if needed. However, since PP tools like LR are available I wonder why anyone would want to clip highlights in the first place...

I just glanced at the post of the individual who is accused of lacking humility. He states that above a certain ISO, the camera merely multiplies the data to obtain a higher "ISO". Actually with the new breed of "ISO less" cameras, there is nothing wrong with this. See this post by Emil regarding this matter. Many MFDBs also use multiplication for ISO.

ETTR should really apply to base ISO. With this type of camera, if the exposure is limited by shutter speed considerations or other constraints, one should give as much exposure as possible, and then increase exposure further in the raw converter. This will allow extra headroom and help prevent blowing out of highlights. In this situation, increasing the ISO moves the histogram to the right but does nothing to increase the number of photons collected.

Increasing the ISO on the camera would allow the ADC (analog to digital converter) to use more of its scale, but the increased number of possible levels would be dithered by noise and of little use. It does not make sense to quantize the data to a finer level than the noise.

Regards,

Bill

[hjulenissen]

As Bill said, ETTR theoretically applies to any (digital) camera.

I believe not necessarily digital cameras with non-linear response (like fujis CCD-HR or whatever it was called).

-h

[bjanes]

I believe not necessarily digital cameras with non-linear response (like fujis CCD-HR or whatever it was called).

-h

I don't know much about Fuji sensors, but if you do not fill the wells of the sensor, you will not get optimal signal:noise. SNR = sqrt(number of photons collected). I think this is simply photon counting statistics.

Regards,

Bill

[allegretto]

I just glanced at the post of the individual who is accused of lacking humility. He states that above a certain ISO, the camera merely multiplies the data to obtain a higher "ISO". Actually with the new breed of "ISO less" cameras, there is nothing wrong with this. See this post by Emil regarding this matter. Many MFDBs also use multiplication for ISO.

ETTR should really apply to base ISO. With this type of camera, if the exposure is limited by shutter speed considerations or other constraints, one should give as much exposure as possible, and then increase exposure further in the raw converter. This will allow extra headroom and help prevent blowing out of highlights. In this situation, increasing the ISO moves the histogram to the right but does nothing to increase the number of photons collected.

Increasing the ISO on the camera would allow the ADC (analog to digital converter) to use more of its scale, but the increased number of possible levels would be dithered by noise and of little use. It does not make sense to quantize the data to a finer level than the noise.

Regards,

Bill

I simply meant that when you call carefully selected parameter compromises made of engineering necessity "stupid", you'd better know a great deal more than those who made those compromises or your comments seem... egotistical at best.

[allegretto]

As a side and off-topic note, sorry but I couldn't resist :Mmmm, what was that old tale of motes and beams?

Wow, tough crowd...

Hey, I don't pretend to know half of what many of you are obviously quite familiar with. Just that when someone starts throwing around "stupid" when speaking of design parameters I get... cautious about their advice unless they are proven well up the curve. My professional crucible is a fierce environment. Often folks who toss off such disparaging terms are unaware of all the considerations that go into the decision and therefore their judgement is of limited application.
In any case, I'm quite certain the EE's and designers at Sony are not "stupid' and don't make "stupid" designs in most cases.

But perhaps you see things differently...

Insofar as ETTR and the A77, it seems easy to blow out the R side of the histogram and then have to take down the exp to remove most of the clipping. Perhaps that has more to do with my subjects than the camera itself. Not sure. But what I can tell you is that in most experience where one must find a balance point and one is dealing with a new system, incremental changes make sense to me.

Again, you may approach things differently. I thought the OP was looking for opinions... truth may be harder to come by.

[jonathanlung]

I don't know much about Fuji sensors, but if you do not fill the wells of the sensor, you will not get optimal signal:noise. SNR = sqrt(number of photons collected). I think this is simply photon counting statistics.

Regards,

Bill

The Fuji SuperCCD-HR sensors is Fuji's high resolution sensor; I think, hjulenissen, you meant the SuperCCD-SR (correct me if I'm wrong) which is found in the S5 Pro and has a greater DR than their other sensors. I also agree that a logarithmic sensor (or a camera that records in a such a manner) would not benefit from ETTR, but this is not, as I understand it, the way Fuji does it. Fuji paired a large photodiode with a small one on each photosite; each photodiode still acts as a photon counter, so Bill's statement above holds for the sensor itself.

Thus, more precisely than "digital" would have been to say that any sensor/recording format that effectively has a finite full-well capacity and records data in a logarithmic or sub-logarithmic manner would not (necessarily) benefit from ETTR.

[bjanes]

The Fuji SuperCCD-HR sensors is Fuji's high resolution sensor; I think, hjulenissen, you meant the SuperCCD-SR (correct me if I'm wrong) which is found in the S5 Pro and has a greater DR than their other sensors. I also agree that a logarithmic sensor (or a camera that records in a such a manner) would not benefit from ETTR, but this is not, as I understand it, the way Fuji does it. Fuji paired a large photodiode with a small one on each photosite; each photodiode still acts as a photon counter, so Bill's statement above holds for the sensor itself.

Thus, more precisely than "digital" would have been to say that any sensor/recording format that effectively has a finite full-well capacity and records data in a logarithmic or sub-logarithmic manner would not (necessarily) benefit from ETTR.

Log recording would improve quantization of the lower bits, but I don't know if quantization or noise is the limiting factor there. At high ISO, I would suspect that noise would be limiting. Emil has stated that raw files are never posterized, and he is usually correct. If he sees this thread, his comments would be helpful. Nonetheless, collecting more photons would improve SNR regardless of the encoding, whether it be linear, log, or gamma. The high luminances have excess number of bits (levels), as shown by the success of Nikon's lossy compressed NEF encoding, where excess numbers of levels are thrown away. There still are enough levels for effective highlight recovery.

Regards,

Bill

[NikoJorj]

Wow, tough crowd...

Sorry if sounded too rude.

Just that when someone starts throwing around "stupid" when speaking of design parameters I get... cautious about their advice unless they are proven well up the curve.

I do completely agree (and even more so if the post is on DPR - good posters, but also much much noise there). What I was saying is that I felt you were just doing the same with your commentary about ETTR.

(edited unquoted quote)

Insofar as ETTR and the A77, it seems easy to blow out the R side of the histogram and then have to take down the exp to remove most of the clipping.

So, let's repeat altogether once more, brothers and sisters :
"ETTR does mean NOT clipping any raw channel, unless it's wanted by the photographer".
If you blow the R side of the histogram, you take risks instead of exposing to the right. If you can recover the blown areas in post then you're lucky (you just blew the jpeg preview but not the raw data), but if you can't recover an important area of the image leading to unwanted hue shifts and/or texture loss, you clipped the raw data where it shoudn't have been and failed to expose to the right.

That brings us to...

In any case, I'm quite certain the EE's and designers at Sony are not "stupid' and don't make "stupid" designs in most cases.

Statistically true maybe, but not logically. If they were that perfect, they would have given us the right tools to perform and assess exposure properly.  

[NikoJorj]

Emil has stated that raw files are never posterized, and he is usually correct.

Guillermo Luijk has a good demo :

(Click to see source page in spanish)
5 bits vs 8bits in B&W, and it's hard to see the difference though, due to noise.
Some examples are also on Emil Martinec's article about noise.
Then, there is the problem of color : from what I've seen, the quantization noise (errors induced by a less-than-optimal quantization) is a bit more visible to the eye in color than in B&W (see example linked below).

But so far the examples I've seen of reduced-bit-depth images (example) show that underquantization must be really severe, say something akin to 10 or 9bits on a linear-encoded raw, to be perceptible.
With 12bits-raws and today's sensors, the limiting factor seems to be noise and not posterization.
14bits seems to benefit a tad on today's better sensors as the k5's one, but it's not very apparent ; we'll need sensors with even less noise at base ISO to see it a real gain with 14 bits.

[maxmelvin19]

Thank you all for taking the time to reply.

I'm still a little muddled. So we're all fairly clear that ETTR probably isn't that helpful above ISO 1000. What about at base ISO? I think one of the claims the source I provided makes is that the A77 raw process is not like, say, the A700 because it is non-linear. Thus...

...more precisely than "digital" would have been to say that any sensor/recording format that effectively has a finite full-well capacity and records data in a logarithmic or sub-logarithmic manner would not (necessarily) benefit from ETTR.

So there seems to be a common responce of the form: 'If the A77 is like all other dslrs then its required (or best low ISO) exposure technique will be like all other dslrs' - undeniably logical but I'm not sure it answers my question for I am questioning the assumption that A77 processes raw files like all/most other dslrs.

So does anyone have any evidence to show that the A77 records data in a linear fashion? If so, I can happily ignore the source. If not, or further anyone has any evidence to the contrary, then some advice on best exposure techniques for a non linearly recorded raw file at low ISOs would be very much appreciated.

Thanks again,
Max

[hjulenissen]

The Fuji SuperCCD-HR sensors is Fuji's high resolution sensor; I think, hjulenissen, you meant the SuperCCD-SR (correct me if I'm wrong) which is found in the S5 Pro and has a greater DR than their other sensors. I also agree that a logarithmic sensor (or a camera that records in a such a manner) would not benefit from ETTR, but this is not, as I understand it, the way Fuji does it. Fuji paired a large photodiode with a small one on each photosite; each photodiode still acts as a photon counter, so Bill's statement above holds for the sensor itself.

Thus, more precisely than "digital" would have been to say that any sensor/recording format that effectively has a finite full-well capacity and records data in a logarithmic or sub-logarithmic manner would not (necessarily) benefit from ETTR.

You are right in that I thought about the CCD-SR, where an on-chip "HDR" is synthesized by having a heterogenous set of sensels - some pixels have less sensitivity (catering to highlights), while others have higher sensitivity (taking care of shadows). At least, that is my simple understanding of it.

I was assuming that the "highlight sensels" might not have the same SNR as the regular sensels, at least not when operating close to the switchover-point. The plot of "per pixel SNR vs input level" should then be more complicated than we are used to (having a wrap-around?)The point of this technology seems to be to provide a non-linear relationship between image brightness (per pixel) and recording accuracy, mimicing more closely how our vision operates, how analog film works etc.  If that is the case, it would seem that ETTR is less useful for those cameras?

http://www.digitalreview.ca/cams/FujiSuperCCD.shtml
-h

[NikoJorj]

So does anyone have any evidence to show that the A77 records data in a linear fashion?

Analysis presented in the DPT thread indicates that encoding is not linear.
There is no evidence relating to the response of the sensor itself ; a non-linear response could be a more serious argument against ETTR, but a non-linear encoding is not.

If not, or further anyone has any evidence to the contrary, then some advice on best exposure techniques for a non linearly recorded raw file at low ISOs would be very much appreciated.

As have already been written here, expose to the right nonetheless, it will reduce noise in the shadows and ensure clean, unclipped highlight as you intend them.
The posterization is not an issue, whereas noise is.

[allegretto]

Sorry if sounded too rude.
I do completely agree (and even more so if the post is on DPR - good posters, but also much much noise there). What I was saying is that I felt you were just doing the same with your commentary about ETTR.

(edited unquoted quote)
So, let's repeat altogether once more, brothers and sisters :
"ETTR does mean NOT clipping any raw channel, unless it's wanted by the photographer".
If you blow the R side of the histogram, you take risks instead of exposing to the right. If you can recover the blown areas in post then you're lucky (you just blew the jpeg preview but not the raw data), but if you can't recover an important area of the image leading to unwanted hue shifts and/or texture loss, you clipped the raw data where it shoudn't have been and failed to expose to the right.

That brings us to...Statistically true maybe, but not logically. If they were that perfect, they would have given us the right tools to perform and assess exposure properly.  

Hey, you're not too rude for me. Just giving it back in a fun way. I think folks here have great Net manners with each other, and in a couple of weeks of lurking I've learned more about digital than I ever knew. Read Emil's posts here and there and I'm understanding and amused by the differences between emulsion (of which I knew some things) and electronic. Funny how some issues are similar and some are exactly the "other way". So, I'm loving this

Yes, I spoke up about my nascent experience with the A77, but I knew no one had much more experience than me and I was only commenting upon what I've seen, not trying to getting technical. It's very interesting camera and takes very nicely exposed photos

As far as the "tools" for ETTR. I won't question the utility if you guys feel you need it. But after reading Emil's input and considering the Exmoor processor I think I'll just leave the ISO at 160 (or anything below 1000 I guess) and take pictures based upon what SS/f-stop I want and work on it in LR. Makes sense to me (once I realized the issues, just fill the darn pixels). Curious as to why you think Sony is perhaps "stupid" for doing what it seems all manufacturers are doing. To ascribe stupidity, or perhaps even worse motives as some seemed to do in the thread you linked seems misguided to me. Yes, they all make profit, as much as they can. But don't go Occupy Wall Street because they have a way of doing things. Bug them, as it seems some are doing. Until they "get it" with RAW histograms, maybe old fashioned Bracketed ETTR is the way to go? With a camera as the A77 it's very fast and easy. The downside isn't any big deal, much less onerous than emulsion...

Anyway, happy to be here and learn. I'll sit down now and learn some more.