Pages: [1] 2   Go Down

Author Topic: Posterized Highlights-Where's my 4096 levels in 1st Stop ETTR shooting Raw?  (Read 9078 times)

Tim Lookingbill

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 2436

I've read all I can on the subject on how digital camera sensors record data linearly where the first stop of brightness takes up half the total 12 or 14bits of recorded linear data. And the only advantage of working in a high bit linear space is to prevent posterization when editing to get all the detail and definition contained in the captured Raw image.

So why do I get posterization editing to get this detail and definition in the upper brightest portions and even farther down the tonal scale of the image.

Below are two screenshots, the first is the full image with ACR 4.6 defaults with histogram to show I've correctly exposed this image. The second is a 200% zoomed in portion of the rock with the lightest highlights that I've attempted to add definition within the rock detail. You'll note the posterization isn't just in the highlights where it would be understandable for this to occur but also down into the upper mids.

Is this a camera sensitivity issue or is the explanation of all that's been written about the subject of high bit linear sensor data mapping characteristics a bit flawed or misunderstood? I admit I have a hard time understanding the subject when I come up with these kinds of results shown in the posted images.

Can someone make this more clear as to the cause of the posterization?

[attachment=12086:PostRIZE...sDefault.jpg] [attachment=12087:PostRIZE...velsZoom.jpg]



Logged

Guillermo Luijk

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 2005
    • http://www.guillermoluijk.com

Looking at image 2, and those gray blotches that look like posterization, I have a feeling there is no posterization issue here but just partial saturation. That gray colour denotes the areas in which ACR tried to rescue as much detail as possible when one or even two of the RAW channels were blown, and ACR performs this in a neutral (R=G=B ) colour.

So the problem is not posterization, but overexposure. No matter if ACR saturation warning stopped being red when the exposure slider was set to the left; the information was blown in your capture, or at least this is what I think.

If you upload the RAW file somewhere I can check it for you in detail.

BR
« Last Edit: March 12, 2009, 01:50:46 pm by GLuijk »
Logged

Tim Lookingbill

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 2436

Hi, GLuijk.

Here's the link to the original Pentax K100D PEF.

http://www.yousendit.com/download/U0d6a3NaTlEzeUt4dnc9PQ

Could you possibly show me where the 4096 levels of the first stop of brightness and detail resides in this image?

It's not just this image. I've been wondering about this since I started shooting and processing raw and getting these kinds of tinker toy previews editing all kinds of images of varying exposures. It's not just editing the highlites when I get this type of posterization but all throughout the tonal scale. I just remember not getting these kinds of previews editing 16bit scans of negatives. I expected more leeway and smoothness editing 16bit digital camera raw data even though it's an 8 bit preview.

Note on the crinkly curve in the posted image there doesn't seem to be many varied levels of tone. I could put more edit nodes on that curve and still not get many different tones which is why I ask where are all these 4096 different tones I'm suppose to be getting in the 1st stop of brightness.
« Last Edit: March 12, 2009, 03:02:13 pm by tlooknbill »
Logged

Panopeeper

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 1805

I am not GLuijk, but I can do this fast. The first capture shows the clipping; magenta indicates green clipping, red shows, that both green and blue clipped. However, the shot was rather low exposed anyway, see the raw histograms. Again, the DR of the camera is not large enough for the scenery (honestly, is it fair to capture a cave and bright, light colored rocks in the same shot?). The colored dots mark the pixels, which "went black". Regarded the subject, it would have been better to expose it one stop higher; only that piece of rock (and a tiny bit more) would have suffered under it.
Logged
Gabor

Tim Lookingbill

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 2436

Gabor,

I appreciate your analysis of what I should've done exposure wise when taking the shot. And I don't intend on turning this into a discussion that questions the fairness of that type of shot. Fair to who? What? That's not what I'm concerned about.

What I'm asking is where are the 4096 levels of detail/data that should be in the first stop of brightness when I edit the tones in this image. It doesn't feel like I'm editing a 12, 14 or 16bit image. It feels like I'm editing an 8bit jpeg.

And I don't just get that on this shot. I get it on a lot of my images regardless the dynamic range of the scene to the camera's DR.
« Last Edit: March 12, 2009, 03:50:15 pm by tlooknbill »
Logged

NikoJorj

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 1082
    • http://nikojorj.free.fr/

Quote from: tlooknbill
What I'm asking is where are the 4096 levels of detail/data that should be in the first stop of brightness when I edit the tones in this image.
There are 4096 steps in the last recorded stop and none in the N stops above!
So if it's clipped (like Gabor showed it), there will be an ugly posterization.
And bear in mind that this reasoning doesn't take the luminance into account, but rather each channel, so to ensure you benefit from the 4096 levels you should make sur that none of the channels is clipped.

It's a question of playing with fire : it's very funny to play, but less so to burn...
« Last Edit: March 12, 2009, 04:04:21 pm by NikoJorj »
Logged
Nicolas from Grenoble
A small gallery

Guillermo Luijk

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 2005
    • http://www.guillermoluijk.com

As Gabor says the DR of the scene (a cave with outdoor sunny areas) was excessive for a single shot. Looking at how the RAW data spreads in a log histogram we could talk of 10EV or more:

« Last Edit: March 13, 2009, 06:50:32 am by GLuijk »
Logged

Tim Lookingbill

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 2436

OK, I'll put it this way.

This is posterization as I understand it. How many DIFFERENT tones are mapped next to each other when recorded by the sensor in high bit whether in the highlights or upper midtones and when increasing these differences by making one level dark and the next one to it lighter in a raw converter why do I not get more differentiating levels. I get flat areas that indicate that there ARE NO DIFFERENTIATING TONES next to each other recorded by the sensor when there should be if working in a high bit capture and editing environment.

Guillermo and Gabor, have you tried the same multiple tightly spaced pinch node curve edits I demonstrated in my posted image? How do you explain the lack of differentiating tones when doing this.

I'm trying to fit this within the understanding that the brightest first stop has the most levels devoted to rendering these tone differences. We're not talking about photographing and editing an image of a flat smooth wall and looking for superfine smooth tone differences. We're talking about microfine multitone detail in rocks posterizing when placing tightly spaced pinch edits along a curve that supposed to show 4096 or there abouts multiples of tones but only gives posterized results.
« Last Edit: March 12, 2009, 04:51:56 pm by tlooknbill »
Logged

Panopeeper

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 1805

Quote from: tlooknbill
I don't intend on turning this into a discussion that questions the fairness of that type of shot. Fair to who? What? That's not what I'm concerned about.
OMG

Quote
What I'm asking is where are the 4096 levels of detail/data that should be in the first stop of brightness when I edit the tones in this image
The sensor of your camera yield all together 3968 levels, so the talk about 4096 levels in the top stop is a bit off.

As to what exactly in that range is: look at the attachment. As already thew histogram has shown this, there is not much in the top stop.

The surrounding of the clipped area, i.e. that, what has not clipped but was close to it, gets "washed out" because of the S-curve, which reduces the contrast at the very left  and very right end. If you go to the tone curves in ACR and turn down the Highlight slider, you get to see all the available detail on that piece of rock; however, this was the range remaining for the other levels is greatly reduced, i.e. the rest of the images loses contrast.

You can do this somewhat better in the Point curve, restricting the contrast change to the very-very right end. Later, if you separate this rock in PS, you can apply a unique curve to it, without affecting the rest - that's the way I am doing such cases.

Quote from: GLuijk
Gabor, will you call me Guillermo one day?  

I do that all the time, when I am writing to you (pls look it up); however, here I referred Tim's (tlooknbill) post.
Logged
Gabor

Wayne Fox

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 4237
    • waynefox.com

Quote from: tlooknbill
I'm trying to fit this within the understanding that the brightest first stop has the most levels devoted to rendering these tone differences.

It seems the explanations offered are fairly simple, it isn't posterization, it is over exposure.  The reason it looks like posterization is because not all 3 colors are blown, so ACR is attempting to figure out what color is supposed to be there based on the channels that are not blown.  It does this well sometimes, sometimes it doesn't, and when it doesn't some strange things can show up.

You're locked onto the concept of the number of levels in your brightest stop as a way to prevent posterization and improve detail, but to take full advantage of those levels requires all pixels to be captured with none of the 3 colors being blown. It doesn't matter if you have a billion levels to work with in that stop if you have blown some of the data.  If that is indeed the most critical part of the scene, then you have to lower the exposure to prevent clipping of all 3 colors, perhaps at the expense of some of the shadow detail.

Logged

Tim Lookingbill

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 2436
Posterized Highlights-Where's my 4096 levels in 1st Stop ETTR shooting Raw?
« Reply #10 on: March 12, 2009, 06:10:44 pm »

Gabor,

The 4096 was just a number I used to communicate my point about how much useable (EDITABLE) data can be derived from a digital camera's sensor.

I believe I've read somewhere that the human eye can't differentiate more than 100-125 levels of density. Maybe this is why the Lab model only uses 100 levels to build density. Anything under that the eye starts to see posterization. The video system gives 255 different levels to render a smooth looking gradient.

My point hasn't been about getting a better looking rendering out of an image. That I can do on my own.

My point is I don't see all of these 1000's of levels mapped the way it has been claimed in explanations on this subject when it is used to render captured detail from a digital camera that claims 4000 levels when editing such an image. The detail when editing seems to be made up of far less than 100 levels per clump of detail. I'ld say it's maybe 25 levels at the most per clump of detail. And no I don't know what measurement a clump of detail I'm referring to.

I'm just saying going by the previews editing raw images I see too much posterization editing small clumps of detail where there shouldn't be any knowing that there's suppose to be this many levels to work with.

And I'm not talking about the highlights. I'm talking about the entire image.
« Last Edit: March 12, 2009, 06:11:28 pm by tlooknbill »
Logged

Panopeeper

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 1805
Posterized Highlights-Where's my 4096 levels in 1st Stop ETTR shooting Raw?
« Reply #11 on: March 12, 2009, 06:35:53 pm »

Quote from: tlooknbill
The detail when editing seems to be made up of far less than 100 levels per clump of detail. I'ld say it's maybe 25 levels at the most per clump of detail
...
And I'm not talking about the highlights. I'm talking about the entire image.
Tim,

I see some communications deficit. The problem may be, that you are addressing a subject from a certain point of view, which is from my point of view perhaps not a coherent issue but a collection of different subjects.

I see, perhaps mistakenly, a starting point in the segments taken out of your post in the quotation. You are looking for the details "in the clumps" - but your controls relate to the entire image. This is just what I mentioned above: I often need to separate the image into parts (most of my panos consist of sky, water surface and the rest, but the "rest" is sometimes broken up in several pieces).

When "commanding" over the entire image, I can not distibute the available levels (of perceivable tonal difference) to different areas of the image. If I want to "assign more levels" to the very bright rock, that takes away those levels in every part of the image, no matter if there is any detail with that brightness. Thus, if I increase the contrast on the bright rock, that increases the number of "assigned levels" in that brightness region. The number of remaining levels is now reduced.

Thus, if you increase the contrast here and there, then you don't have much for the rest. This is, what "tone mapping" is about. That concept is used usually in conjunction with high dynamic range, but that is totally unjust. The issue is more simple: how can I increase the contrast among the dark details AND among the bright details AND among the rest. This is, what is not working without either separating the areas in the image, or using some product, which makes the separation for you (tone mapping).

However, all the above is not the issue of raw development, at least not with ACR.
Logged
Gabor

Tim Lookingbill

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 2436
Posterized Highlights-Where's my 4096 levels in 1st Stop ETTR shooting Raw?
« Reply #12 on: March 12, 2009, 08:12:51 pm »

Gabor,

I get what you're saying. Tonemapping but not having enough different tones to work with without sacrificing one area for the other.

I guess I'll just come out and say what my suspicions have been from the outset and the main reason for my posting on this subject.

However it's explained and I'm not saying it hasn't been covered thoroughly, I don't see a great advantage from an editing standpoint to expose to the right when basing it on the logic that the brightest first stop is mapped to half of the total available high bit levels of the linear sensor data captured. I just don't see it in my edits. And from a noise in the shadows standpoint I don't see a huge problem not ETTR-ing either.

Not even editing in interpolated 16bit in ACR offers much advantage except for correcting for vignette and getting smooth transitions in deep blue skies. However, this is not to say a great looking image can't be attained whether or not one believes or practices ETTR. It's just a hassle calculating for it when shooting outdoors.

I'm just tired of using my DSLR like a "GameBoy" trying to override my camera's metering into underexposing or overexposing by constantly manually adjusting exposure shooting outdoors. No matter what metering mode be it pattern or center weighted, when I let my camera choose exposure for me I notice the Av, Tv number readouts on the top of my camera constantly changing wherever I point the lens. That means if I shoot manual that's what I have to do...constantly change the exposure.

Well I'm not going to change my exposure habits now because I usually fit the image within my camera's DR just going by the LCD's histogram and watching for flashing highlights in the preview. I'm just not going to be concerned anymore with pushing the exposure to get an ETTR advantage because I don't believe such an advantage exists from an editing standpoint. I just don't see it.

I'm not looking for an argument. I'm just conveying what I see.
« Last Edit: March 12, 2009, 08:14:56 pm by tlooknbill »
Logged

Guillermo Luijk

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 2005
    • http://www.guillermoluijk.com
Posterized Highlights-Where's my 4096 levels in 1st Stop ETTR shooting Raw?
« Reply #13 on: March 12, 2009, 08:46:08 pm »

The problem here is not with ETTR, but with the fact you took exposure a bit too far, and mainly because you didn't process your RAW file properly to get the most of it.
The RAW was plenty of levels in the last f-stop, and a good image can be obtained in the highlights area from it:

« Last Edit: March 12, 2009, 08:49:16 pm by GLuijk »
Logged

DarkPenguin

  • Guest
Posterized Highlights-Where's my 4096 levels in 1st Stop ETTR shooting Raw?
« Reply #14 on: March 12, 2009, 10:45:32 pm »

Quote from: GLuijk
Looking at image 2, and those gray blotches that look like posterization, I have a feeling there is no posterization issue here but just partial saturation. That gray colour denotes the areas in which ACR tried to rescue as much detail as possible when one or even two of the RAW channels were blown, and ACR performs this in a neutral (R=G=B ) colour.

It crams gray in there?  I thought it might do something more interesting.
Logged

Luis Argerich

  • Full Member
  • ***
  • Offline Offline
  • Posts: 155
  • Astrolandscaper
    • http://www.luisargerich.com/
Posterized Highlights-Where's my 4096 levels in 1st Stop ETTR shooting Raw?
« Reply #15 on: March 13, 2009, 12:13:27 am »

Maybe the problem is what you think about ETTR and what it really is?
The real benefit of ETTR is IMO to reduce noise, the correct exposure of the image is always more important than ETTR, I think you should first expose correctly and then to the right if you think your image can improve reducing noise. I don't think it is a good idea to just expose to the right as an exposure method but more as a noise-reduction technique that can be applied thru correction of a previously determined correct exposure.

Luigi

Guillermo Luijk

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 2005
    • http://www.guillermoluijk.com
Posterized Highlights-Where's my 4096 levels in 1st Stop ETTR shooting Raw?
« Reply #16 on: March 13, 2009, 06:39:08 am »

Quote from: DarkPenguin
It crams gray in there?  I thought it might do something more interesting.
Commercial RAW developers usually do neutral highlight recovery, and I think it's OK:
- First because the recovery only affects a very small part of the images: don't be mistaken thinking the large areas that seem to be 'recovered' when setting the exposure slider to the left to some negative value is highlight recovery, that is just preventing the white balance from blowing those large areas it blows at 0.0EV exposure. The information was intact in the RAW file.
- Second because colour recovery can be dangerous: if the colours interpolated are correct then it's fine, but if colours recovered are wrong the result is worse than assuming gray highlights. A non commercial software like DCRAW can take the risk to produce wrong colours, and many times it does.

Here DCRAW worked quite nicely:
Highlight recovery: ACR (left) vs DCRAW (right)
« Last Edit: March 13, 2009, 06:43:07 am by GLuijk »
Logged

bjanes

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 3387
Posterized Highlights-Where's my 4096 levels in 1st Stop ETTR shooting Raw?
« Reply #17 on: March 13, 2009, 11:16:27 am »

Quote from: tlooknbill
Below are two screenshots, the first is the full image with ACR 4.6 defaults with histogram to show I've correctly exposed this image. The second is a 200% zoomed in portion of the rock with the lightest highlights that I've attempted to add definition within the rock detail. You'll note the posterization isn't just in the highlights where it would be understandable for this to occur but also down into the upper mids.

Tim,

Another thing to consider, in addition to what Gabor and Guillermo have pointed out, is that ACR uses an exposure offset of -0.5 EV for your camera. See page 32 in the DNG Specification on the Adobe web site. This means that you have to use an exposure correction of -0.5 EV in ACR to get an accurate histogram of your image. You can determine this offset by converting the image to the DNG format with the DNG converter and then looking at the exif file. Here is what I got

[attachment=12112:PentaxBaselineExp.gif]

When you use the ACR defaults, you will see quite a bit of highlight clipping in the rocks in the upper right area of your image If you look carefully at the histogram you will see the clipping on the right in the histogram. I held down the alt button when using the exposure slider to show clipping. In the white areas of the preview, all channels are clipped. In the yellow areas, green and red are clipped.

[attachment=12113:PentaxSCRCapB.png]

If you use an exposure of -0.5 EV, the situation is improved, but there still is clipping as shown in Gabor's illustration and in the ACR preview shown below. Here, yellow indicates clipping in the green and red channels with an intact blue channel, and white indicates clipping in all channels.

[attachment=12114:PentaxSCRCapC.png]

In looking at your file, an exposure correction of about -0.65 was needed to eliminate all clipping. Try this and repeat your experiment. ACR highlight recovery works reasonably will when at least one raw channel is not clipped. Usually the green clips first (as in your file), since most sensors are most sensitive to green. Since all clipping could be removed with an exposure compensation of -0.65, this means that the blue channel was not clipped in the raw file--ACR can't bring a clipped channel below 255 (in 8 bit notation) regardless of how much highlight recovery is used. This is consistent with Gabor's raw histogram, which showed no clipping in the blue channel. The BaselineOffset of -0.5 was only approximate.

[attachment=12115:PentaxSCRCapD.png]

Bill




Logged

DarkPenguin

  • Guest
Posterized Highlights-Where's my 4096 levels in 1st Stop ETTR shooting Raw?
« Reply #18 on: March 13, 2009, 11:17:41 am »

Quote from: GLuijk
Commercial RAW developers usually do neutral highlight recovery, and I think it's OK:
- First because the recovery only affects a very small part of the images: don't be mistaken thinking the large areas that seem to be 'recovered' when setting the exposure slider to the left to some negative value is highlight recovery, that is just preventing the white balance from blowing those large areas it blows at 0.0EV exposure. The information was intact in the RAW file.
- Second because colour recovery can be dangerous: if the colours interpolated are correct then it's fine, but if colours recovered are wrong the result is worse than assuming gray highlights. A non commercial software like DCRAW can take the risk to produce wrong colours, and many times it does.

Seems like something begging for a check box or slider somewhere in ACR.  Would be cool to use with the brush.  Select a spot to use as a base color and have it try to recover using that.

As it is I think I'll load up DCRaw and keep it around for "problem images".
Logged

bjanes

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 3387
Posterized Highlights-Where's my 4096 levels in 1st Stop ETTR shooting Raw?
« Reply #19 on: March 13, 2009, 11:29:34 am »

Quote from: DarkPenguin
Seems like something begging for a check box or slider somewhere in ACR.  Would be cool to use with the brush.  Select a spot to use as a base color and have it try to recover using that.

As it is I think I'll load up DCRaw and keep it around for "problem images".

And if you do, be sure to check out Guillermo's DCraw tutorial.

Bill
Logged
Pages: [1] 2   Go Up