Posterization in Photoshop

[pcox]

Hi folks -
I've just started experiencing a problem in Photoshop. At first, I thought it was due to the new OpenGL features in CS4, but it's a problem there even with that acceleration turned off. It also happens in CS3.

Here's a screenshot of what I'm talking about:


As you can see, the sky is severely posterized. This is after a couple of enthusiastic Curves applications, darkening it down quite a bit. However, the problem is only visible at magnifications of up to 66% (on this particular file). Once I zoom in further, it vanishes, and it doesn't show up in a print. Its appearance does seem tied to the size of the file. I only started seeing it when processing images out of my 5D Mk II.

Anyone got any idea what's causing this and how I can correct for it? It's driving me batty.

Cheers,
Peter

[jjj]

I'd guess it's more an effect of your monitor not drawing those fine gradations too well, which are less fine once you start to zoom in.

[Panopeeper]

However, the problem is only visible at magnifications of up to 66% (on this particular file). Once I zoom in further, it vanishes

Make following test: downsize the image to a size, which does show the posterization (50%?), and then look at that in 100%. I guess the posterization vanishes.

If so, then there is no posterization at all. The downsizing algorythm for *displaying* the image is not as sophisticated as the algorythm of "real downsizing"; that is a performance (speed) issue.

[pcox]

Make following test: downsize the image to a size, which does show the posterization (50%?), and then look at that in 100%. I guess the posterization vanishes.

If so, then there is no posterization at all. The downsizing algorythm for *displaying* the image is not as sophisticated as the algorythm of "real downsizing"; that is a performance (speed) issue.

Folks -
Clearly this is a performance issue, similar to Photoshop not doing a full histogram calculation every time you make a change. The fact that the posterization disappears when zooming in, or when printing proves this. What I'm asking is if there's a way to stop it from happening in the first place, as it's very distracting.

Peter

[Eyeball]

Peter, no guarantees but here are some things I would check/try:

• Check what you have set for cache levels in Edit>Preferences>Performance.  The default is 4. The max is 8.  A lower cache setting MIGHT improve display quality but I have found that extreme settings of cache levels (1 or 8, for example) sometimes cause strange behavior.
• If you are calibrating/profiling your monitor, review your process and settings - particularly if you are using an LCD display.  If your calibrator is making large shifts to achieve proper calibration or you yourself are making significant adjustments to the on-monitor controls to achieve calibration, those things can worsen display posterization.  Using native white point and native gamma may help, obviously sacrificing your desired gamma and white point.
• If you are currently using CS4 with OpenGL turned on, try checking "Advanced Drawing" and "For Image Display" in the Advanced OpenGL settings if they are not already checked.

[KeithR]

Nothing to really add to the discussion, but the image is very nice. Reminds me of an 18th or 19th century painting.

[pcox]

Eyeball -
Thanks for your suggestions. The cache level is set to the default of 4, display calibration is unremarkable - no large shifts are needed to achieve accuracy. I have tried setting 'Advanced Drawing' in the OpenGL prefs, but it has no effect. OpenGL isn't a factor here anyway as the issue occurs even when it's turned off, and also occurs in CS3.

For the record, the machine is a Mac Pro, 2x2.8GHz with 10GB of RAM. The monitor is a NEC 2690WUXi.

The behaviour is the same on my uncalibrated Macbook.

Cheers,
Peter

Peter, no guarantees but here are some things I would check/try:

• Check what you have set for cache levels in Edit>Preferences>Performance.  The default is 4. The max is 8.  A lower cache setting MIGHT improve display quality but I have found that extreme settings of cache levels (1 or 8, for example) sometimes cause strange behavior.
• If you are calibrating/profiling your monitor, review your process and settings - particularly if you are using an LCD display.  If your calibrator is making large shifts to achieve proper calibration or you yourself are making significant adjustments to the on-monitor controls to achieve calibration, those things can worsen display posterization.  Using native white point and native gamma may help, obviously sacrificing your desired gamma and white point.
• If you are currently using CS4 with OpenGL turned on, try checking "Advanced Drawing" and "For Image Display" in the Advanced OpenGL settings if they are not already checked.

[pcox]

Thanks, Keith -
It's not quite finished yet, as I've been sidetracked by trying to fix this problem =)

Cheers,
Peter

Nothing to really add to the discussion, but the image is very nice. Reminds me of an 18th or 19th century painting.

[Eyeball]

Did you try reducing the cache levels to see if it made a difference?  It may require a re-start of PS to apply the change.  I would suggest starting at 1 and see if that makes a difference.  If not, well, forget it and go back to 4.  If it does help, then try moving it back towards 4 until you find the highest cache level that doesn't present the banding.

Also, another thing comes to mind to check if you haven't already.  Make sure your video card settings are using the highest color quality setting - 32bit if you have it.

[pcox]

Eyeball -
Just tried that. At 1 Photoshop crashes when opening the file. At 2, there's no change in the problem. I also tried it at 8, still no change.

And yes, the video card is set to the highest colour depth.

I'm going to call Adobe support in the morning and see what they make of it. This doesn't seem to be a common problem.

Cheers,
Peter

Did you try reducing the cache levels to see if it made a difference?  It may require a re-start of PS to apply the change.  I would suggest starting at 1 and see if that makes a difference.  If not, well, forget it and go back to 4.  If it does help, then try moving it back towards 4 until you find the highest cache level that doesn't present the banding.

Also, another thing comes to mind to check if you haven't already.  Make sure your video card settings are using the highest color quality setting - 32bit if you have it.

[Guillermo Luijk]

What gamma are you using in your colour profile? I experience this problem you describe, but only with linear images (i.e. those to which gamma compensation has not been applied yet). I guess the quick display update in PS when using linear images equals to a linear 8-bit display and that guarantees posterization.

This is reported in the DCRAW tutorial since until now DCRAW only provided linear 16-bit output: (left linear image at 36% zoom, right 2,2 gamma image)



I have never experienced all that in non-linear images though.

BR

[pcox]

GL:
Thanks for your help. I'm using 2.2 for my gamma, so this oughtn't to be an issue. It's interesting to see the problem in a linear image, though.

Cheers,
Peter

What gamma are you using in your colour profile? I experience this problem you describe, but only with linear images (i.e. those to which gamma compensation has not been applied yet). I guess the quick display update in PS when using linear images equals to a linear 8-bit display and that guarantees posterization.

This is reported in the DCRAW tutorial since until now DCRAW only provided linear 16-bit output: (left linear image at 36% zoom, right 2,2 gamma image)



I have never experienced all that in non-linear images though.

BR