Pages: [1]   Go Down

Author Topic: Photoshop 10 v 8 bit displays and effect of adding noise to smooth gradients  (Read 2369 times)

Doug Gray

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 2197

I have an Eizo CG318 with a 30 bit Quadro card in a x64 Win 10 machine and have noticed some banding, especially in the darker gradients. While significantly less than 24 bit mode, it should be virtually undetectable. I decided to more thoroughly test this and find out what's really going on and provide the test image for anyone here that's interested in testing it out themselves.

See the measured results from photographing the monitor in the attached chart at the bottom.

Using MATLAB, I created a 16 bit test image in ProPhoto RGB with L* changing linearly from 5 to 25. There are 6 horizontal gradients on the image in the following forms:

1. The reference, 16 bit image gradient with no noise added.
2. The reference gradient rounded to 8 bits.
3. The reference gradient with +/- 0.5 bits (.2%) of linearly added noise then rounded to 8 bits.
4. The reference gradient with +/- 0.25 bits (.1%) of linearly added noise then rounded to 8 bits.
5. The reference gradient with +/- 0.125 bits (.05%) of linearly added noise then rounded to 8 bits.
6. The reference gradient with +/- 0.0625 bits (.03%) of linearly added noise then rounded to 8 bits.

The added noise is similar to adding a very small amount of noise (.1 to .2 can't go lower than .1) in Photoshop's Filter dialog then converting to 8 bits.

The monitor was set at 200 nits to magnify the darker gradients then a photo was taken and processed in linear, 16 bit RGB space using dcraw. Each of the 6 gradients was sampled and vertical values across 50 pixels were averaged to reduce camera pixel noise. This is effective since the banding is vertical. Each gradient's L* was then plotted with the L* values along the vertical axis. Since all the gradients have nearly the same L*, Each was separated from the previous by adding increments of L*=2. The top line corresponds to the top gradient. The lines have a slight curvature to them du to background glare and the monitor's non zero, black point.

The monitor's gamma was set at 2.2, ProPhoto's gamma is 1.8.

Conclusions:

1. The 30 bit path is not being fully used by Photoshop. Rather, the desired display values are being computed in 8 bit steps of ProPhoto, and transmitted to the monitor using the 32 bit path. The gradient steps in 0.6 dE 1976 levels. Even though small, these are very visible when looking at low key images. This should not occur if Photoshop was fully using high precision and a 30 bit path.

2. This can be seen by comparing the relatively even stepping on the first gradient to the "chunky" stepping on the second gradient. The "chunky" gradient seen on the second gradient is due to the conversion of ProPhoto to 8 bits which creates a regular "gap" where you see the steps make a 2x larger jump upwards every fifth step. This occurs when the 8 bit ProPhoto values are converted to the monitor's 2.2 gamma RGB levels. This is intrinsic and not a failure of the 30 bit path. It's intrinsic to using 8 bit discreet values and having a gamma change.

3. The effects of even a small amount of dither noise is astonishing. The third and fourth gradients (+/-.5 bits) and (+/- .25 bits) of added noise effectively eliminates the visual (and measured) stepping. The tiny amount of noise (+/-.0625 bits) added to the last gradient smooths out the transitions but leaves the bulk of the stepping visible.

See attached images. The first is the actual tiff image of the gradients and is suitable for testing your own monitors. The second is a heavily reduced to fit image of the photo I took to gather these curves.

If any of you test your monitors with this I'd be interested in what you see and in particular, whether this is different on iOS or other configurations.

« Last Edit: May 14, 2018, 06:22:57 pm by Doug Gray »
Logged

Peano

  • Full Member
  • ***
  • Offline Offline
  • Posts: 155
    • Radiant Pics

I have an Eizo CG318 with a 30 bit Quadro card in a x64 Win 10 machine and have noticed some banding, especially in the darker gradients. While significantly less than 24 bit mode, it should be virtually undetectable.
When I do retouching -- and I do a lot of it, very capably, if I may say so -- any problem that is "virtually undetectable" isn't a problem worth worrying about. In fact it isn't a problem at all.

Quote
I decided to more thoroughly test this and find out what's really going on and provide the test image for anyone here that's interested in testing it out themselves.
I see two big blank frames and then a picture of a graph with some squiggly lines on it. But what I still don't get is, What is the point of carrying out a bunch of tests to determine "what is really going on" with something that is virtually undetectable? If your field is theoretical physics, I can see the point. But for people who work with photographs -- a visual medium -- what is the point?

Doug Gray

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 2197

When I do retouching -- and I do a lot of it, very capably, if I may say so -- any problem that is "virtually undetectable" isn't a problem worth worrying about. In fact it isn't a problem at all.
I see two big blank frames and then a picture of a graph with some squiggly lines on it.

The "squiggly lines" are graphs of the L* (luminance in LAB) from the left edge of the 6 gradients to the right edge as displayed on my monitor. They quantify the effect and allow analysis as opposed to just eyeballing the banding.

Quote
But what I still don't get is, What is the point of carrying out a bunch of tests to determine "what is really going on" with something that is virtually undetectable? If your field is theoretical physics, I can see the point. But for people who work with photographs -- a visual medium -- what is the point?

Because it is very detectable and, to me, annoying as hell when working with certain low key images and trying to get a good soft proof so you don't waste paper. If you think the banding on the top gradient is not bothersome cool.

That's why I shelled out the money for a 30 bit display and card. Up until a couple years ago it didn't work, then Adobe fixed it. Worked great for a while. Now it appears broken again.

However, to your point this is a problem with synthetic images or ones that have been strongly background blurred for artistic purposes. Unmodified photos have enough noise already in the image (it's really hard to get pixel to pixel noise below 1 bit from a camera) to prevent the problem outside of heavy use of things like Gaussian blur for smooshing backgrounds.
« Last Edit: May 14, 2018, 11:16:40 pm by Doug Gray »
Logged

Jack Hogan

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 798
    • Hikes -more than strolls- with my dog

Hi Doug, I am a fan of your tests :)

A couple of questions:  Is the monitor connected to the video card with a cable that is 10-bit/ch capable?  (How) Did you apply gamma to your gradients?  Are you viewing the images in ProPhotoRGB?  What gamma did you profile your monitor to?  Does your display perform any dithering?

And a comment:  It turns out that humans by and large do not see banding in a gradient if adjacent steps are less than about 1% apart in linear Luminance.  This is called the Weber-Fechner Fraction and holds true not just for vision but for many other phenomena where humans are involved.  I seem to see many contiguous steps in your gradient plot that are much, much more than 1% apart linearly, hence those should be visible.  Also there is some quantization of luminance to be expected by the display itself in the darkest tones, if it is not dithered.  For instance in theory going from linear step 10 to 11 there is a 10% luminance difference.

As far as added random noise is concerned, the amount that is generally recommended (and that is typically present at base ISO in our raw files) has a standard deviation equal to the step.  We call it read noise and it is only relevant in the darkest tones since above that shot noise takes over.

So why not prepare in Matlab a 16-bit Luminance gradient going up 1 DN at a time every 10 pixels or so, then applying 1/x gamma to the resulting image (where x is the gamma that your monitor has been profiled to), with and without noise of standard deviation of 1 DN.  As long as your video path and cable are capable of 10 bits you should be good to go.  Without noise you should not be able to see banding above about 100 DN before gamma was applied (some 8-bit examples in the link above).  With the noise typically present in our captures even at base ISO you should not see it at all, especially once you bring maximum brightness to typical working conditions (e.g. 80-160 cd/m^2): the WB Fraction grows substantially below about 1 cd/m^2, as shown in the table in the link.

Let us know.

Jack
« Last Edit: May 15, 2018, 03:50:27 am by Jack Hogan »
Logged

samueljohnchia

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 498

Hi Doug,

Thanks for sharing this. Your experiments are always fascinating. Have you noticed soft proofing oddities with 40 bit display output? I have. Gradients are not smooth and show strange reversals in the shadow end when my Firepro card is in 30 bit mode but its fine when I put it back in 24 bit mode. The same gradient without soft proofing activated looks very smooth. I have yet to notice the banding you did.
Logged

Doug Gray

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 2197

Hi Doug,

Thanks for sharing this. Your experiments are always fascinating. Have you noticed soft proofing oddities with 40 bit display output? I have. Gradients are not smooth and show strange reversals in the shadow end when my Firepro card is in 30 bit mode but its fine when I put it back in 24 bit mode. The same gradient without soft proofing activated looks very smooth. I have yet to notice the banding you did.
Yes, I've noticed this too, and it was when the 30 bit mode was "working."  It's been "working" on and off in Photoshop for years. It seems when they fix it, it gets broken later. And, even when it's "working" it is only doing so w/o soft proofing. Soft proofing puts it back in 24 bit mode instead of 30 bit mode. It seems to be poor coordination between nvidia and Adobe. The last time it broke, the Win10 fall update, it could be "fixed" by installing the NVidia drivers dates spring of 2017. The last update broke it again and re-installing the recommended driver no longer fixes it.

Aside from this, there are differences in soft proofing much worse than the 8 bit/10 bit issue. Changing the color settings to enable desaturation but setting the % at 0 should have no effect but it does. On my dark gradient it introduces a weird hue with the left and right portions getting a slight purplish color. It goes away when unchecking the desaturate check box.
« Last Edit: May 15, 2018, 10:38:07 am by Doug Gray »
Logged

Doug Gray

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 2197

Hi Doug, I am a fan of your tests :)

A couple of questions:  Is the monitor connected to the video card with a cable that is 10-bit/ch capable?  (How) Did you apply gamma to your gradients?  Are you viewing the images in ProPhotoRGB?  What gamma did you profile your monitor to?  Does your display perform any dithering?
Images posted are in ProPhoto which has a gamma=1.8. The first one's top gradient was made in 16 bits using Matlab and linearly changes L* from 5 to 25 left to right. The next ones are rounded to 8 bits with increasing amounts of noise. The monitor, a CG318, is a 30 bit monitor driven by a quadro and recommended cable.
Quote

And a comment:  It turns out that humans by and large do not see banding in a gradient if adjacent steps are less than about 1% apart in linear Luminance.  This is called the Weber-Fechner Fraction and holds true not just for vision but for many other phenomena where humans are involved.  I seem to see many contiguous steps in your gradient plot that are much, much more than 1% apart linearly, hence those should be visible.  Also there is some quantization of luminance to be expected by the display itself in the darkest tones, if it is not dithered.  For instance in theory going from linear step 10 to 11 there is a 10% luminance difference.
Yes, though the 1% linear luminance change perception does require enough luminance. Since the test I ran was with the monitor set to 200 cd/m^2, (200 nits), the light level at L*=5 is just under 1 nit. L* has a linear ramp up to about L*=11 so a change from say 10 to 11 is a change of 10% and is approx. 1.8 nits to 2.0 nits. Very visible on low key images where eyes adapt to the lower brightness. At those levels I can't see step changes of .3 dE 76 but changes of .6 dE are visible and annoying.

To your point, step changes of L*=1 are very apparent but only when adjacent. With even a small separation of steps (say a black border in between), luminance change is much harder to see.
Quote


As far as added random noise is concerned, the amount that is generally recommended (and that is typically present at base ISO in our raw files) has a standard deviation equal to the step.  We call it read noise and it is only relevant in the darkest tones since above that shot noise takes over.
Right. In fact the +/- .5 bit dither of the third gradient corresponds roughly to this. What I did was add linearly distributed noise from -.5 to +.5 of a step. When they are rounded it produces a distribution such that, for say an 8 bit value of 4.25 will be rounded to 5 25% of the time and rounded to 4 75% of the time. So the averages should be quite good. The successive gradients have the noise further reduced by factors of 2. Base photo image noise, below shot noise, is often corrupted and creates patterns that effectively reduce camera's the dynamic range as it can be quite objectionable.
Quote


So why not prepare in Matlab a 16-bit Luminance gradient going up 1 DN at a time every 10 pixels or so, then applying 1/x gamma to the resulting image (where x is the gamma that your monitor has been profiled to), with and without noise of standard deviation of 1 DN.  As long as your video path and cable are capable of 10 bits you should be good to go.  Without noise you should not be able to see banding above about 100 DN before gamma was applied (some 8-bit examples in the link above).  With the noise typically present in our captures even at base ISO you should not see it at all, especially once you bring maximum brightness to typical working conditions (e.g. 80-160 cd/m^2): the WB Fraction grows substantially below about 1 cd/m^2, as shown in the table in the link.

Let us know.

Jack
I've since made a few display profiles using various gammas. I now have strong evidence the 30 bit path no longer functions on my system. Adobe/Nvidia/Microsoft? broke it again.

The following approach is a good way to check if a system is functional in 30 bit or 24 bit RGB if your monitor profiling software allows setting gamma=1 such as ColorNavigator.

1. Make a profile setting the gamma=1 and the luminance to maximum.
2. In Photoshop, create a neutral gradient from black to L*=15 using a 16 bit RGB working space.
3. If the display shows step changes approx. every 3.5 L* (about five of them) then your system is opverating in 24 bits, 8 bits per channel. These step changes are very visible!
4. If the display shows more than 10 steps and it's not easy to see the steps, your monitor is operating in 30 bit mode or 10 bits per channel.

« Last Edit: May 15, 2018, 06:33:54 pm by Doug Gray »
Logged

Jack Hogan

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 798
    • Hikes -more than strolls- with my dog

Images posted are in ProPhoto which has a gamma=1.8. The first one's top gradient was made in 16 bits using Matlab and linearly changes L* from 5 to 25 left to right. The next ones are rounded to 8 bits with increasing amounts of noise. The monitor, a CG318, is a 30 bit monitor driven by a quadro and recommended cable.

Good.  Ideally for this test you would then have your monitor profiled to a gamma of 1.8 (or, assuming your monitor is profiled with a gamma of 2.2, convert the files to Adobe RGB which has a gamma of 2.2)

Yes, though the 1% linear luminance change perception does require enough luminance. Since the test I ran was with the monitor set to 200 cd/m^2, (200 nits), the light level at L*=5 is just under 1 nit. L* has a linear ramp up to about L*=11 so a change from say 10 to 11 is a change of 10% and is approx. 1.8 nits to 2.0 nits. Very visible on low key images where eyes adapt to the lower brightness. At those levels I can't see step changes of .3 dE 76 but changes of .6 dE are visible and annoying.

Right, linear to L*8.  I suggest setting up the gradients linearly in DN in Matlab because if one uses L* as a base ALL integer steps from L*1 to L*100 have a WB-Fraction above 1%.  For instance the lowest photometric Luminance change in the sequence is about 2.6% between L*99 and L*100.  That should be visible according to the table in the earlier link, so absent dithering all lower steps will most likely be too.

To your point, step changes of L*=1 are very apparent but only when adjacent. With even a small separation of steps (say a black border in between), luminance change is much harder to see.Right. In fact the +/- .5 bit dither of the third gradient corresponds to this. The successive gradients have the noise further reduced by factors of 2. Base photo image noise, below shot noise, is often corrupted and creates patterns that effectively reduce camera's the dynamic range as it can be quite objectionable.I've since made a few display profiles using various gammas. I now have strong evidence the 30 bit path no longer functions on my system. Adobe/Nvidia/Microsoft? broke it again.

Sorry to hear that.
Jack
Logged

Doug Gray

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 2197

Right, linear to L*8.  I suggest setting up the gradients linearly in DN in Matlab because if one uses L* as a base ALL integer steps from L*1 to L*100 have a WB-Fraction above 1%.  For instance the lowest photometric Luminance change in the sequence is about 2.6% between L*99 and L*100.  That should be visible according to the table in the earlier link, so absent dithering all lower steps will most likely be too.
L* changes linearly in the top gradient of my attached image through L*5 to 25 so each step is about .008 L* over the 3000 pixel width.

And this illustrates the difference between visible banding, which occurs with a far smaller dE than 1 but with separated patches. dE 2000 is even further away.  At 100 nits, a dE00 around L* of 1 near max is almost a 5% change in photometric luminance. Very visible banding from adjacent steps but is just noticeable for two slightly separated patches in a L*=50 gray surround.
« Last Edit: May 15, 2018, 06:39:05 pm by Doug Gray »
Logged

Doug Gray

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 2197

Update!

Updated Photoshop last month to 20.0.1 but hadn't had time to do a complete check. However, the subtle banding described above was still there.

I finally updated my Quadro K2000D drivers which have been out a few months. Yes! I now have a good working 30 bit display!

Not only that but the bugs in the deep shadows when soft proofing are gone!  I now get exactly the same results soft proofing as converting to printer RGB then back to working RGB.

I wonder if the second problem was Photoshop or buggy GPU drivers in the Quadro card. The latter could account for both  effects.

In any case I'm quite relieved to see both of these bugs vanish.
Logged

Jack Hogan

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 798
    • Hikes -more than strolls- with my dog

Excellent!
Logged
Pages: [1]   Go Up