Pages: [1]   Go Down

Author Topic: Image Resampling and SNR  (Read 9760 times)

joofa

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 544
Image Resampling and SNR
« on: July 27, 2015, 11:11:31 pm »

A simulation of SNR in image resampling:

http://www.dpreview.com/forums/post/56216801

Joofa
Logged
Joofa
http://www.djjoofa.com
Download Photoshop and After Effects plugins

Bart_van_der_Wolf

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 8913
Re: Image Resampling and SNR
« Reply #1 on: July 28, 2015, 09:56:39 am »

A simulation of SNR in image resampling

Hi,

I'm not fluent in Matlab code, so I may not yet see exactly what you're doing, but correct me if I'm wrong.

1. You seem to basically assume that the image is noise free (or at least the noise is considered as pure signal).
2. You add (shot-)noise with a Poisson distribution, in line with the signal levels.
3. You add uniform (read-)noise with a Gaussian distribution.
The difference between the image (1) and the result of (1+2+3), or the sum of (2+3), is basically a signal level weighted total noise only image.

4. You upsample/downsample the separate signal image and the separate total noise image.
5. You calculate the dB of the S/N ratio

Questions/observations:
ad 1: The signal image already has a mix of shot and read noise, and demosaicing of all that.

ad 2: The poisson noise should be a function of the number of photons, so for an ISO 100 image white would probably be close to 65535 photons, but might be higher or lower depending on (pre and post) ADC conversion gain. It's the square root of those quantities that would dictate the actual Poisson noise standard deviation. Also, and I know it's difficult to create a signal only image, adding Poisson noise to the already 'noise infested' signal levels, would increase the standard deviation. So for higher precision you'd probably have to correct the assumed level of 'shot'noise by some factor. Maybe not by dividing it by sqrt(2), because the demosaiced image will not have the pure Raw read+shotnoise signature, but it might be closer than not correcting it at all.

ad 3: Similar considerations, adding Gaussian distributed noise to an already 'noise infested' Raw converted image will increase the standard deviation, and mixing different types of uncorrelated noise will change the distribution to something more Gaussian than Poissonian. Not that it would have a large impact in general, but dark images would be more affected by this than relatively bright images, so there is some image content bias to be expected.

ad 4: The problem with interpolating uncorrelated noise is that the fluctuations can be relatively large between neighboring pixels. Given that the Lanczos (a Sinc windowed Sinc) FIR filter exhibits over- and under-shoots (ringing) at the immediate neighbors, may overestimate the standard deviation. It will also do more  of that at higher image signal levels and less at lower image signal intensity levels, so average image brightness will introduce a bias.

ad 5: Photographers are more familiar with S/N ratios than with (electro/acoustical) engineering decibels. As for the latter one could debate whether to use a multiplier of 10 or 20 to the logarithm base 10 of the S/N ratio, I'm not sure. I prefer S/N ratios, also to avoid having to interpret logarithms.

While I understand that it may be useful to apply some sort of signal level weighting to the noise distribution spectrum, I've always found it very useful to up- and down-sample uniform (white) noise images (which incorporates all spatial frequencies). It's then much easier to make a logarithm of a Fourier transform magnitude image, and visualize the noise spectrum (as an image or periodogram and/or as a noise spectrum plot).

Alternatively, even if only to verify the validity of the above method you outlined above, one could use a sinusoidal grating of known period to simulate a single spatial frequency (or a zoneplate with many, in many isotropic orientations), and add noise to that and resample that in order to study a specific spatial frequency and the effect the resampling algorithm also has on the signal, not only on the noise.

Fun stuff, isn't it?

Anyway, folks like (me and) Jim Kasson also did some up/down-sampling analysis on noise in the specific context of different resampling algorithms, which is another can of worms.

Cheers,
Bart
Logged
== If you do what you did, you'll get what you got. ==

sandymc

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 350
Re: Image Resampling and SNR
« Reply #2 on: July 28, 2015, 11:03:04 am »

....and the effect the resampling algorithm also has on the signal, not only on the noise.

Yes indeed - there is classic trade-off of noise vs frequency domain performance here. A resampling algo is essentially a filter - a filter which impacts on both signal and noise.

Sandy
Logged

joofa

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 544
Re: Image Resampling and SNR
« Reply #3 on: July 28, 2015, 01:34:05 pm »

Hi,

I'm not fluent in Matlab code, so I may not yet see exactly what you're doing, but correct me if I'm wrong.

1. You seem to basically assume that the image is noise free (or at least the noise is considered as pure signal).
2. You add (shot-)noise with a Poisson distribution, in line with the signal levels.
3. You add uniform (read-)noise with a Gaussian distribution.
The difference between the image (1) and the result of (1+2+3), or the sum of (2+3), is basically a signal level weighted total noise only image.

4. You upsample/downsample the separate signal image and the separate total noise image.
5. You calculate the dB of the S/N ratio

Yes. The purpose of this experiment is to see the effect of the following on SNR during resampling (though I have presented result for only a single data point):

(1) Different resampling kernels.
(2) Different resampling ratios.
(3) Different images, i.e., signals.

In (1) and (2), an ideal filter mostly effects only downsampling, and not upsampling. Lanczos and others are not ideal, and can effect upsampling also. Only sinc is ideal. But sinc is not practical for other reasons. In (3), mostly people assume a flat, gray patch. But, I used an actual, natural image. An important expect of the experiment is to show the differences in downsampling and upsampling on SNR.

Questions/observations:
ad 1: The signal image already has a mix of shot and read noise, and demosaicing of all that.

Firstly, given a natural image, you can't separate the signal and noise. The presented experiment is one way of dealing with that in an approximate sense. However, on a closer examination it probably is not a big deal, as it is fine to take that image as a 'pure' signal that nature might have presented to an imaging system. For Poisson shot noise I don't see that as a problem. For Gaussian read noise it can expand the power spectrum a bit. However, in relation to shot noise its power is quite less for a good illuminated image.

ad 2: The poisson noise should be a function of the number of photons, so for an ISO 100 image white would probably be close to 65535 photons, but might be higher or lower depending on (pre and post) ADC conversion gain.

One can add stuff such as ISO. But, it does not matter for the sake of this experiment. We are just seeing the effect of resampling on signal + noise and ISO does not matter here.

ad 3: Similar considerations, adding Gaussian distributed noise to an already 'noise infested' Raw converted image will increase the standard deviation, and mixing different types of uncorrelated noise will change the distribution to something more Gaussian than Poissonian. Not that it would have a large impact in general, but dark images would be more affected by this than relatively bright images, so there is some image content bias to be expected.

Please see above.

ad 4: The problem with interpolating uncorrelated noise is that the fluctuations can be relatively large between neighboring pixels. Given that the Lanczos (a Sinc windowed Sinc) FIR filter exhibits over- and under-shoots (ringing) at the immediate neighbors, may overestimate the standard deviation. It will also do more  of that at higher image signal levels and less at lower image signal intensity levels, so average image brightness will introduce a bias.

Firstly, one can change the resampling algorithm from Lanczos to another one if so desired. Secondly, in natural images really sharp edges (a single pixel wide or so) would be a small percentage of the overall number of pixels, since these images exhibit a 'low pass' power spectrum. I would tend to believe that ringing does not play a big role in SNR calculation in natural images.

BTW, did you see any ringing on the image I resampled in the original link? I didn't particularly look for it, though.

ad 5: Photographers are more familiar with S/N ratios than with (electro/acoustical) engineering decibels. As for the latter one could debate whether to use a multiplier of 10 or 20 to the logarithm base 10 of the S/N ratio, I'm not sure. I prefer S/N ratios, also to avoid having to interpret logarithms.

Both 10 and 20 multiplier with log should give the same answer. People get confused sometimes as to which one to use since they don't realize if they are dealing with the absolute value of a quantity or the squared absolute value of that quantity. I.e, 20 log sqrt(.) = 10 log (.). I used 10 because I was computing power. There is no need to first take sqrt of power and then use 20, as it will give the same answer.

While I understand that it may be useful to apply some sort of signal level weighting to the noise distribution spectrum, I've always found it very useful to up- and down-sample uniform (white) noise images (which incorporates all spatial frequencies). It's then much easier to make a logarithm of a Fourier transform magnitude image, and visualize the noise spectrum (as an image or periodogram and/or as a noise spectrum plot).

It is the same approach that I have taken. You can take the Fourier / power spectrum of the noises if you wanted to.

Fun stuff, isn't it?

Absolutely!

Anyway, folks like (me and) Jim Kasson also did some up/down-sampling analysis on noise in the specific context of different resampling algorithms, which is another can of worms.

Seems good work. I need to go and read it.
« Last Edit: July 28, 2015, 01:46:17 pm by joofa »
Logged
Joofa
http://www.djjoofa.com
Download Photoshop and After Effects plugins

Bart_van_der_Wolf

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 8913
Re: Image Resampling and SNR
« Reply #4 on: July 29, 2015, 04:46:42 am »

ad 4: The problem with interpolating uncorrelated noise is that the fluctuations can be relatively large between neighboring pixels. Given that the Lanczos (a Sinc windowed Sinc) FIR filter exhibits over- and under-shoots (ringing) at the immediate neighbors, may overestimate the standard deviation. It will also do more  of that at higher image signal levels and less at lower image signal intensity levels, so average image brightness will introduce a bias.

Firstly, one can change the resampling algorithm from Lanczos to another one if so desired. Secondly, in natural images really sharp edges (a single pixel wide or so) would be a small percentage of the overall number of pixels, since these images exhibit a 'low pass' power spectrum. I would tend to believe that ringing does not play a big role in SNR calculation in natural images.

BTW, did you see any ringing on the image I resampled in the original link? I didn't particularly look for it, though.

Maybe there were not enough high spatial frequency details in this particular image for it to be clearly noticeable, but it is there. It's a characteristic of Lanczos windowed Sinc interpolation filters, that's why they produce sharp looking output. The first lobe has a strong negative contribution to the positive ones at its side. A Lanczos2 might be more benign than a Lanczos3, while higher orders will only increase the amplitude of the first lobe and thus affect noise more than most (lower) spatial frequencies in the image.

But the main issue is that it may be better to downsample the combined image + added noise, and then subtract the also downsampled image from that to leave the noise as a result. I have not tried it, so it may not make much of a difference on some images, but it does seem a better approach, also closer to what actually happens to a noisy image when we resample it. Of course one could also use a resampling filter that doesn't generate over- and under-shoots at sharp edges, but the result will be somewhat blurrier altogether, although without many additional artifacts due to the resampling itself. In my tests, a Quadratic filter (while soft) is almost as sharp as a Gaussian, but still (almost, depending on implementation) without risk of halo generation and a smaller support (and therefore it's faster).

Just some food for thought.

Cheers,
Bart
« Last Edit: July 29, 2015, 11:02:35 am by BartvanderWolf »
Logged
== If you do what you did, you'll get what you got. ==

joofa

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 544
Re: Image Resampling and SNR
« Reply #5 on: July 29, 2015, 12:14:28 pm »


Maybe there were not enough high spatial frequency details in this particular image for it to be clearly noticeable, but it is there. It's a characteristic of Lanczos windowed Sinc interpolation filters, that's why they produce sharp looking output. The first lobe has a strong negative contribution to the positive ones at its side. A Lanczos2 might be more benign than a Lanczos3, while higher orders will only increase the amplitude of the first lobe and thus affect noise more than most (lower) spatial frequencies in the image.

Firstly, thanks for the links to your earlier work on resmapling. There are a lot of pages and would take a while to read them.

Secondly, as I mentioned before, I would tend to believe that for natural images, in general, the percentage of pixels 'infested'  ;D with ringing is much less compared to the overall number of pixels in an image. So I don't think that it would effect SNR calculations a whole lot.

But the main issue is that it may be better to downsample the combined image + added noise, and then subtract the also downsampled image from that to leave the noise as a result. I have not tried it, so it may not make much of a difference on some images, but it does seem a better approach, also closer to what actually happens to a noisy image when we resample it.

We are using linear sampling. So it would appear to me that you if you downsample (image + noise) and later subtract downsampled image from it the result would be downsampled noise only. And, I produce that as part of the code in the original link.

Of course one could also use a resampling filter that doesn't generate over- and under-shoots at sharp edges, but the result will be somewhat blurrier altogether, although without many additional artifacts due to the resampling itself. In my tests, a Quadratic filter (while soft) is almost as sharp as a Gaussian, but still (almost, depending on implementation) without risk of halo generation and a smaller support (and therefore it's faster).

The usual Quadratic/Cubic filter approaches are actually interpolation techniques. When used directly for downsampling they work for small reduction ratios. For higher downsampling ratios it has been reported that aliasing creeps in. Which is understandable, as the filter is not adjusted in proportion to that ratio. Then people start using ad hoc approaches in downsampling such as using a Gaussian pre-blur before, say bicubic filter, etc. OTOH, when used properly, a Lanczos or another canonical downsampling filter does adjust its width in accordance. Though, there is still some chance of aliasing as they are not an ideal filter like sinc.

In any case, the purpose of this experiment is to measure the effect of SNR on image resampling and not to make an image pleasing or ringing free. Therefore, we should live with what we have and report the numbers for various algorithm as they behave.

Sincerely,

Joofa
« Last Edit: July 29, 2015, 12:26:06 pm by joofa »
Logged
Joofa
http://www.djjoofa.com
Download Photoshop and After Effects plugins
Pages: [1]   Go Up