Sony a7 and a7R Raw Compression

[Jim Kasson]

The Sony raw compression controversy seems to be heating up again, prompting me to do something that I've been resisting for months: simulating it.

There are two pieces to the Sony compression algorithm. The tone curve leaves out progressively more possible values as the pixel gets brighter. This kind of mimics the 1/3 power law that defines human luminance response. Are there a sufficient quantity of buckets at all parts of the tone curve, and after an image is manipulated in editing, do artifacts that were formerly invisible become intrusive?

The other possible place where visible errors could be introduced is in the delta modulation/demodulation. If the maximum and minimum values in a 16-bit row chunk are further apart than 128, information will be lost. Is that a source of visible errors?

And the last question: even if the above errors could be visible with synthetic images, are they swamped out by photon noise in real camera images?

Drawing on work by Lloyd Chambers and LuLa'er Alex Tutubalin, I wrote Matlab code to look at the effects of the Sony compression/decompression algorithm on real or synthetic images. The input image is sampled onto a simulated RGGB Bayer array, compressed, decompressed, demosaiced with bilinear interpolation, and compared with an image that's simply sampled and demosaiced. I just got it working this morning. So far, I've run one synthetic image (the ISO 12233 target) through the code.

Here's a link to the result.

For those of you who just want me to cut to the chase, the differences between the two images is very small.

I invite anyone to critique the algorithm I implemented as described in the page linked to above.

I invite anyone who'd like to see what happens when one of their images is run through the simulator to PM me, and I'll do what I can to make it happen.

Jim

[Vladimirovich]

And the last question: even if the above errors could be visible with synthetic images, are they swamped out by photon noise in real camera images?

you apply the curve to reduce gradations to the data that is already signal + noise... so how the noise is relevant at all ?

[Jim Kasson]

you apply the curve to reduce gradations to the data that is already signal + noise... so how the noise is relevant at all ?

Visually, it might be. Noise may provide a visually confusing stimulus that covers up some artifacts, the same way that noise added to an 8-bit-per-color-plane image or a indexed-color image can reduce visible contouring.  I'm not a proponent of this theory; I really don't have a position on it. But I've seen it proposed by others, and I don't think it's unreasonable.

Jim

[Vladimirovich]

Visually, it might be. Noise may provide a visually confusing stimulus that covers up some artifacts

but you reduce gradations (and severely) __after__ noise did what it does (dither)... so unless your raw converter/PP software dither during operations you can consider that there is no noise in data at all... so only the amount of gradations and what you do matters... assume that you have 16 bit raw where 15 bit is noise and 1 bit of data... nicely dithered image... now if you reduce gradations to only 2 then it really does not matter whether there were 15 bits of noise or no noise at all...

[ErikKaffehr]

Hi,

Jim, thanks for your effort!

I am pretty sure that photon statistics dominate over compression artefacts. The compression scheme seems reasonable to me.

BTW, I would be much more concerned about the moiré.

Best regards
Erik

but you reduce gradations (and severely) __after__ noise did what it does (dither)... so unless your raw converter/PP software dither during operations you can consider that there is no noise in data at all... so only the amount of gradations and what you do matters... assume that you have 16 bit raw where 15 bit is noise and 1 bit of data... nicely dithered image... now if you reduce gradations to only 2 then it really does not matter whether there were 15 bits of noise or no noise at all...

[jrsforums]

Is all RAW compression, such as cr2, lossy?

[Vladimirovich]

I am pretty sure that photon statistics dominate over compression artefacts.

and I am sure that just enough gradations helps... noise is irrelevant... reduce number of gradations and no matter how much noise is in your data you will see banding

[ErikKaffehr]

How?

Can you show an example? Of any case where there is a significant number of discrete levels within the say 2 sigma of the noise?


Best regards
Erik

and I am sure that just enough gradations helps... noise is irrelevant... reduce number of gradations and no matter how much noise is in your data you will see banding

[dds]

Would these artifacts be relevant?

http://diglloyd.com/blog/2014/20140214_1-SonyA7-artifacts-star-trails.html

[Vladimirovich]

How?

Can you show an example?

I think you mix 2 different things - one is how much bits is enough to encode a signal sans photon/shot noise (let it be X bits) vs how many gradations (let it Y, where log2(Y) < X ) after you decode back (w/o dithering the data after decoding) is enough not to have posterization if you start heavy image processing...  how big that difference between log2(Y) and X is tolerable for a heavy postprocessing do you think ? and if you do not see the banding it is not because there was a lot of noise before the encoding - but because there are either enough gradations present after decoding or you don't push in processing for "99%" of shots... reduce the number of gradations sufficiently and all the shot noise in the world is not going to be enough not to have banding (unless you dither in software after decoding)... now certainly Sony engineers were not that stupid not to leave just enough gradations to make this a non issue for "99.99%" of users

[ErikKaffehr]

Hi,

I guess that could come from the delta compression. It would be sensitive to steep gradients.

Best regards
Erik

Would these artifacts be relevant?

http://diglloyd.com/blog/2014/20140214_1-SonyA7-artifacts-star-trails.html

[Jim Kasson]

Simulated raw gradient quantized by a simulated 4 bit ADC, then demosaiced with bilinear interpolation:



Simulated raw gradient with 20% noise added quantized by a simulated 4 bit ADC, then demosaiced with bilinear interpolation:



Jim

[Jim Kasson]

I was surprised how small the errors were on my first test. I spent some time looking for a bug. Then it hit me. I computed the difference image by subtracting the two images using the Photoshop "difference" blending mode. I believe that works in the working color space, which was Adobe RGB. Because of the gamma of 2.2 in that space, taking the difference the way I did will attenuate highlight deltas compared to shadow differences, when compared to computing the difference in a linear RGB space.

Jim

[Jim Kasson]

BTW, I would be much more concerned about the moiré.

Erik, remember that this is a simulation, and I simulated a perfect lens. No aberrations, no diffraction, no field curvature, no mis-focusing, etc. A real lens would blur the moire somewhat. I did simulate capture averaging with a 100% fill factor. If you simulate point capture, the false colors are something else. Also, bilinear interpolation is far from the best demosaicing method. It has the advantage that it is readily available, non-proprietary, and reproducible by others.

Jim

[madmanchan]

I was surprised how small the errors were on my first test. I spent some time looking for a bug. Then it hit me. I computed the difference image by subtracting the two images using the Photoshop "difference" blending mode. I believe that works in the working color space, which was Adobe RGB. Because of the gamma of 2.2 in that space, taking the difference the way I did will attenuate highlight deltas compared to shadow differences, when compared to computing the difference in a linear RGB space.

Right, Jim.  If you're looking for the actual numerical deltas/errors, then using the linear space for the diffs makes sense.  But what you did the first time (in a 2.2 encoding) makes more sense from a visual evaluation.

[Jim Kasson]

Right, Jim.  If you're looking for the actual numerical deltas/errors, then using the linear space for the diffs makes sense.  But what you did the first time (in a 2.2 encoding) makes more sense from a visual evaluation.

Eric, I totally agree. From a visual perspective, it would have been better to use a gamma of three, which would have compressed the highlights even more. OTOH, I wanted to make sure that what I did was clear, and to relate it to comparisons that are out there using linear spaces.


Thanks,

Jim

[Vladimirovich]

Simulated raw gradient with 20% noise added quantized by a simulated 4 bit ADC, then demosaiced with bilinear interpolation:

so that will be 2 bits signal and 2bits noise in 4bit raw before non linear encoding of post ADC data, right ?

[Jim Kasson]

so that will be 2 bits signal and 2bits noise in 4bit raw before non linear encoding of post ADC data, right ?

I added the noise in Photoshop in Adobe RGB, using the add noise filter. I think, but I don't know, that when the filter is set to 20%, Photoshop adds noise in the working color space with either the peak, or the peak-to-peak, or the rms value equal to one-fifth of the full-scale signal or one fifth of the current value. If it's important to figure that out, I can do some research, or just add the noise in Matlab, where I can know exactly what's going on. The gamma-encoded file was linearized before the simulated sampling.

You're right about that all being before the nonlinear encoding of the post-ADC data.

So, in the gamma 2.2 space, I'd say 4 bits signal and 1.678 bits (log2(16*0.2)) noise.  But there's a lot of guessing in that calculation.

I was just trying to get a feel for whether noise pre-quantization could reduce contouring. It looks like it can.

Jim

[madmanchan]

I was just trying to get a feel for whether noise pre-quantization could reduce contouring. It looks like it can.

Absolutely -- that's exactly what dither is (randomizing the quantization error).

[Jim Kasson]

Absolutely -- that's exactly what dither is (randomizing the quantization error).

Eric, I tried to send you a PM, but it looks like it didn't work. Anyway, to your point, we're in violent agreement.

http://patents.com/us-3999129.html

https://www.google.com/patents
/US4187466

Thanks,

Jim