Demosaic - Quick comparison of three raw converters and four algorithms

[ErikKaffehr]

Hi,

I was looking at one of my shots with my new A7rII. This camera has a decent resolution (4.5 micron pitch) without OLP filtering. Personally I expected to see quite some aliasing artefatcts like Moiré, but it has actually not been that bad. I have seen some.

This shot doesn't show moiré but it has stair case artefacts and it is interesting to see how different raw processors handle it.

The attached sample shows four conversions:

Raw Therapee using LMMSE	Capture One
Lightroom 6.2	Raw Therapee using Amaze

In all raw processors I disabled noise reduction and sharpening and all the images were sharpened using FocusMagic with its default settings. FocusMagis is a good sharpening tool that does not introduce a lot of artefacts of its own. So all images should have a decent level of sharpening that is comparable by and large.

What I see is Capture One has produced a very clean image and so did Raw Therapee with Amaze which is the default demosaicer. Lightroom produced a lot of artefacts. Raw Therapee with LMMSE did produce similar artefacts as Lightroom.

What are my conclusions? Well, it seems that a good demosaicer may be able to avoid stair case aliasing. Lightroom is very clearly not doing that and I think Adobe needs to realise that we are getting a lot of cameras without OLP filtering and need to improve their algorithms.

This image was shot with a Canon 24/3.5 TSE LII at f/11 so it is affected by diffraction, but that amount of diffraction is not enough to suppress aliasing, it seems.

Best regards
Erik

[Mark D Segal]

Hi Erik,

To see any differences at all I had to pump your image up very heavily on my (high-res) display. So that immediately says something, consistent with my general observation that using LR 6.2 with its own sharpening tools on 13*19 prints of even crops of photos I make with my 24MP Sony a6000 I don't see the kind of artifacting you show here.

That said, with your comparison image really pumped up, to my eyes, I thought LMMSE was the the worst result, Lightroom second worst, Amaze second best and C1 the best, but these latter two are very close. I also thought the C1 result overall less sharp and of slightly flatter contrast than the three others. I have no idea whether that is on account of C1 itself or how you deployed the various tools on each of the raw files. It could be interesting to repeat this test using each application's in-built sharpening.

I also notice that opened in Photoshop you are using 8 bit depth rather than 16. I wonder whether this is influencing any of the outcomes?

I only use LR, and I would be interested to see what results I can produce with the same photo in the manner I process photos through LR. If you would be willing to send me your original raw photo with the sole purpose of trying this and reporting back we can arrange off-line.

Cheers,

Mark

[ErikKaffehr]

Hi Mark,

The raw image is here: http://echophoto.dnsalias.net/ekr/Articles/A7rIIJourney/Demosaic/20151003-_DSC2833.ARW

Some observations:

To begin with I prepared the images an early morning and I had some obscure reason of saving bandwidth that I processed with 8 bits. But it doesn't really matter in this case in my humble opinion.

The other point it is that I have been shooting with a P45+ for a while and not been entirely happy with LR-s conversions. The P45+ has pretty large pixels, 6.8 micron pitch, and I always felt that LR did not handle fine detail really well. With the A7rII I was expecting some artefacts as they have dropped the OLP filter but the pixels are still pretty large. Indeed, I had some well deserved observations on a couple of images I posted which caused me too look a bit into what was going on. I felt that this image would be interesting to study.

The reason I have chosen to sharpen in FocusMagic is that I wanted to remove sharpening methods from the analysis, but also because I wanted to have a decent level of sharpening so I don't get a lot of nasty comments on my processing. Another factor is that these artefacts show up with sharpening.

Now, I normally print at A2 size, that is around 16"x23". What I have seen is that 16-24 MP is enough to produce really good A2 size prints with Lightroom's sharpening. On the other hand I sometime print larger, like 30"x40" and I have seen some artefacts from Lightroom preparing 24MP images for that size.

What I have noticed with my P45+ images is that Lightroom has a tendency to produce a halo at high contrast edges, which is enhanced by sharpening. I can see that halo even with sharpening set to zero, so I think that it is a weakness of the demosaic algorithm.

What I found is that this is a bit more pronounced in my A7rII images than with the P45+. I don't necessarily think it is a problem print A2 size, but I am pretty sure it needs some care when printing 30"x40".

Looking at the crops at actual pixels I felt the issue was quite visible from a close viewing distance, but much less visible at normal viewing distance.

The amount of sharpening is another discussion. I think that many of us oversharpen to get impressive look at actual pixels, but I would think that we really need sharpening for lower frequencies for optimal prints. So pixel peeping seduces to extensive sharpening at the pixel level at cost of sharpening of lower frequency detail which probably is much more important in prints. But that discussion deserves a separate thread.

Please, feel free to use the image and it is very much OK to share your observations on this thread or in any other relevant context. You can of course PM me, if you prefer so, but this thread is really an invitation to a discussion so feel free to comment.

Best regards
Erik

Hi Erik,

To see any differences at all I had to pump your image up very heavily on my (high-res) display. So that immediately says something, consistent with my general observation that using LR 6.2 with its own sharpening tools on 13*19 prints of even crops of photos I make with my 24MP Sony a6000 I don't see the kind of artifacting you show here.

That said, with your comparison image really pumped up, to my eyes, I thought LMMSE was the the worst result, Lightroom second worst, Amaze second best and C1 the best, but these latter two are very close. I also thought the C1 result overall less sharp and of slightly flatter contrast than the three others. I have no idea whether that is on account of C1 itself or how you deployed the various tools on each of the raw files. It could be interesting to repeat this test using each application's in-built sharpening.

I also notice that opened in Photoshop you are using 8 bit depth rather than 16. I wonder whether this is influencing any of the outcomes?

I only use LR, and I would be interested to see what results I can produce with the same photo in the manner I process photos through LR. If you would be willing to send me your original raw photo with the sole purpose of trying this and reporting back we can arrange off-line.

Cheers,

Mark

[Jack Hogan]

The reason I have chosen to sharpen in FocusMagic is that I wanted to remove sharpening methods from the analysis, but also because I wanted to have a decent level of sharpening so I don't get a lot of nasty comments on my processing. Another factor is that these artefacts show up with sharpening.

Hi Erik, I don't know about C1, but LR/ACR appears to apply some sort of sharpening (perhaps just part of demosaicing) even with all sliders at zero http://www.strollswithmydog.com/raw-converter-sharpening-with-sliders-at-zero/.  I would guess unticking RT's sharpening section means zero additional sharpening.  For a surely unsharpened image you can try DCRAW (it defaults to AHD).

Jack

[Mark D Segal]

Hi Erik,

Thanks for the explanation of the context and the access to the file. I shall give it a whirl in LR and see what I come up with.

Mark

[ErikKaffehr]

Hi Jack,

Thanks for the article!

I am actually aware of LR applying sharpening even with sliders at zero, but I didn't want to elaborate to much on the issue. The thing is that I wanted to compare Lightroom with some other converters and wanted eliminate reduce the effects of sharpening. So I guess that the results I got from Lightroom is the best I could get from Lightroom, regarding artefacts.

My take this far is that I guess I print from Lightroom up to A2, but when sending files to my lab to print larger I might use RawTherapy to produce a TIFF. In general I am quite happy with LR, with the demosaic part being the exception. So I will process that TIFF with Lightroom. It may take some experimentation to find a good process.

Best regards
Erik

Hi Erik, I don't know about C1, but LR/ACR appears to apply some sort of sharpening (perhaps just part of demosaicing) even with all sliders at zero http://www.strollswithmydog.com/raw-converter-sharpening-with-sliders-at-zero/.  I would guess unticking RT's sharpening section means zero additional sharpening.  For a surely unsharpened image you can try DCRAW (it defaults to AHD).

Jack

[Tim Lookingbill]

Hi Erik,

To see any differences at all I had to pump your image up very heavily on my (high-res) display. So that immediately says something, consistent with my general observation that using LR 6.2 with its own sharpening tools on 13*19 prints of even crops of photos I make with my 24MP Sony a6000 I don't see the kind of artifacting you show here.


Cheers,

Mark

I'm viewing on a 1920x1080 calibrated LG display and even at the initial LuLa zoom view of that screengrab I'm seeing a ton of compression artifacts and even more after enlarging by clicking the magnifying glass mouse cursor. See my unedited Raw shot below I took of the Raw Therapee Amaze version, the best one of the four.

Aside from that, Erik, thanks for the post. It's informative to see how demosaicing algorithms between imaging apps affects edge detail which is also nice to know because it isn't camera brand/model related as I was seeing (and suspecting) at dpreview Raw format preview comparison of their test image where I was seeing the zipper/stairstep artifacts on high contrast/saturated color edges.

[kers]

Staircasing is - i think - a logical outcome of a 100% digital image.
If you print at usually about 25 % it has gone and probably is a kind of sharpening that you can use at that level.

[Mark D Segal]

Hi again Erik,

Thanks for the opportunity to try your image in Lightroom 6.2 on my computer and with my printer (Epson 4900). I think the so-called "artifacting" one sees under great magnification and only where extreme light hits dark is simply a limitatio of anti-aliasing. It is non-smoothed pixel corners at very stark edges. Otherwise I see no artifacting or aliasing no matter what. I opened the photo in LR, and made a very small crop of roughly the same image area you show - in my case it was 605*813 pixels of a total image dimension of 5304 * 7952 pixels, or 1.2% of the total image area. I then sharpened it using the LR "Scenic" preset, and normalized the histogram by slightly reducing the highlights and more substantially opening the shadows. I then printed it at 3.35 inches by 4.52 inches, (i.e. providing an input of 180 PPI), which LR then upsampled to 360 PPI on the fly for printing (on Ilford Gold Fibre Silk). Looking at this print with the naked eye really up close, one sees zero artifacting or aliasing. So I then examined those above-mentioned edges, and the rest of the print, under a 10x aspherical loupe and only then saw the stair-stepping at those high contrast edges. If this photo were printed in its entirety using the same settings I used for this snippet, the resulting print would be 29.5 by 44.2 inches, and without using the aspherical loupe it would look stunning. So as far as I'm concerned, none of this gives me pause for one iota of concern about practically useful image quality in a Lightroom workflow.

[Tim Lookingbill]

Staircasing is - i think - a logical outcome of a 100% digital image.
If you print at usually about 25 % it has gone and probably is a kind of sharpening that you can use at that level.

That has not been the case with 1:1 crop zoom in ACR/LR of my Raw images. Fortunately I don't see this on all my images which suggests it has something to do with a type of sensor pixel grid screen frequency to angle of the edges it happens to affect much like what happens with moire. Moire usually kicks up on some of my images depending on the distance of the subject and fineness of the pattern slightly off angle to the sensor's pixel grid.

Some interesting reading on the subject here...

http://elynxsdk.free.fr/ext-docs/Demosaicing/more/JEI013003.pdf

And a demonstration of the affects caused by various demosaicing algorithms at the bottom of this page...

http://www.linuxphoto.org/html/test_demosaicing.html

[ErikKaffehr]

Hi Mark,

Thanks for making that experiment. I am in no way surprised at your findings. On the other hand it is nice to have views on the issue from an experienced printer.

The amount of sharpening is an interesting issue. Image detail is often viewed at actual pixels and sharpening is often chosen to get a good look at actual pixels.

The landscape sharpening in LR is based on the work of the late Bruce Fraser and Jeff Schewe as far as I know and I presume there is a lot of research based on actual prints behind. I guess that I go back to that sharpening as default instead of the more intensive sharpening I normally use.

Just to say, the largest I print normally is about 28"x40" although I just have a few prints of that size. Wall space limitations…

Thanks a lot for your observations!

Best regards
Erik

Hi again Erik,

Thanks for the opportunity to try your image in Lightroom 6.2 on my computer and with my printer (Epson 4900). I think the so-called "artifacting" one sees under great magnification and only where extreme light hits dark is simply a limitatio of anti-aliasing. It is non-smoothed pixel corners at very stark edges. Otherwise I see no artifacting or aliasing no matter what. I opened the photo in LR, and made a very small crop of roughly the same image area you show - in my case it was 605*813 pixels of a total image dimension of 5304 * 7952 pixels, or 1.2% of the total image area. I then sharpened it using the LR "Scenic" preset, and normalized the histogram by slightly reducing the highlights and more substantially opening the shadows. I then printed it at 3.35 inches by 4.52 inches, (i.e. providing an input of 180 PPI), which LR then upsampled to 360 PPI on the fly for printing (on Ilford Gold Fibre Silk). Looking at this print with the naked eye really up close, one sees zero artifacting or aliasing. So I then examined those above-mentioned edges, and the rest of the print, under a 10x aspherical loupe and only then saw the stair-stepping at those high contrast edges. If this photo were printed in its entirety using the same settings I used for this snippet, the resulting print would be 29.5 by 44.2 inches, and without using the aspherical loupe it would look stunning. So as far as I'm concerned, none of this gives me pause for one iota of concern about practically useful image quality in a Lightroom workflow.

[ErikKaffehr]

Hi Mark,

I pretty much repeated your experiment, perhaps taking less care to match the images. My results were somewhat different, though I ended up around 32"x48" for the uncropped print. For me the staircase effect was clearly visible on the green blade in Lightroom while the image produced by RawTherapee was pretty clean.

I was just looking with the naked eye at say 25 cm, but I am near sighted. Looking at say 50 cm with progressive glasses I cannot see the jaggies. I would agree that it is only the high contrast edges on the green blades are affected and also that it is an aliasing artefact, but more and more cameras drop or zero out OLP filtering so I feel that raw converters need to improve in suppressing the resulting artefacts.

I would also agree that it would make for a stunning print. Modern technology gives up remarkable tools, I must say.

Best regards
Erik

Hi again Erik,

Thanks for the opportunity to try your image in Lightroom 6.2 on my computer and with my printer (Epson 4900). I think the so-called "artifacting" one sees under great magnification and only where extreme light hits dark is simply a limitatio of anti-aliasing. It is non-smoothed pixel corners at very stark edges. Otherwise I see no artifacting or aliasing no matter what. I opened the photo in LR, and made a very small crop of roughly the same image area you show - in my case it was 605*813 pixels of a total image dimension of 5304 * 7952 pixels, or 1.2% of the total image area. I then sharpened it using the LR "Scenic" preset, and normalized the histogram by slightly reducing the highlights and more substantially opening the shadows. I then printed it at 3.35 inches by 4.52 inches, (i.e. providing an input of 180 PPI), which LR then upsampled to 360 PPI on the fly for printing (on Ilford Gold Fibre Silk). Looking at this print with the naked eye really up close, one sees zero artifacting or aliasing. So I then examined those above-mentioned edges, and the rest of the print, under a 10x aspherical loupe and only then saw the stair-stepping at those high contrast edges. If this photo were printed in its entirety using the same settings I used for this snippet, the resulting print would be 29.5 by 44.2 inches, and without using the aspherical loupe it would look stunning. So as far as I'm concerned, none of this gives me pause for one iota of concern about practically useful image quality in a Lightroom workflow.

[jrp]

A difference among the images to my eye that is almost as important as the amount of staircasing is the level of purple fringing that is left.

[Denis de Gannes]

Quote
" In all raw processors I disabled noise reduction and sharpening and all the images were sharpened using FocusMagic with its default settings. FocusMagic is a good sharpening tool that does not introduce a lot of artefacts of its own. So all images should have a decent level of sharpening that is comparable by and large."

AFAIK all raw processing software apply sharpening and noise reduction as part of their default rendering process, how do you go about disabling that process. If disabled it means that the sharpening and noise reduction is being applied by Focus Magic after the raw data has been done. To my mind the capture sharpening and noise reduction is a functional part of of the rendering process so you are comparing partially rendered raw data and then applying, sharpening and noise reduction with Focus Magic at the same level?. Why?
As far as I am aware LR / ACR is applying default sharpening and noise reduction to raw files specific to Camera Model and ISO of the image capture. So the amount of sharpening and Noise reduction applied to a Canon 7D Mk II @ ISO 200 file and a Canon 7D Mk II file @ ISO 1,600 would be different. However you the user do not know what the difference is because it is reported as 0 or a set default number.

[ErikKaffehr]

Hi,

Yes you are right, I should have used the term "minimize".

Anyway, the idea was to reduce/eliminate halo effects coming from sharpening and apply a pretty standards sharpening on top so artefacts were made visible.

Best regards
Erik

Quote
" In all raw processors I disabled noise reduction and sharpening and all the images were sharpened using FocusMagic with its default settings. FocusMagic is a good sharpening tool that does not introduce a lot of artefacts of its own. So all images should have a decent level of sharpening that is comparable by and large."

AFAIK all raw processing software apply sharpening and noise reduction as part of their default rendering process, how do you go about disabling that process. If disabled it means that the sharpening and noise reduction is being applied by Focus Magic after the raw data has been done. To my mind the capture sharpening and noise reduction is a functional part of of the rendering process so you are comparing partially rendered raw data and then applying, sharpening and noise reduction with Focus Magic at the same level?. Why?
As far as I am aware LR / ACR is applying default sharpening and noise reduction to raw files specific to Camera Model and ISO of the image capture. So the amount of sharpening and Noise reduction applied to a Canon 7D Mk II @ ISO 200 file and a Canon 7D Mk II file @ ISO 1,600 would be different. However you the user do not know what the difference is because it is reported as 0 or a set default number.

[Bart_van_der_Wolf]

AFAIK all raw processing software apply sharpening and noise reduction as part of their default rendering process, how do you go about disabling that process.

Hi Denis,

In general, Raw processors do not add sharpening, other than that some demosaicing methods handle fine detail more or less aggresssively. So zeroing of the sharpening controls, or in some programs a minus setting, will not add sharpening. However, there is a difference in how much resolution different demosaicing algorithms are able to extract from the undersampled Raw (Bayer-)CFA data.

So what Erik set out to do, seems to create a relatively good baseline for further processing and comparison. What his test confirmed is something that e.g. happened with Capture One when it went from version 6 to version 7 and later. The stairstepping artifacts which were a direct translation of the underlying undersampled data, were significantly reduced with the newer Raw conversion engine. That improvement was also apparent on Optical Low-Pass Filtered sensor data, but OLPF-less cameras got the most benefit, because they by definition produce aliased data samples (and stairstepping is one manifestation of aliasing).

As far as I am aware LR / ACR is applying default sharpening and noise reduction to raw files specific to Camera Model and ISO of the image capture.

Raw converters can use different settings for different cameras at different ISO settings. But here as well, with sliders set to zero, there usually is no noise reduction. With the siiders set to reduce noise (>0), the camera and ISO specific settings for the NR algorithm are used.

Attached, a 400% zoomed crop of a shot with my EOS-1Ds Mark III, with the EF 135mm L lens at its optimum aperture. The same Raw file was converted with a number of Raw converters, No sharpening, No noise reduction, No False Color moiré suppression, of which 3 are shown in an animation.

The LR 5.7 and Capture One Pro 6.4 versions are somewhat similar, but the new engine of Capture One Pro 7.2.3 (which produces identical results as the current version 8.3.x) progressed a significant amount, where LR stood still in its development. The higher resolution that CO since version 7 produces, already helps to reduce aliasing, but the algorithm also tries to avoid other aliasing artifacts with good results. RawTherapee's Amaze algorithm does even marginally better on low ISO material, but is very close to the current Capture One versions.

Cheers,
Bart

[ErikKaffehr]

Hi Bart,

Thanks for explaining both my intention and your findings in a clear and lucid manner. I clearly see that Adobe is lagging behind in doing an accurate raw conversion on non OLP-filtered images.

Mark Segal has a good point that those artefacts I see may not be visible with the naked eye. Still, if I would print large I may use RawTherapy for conversion.

Best regards
Erik

Hi Denis,

In general, Raw processors do not add sharpening, other than that some demosaicing methods handle fine detail more or less aggresssively. So zeroing of the sharpening controls, or in some programs a minus setting, will not add sharpening. However, there is a difference in how much resolution different demosaicing algorithms are able to extract from the undersampled Raw (Bayer-)CFA data.

So what Erik set out to do, seems to create a relatively good baseline for further processing and comparison. What his test confirmed is something that e.g. happened with Capture One when it went from version 6 to version 7 and later. The stairstepping artifacts which were a direct translation of the underlying undersampled data, were significantly reduced with the newer Raw conversion engine. That improvement was also apparent on Optical Low-Pass Filtered sensor data, but OLPF-less cameras got the most benefit, because they by definition produce aliased data samples (and stairstepping is one manifestation of aliasing).

Raw converters can use different settings for different cameras at different ISO settings. But here as well, with sliders set to zero, there usually is no noise reduction. With the siiders set to reduce noise (>0), the camera and ISO specific settings for the NR algorithm are used.

Attached, a 400% zoomed crop of a shot with my EOS-1Ds Mark III, with the EF 135mm L lens at its optimum aperture. The same Raw file was converted with a number of Raw converters, No sharpening, No noise reduction, No False Color moiré suppression, of which 3 are shown in an animation.

The LR 5.7 and Capture One Pro 6.4 versions are somewhat similar, but the new engine of Capture One Pro 7.2.3 (which produces identical results as the current version 8.3.x) progressed a significant amount, where LR stood still in its development. The higher resolution that CO since version 7 produces, already helps to reduce aliasing, but the algorithm also tries to avoid other aliasing artifacts with good results. RawTherapee's Amaze algorithm does even marginally better on low ISO material, but is very close to the current Capture One versions.

Cheers,
Bart

[Denis de Gannes]

@Erik and Bart, thanks for your informative posts and the time you have taken to look into the merits of different software applications for the rendering of raw data from digital cameras. This reassures me that when I say that the process of rendering and processing raw files is still in the early stages of development that my thinking is not far off. While Adobe Photoshop may be considered and rightly so the premier photo editing software on the market, Lightroom/ Adobe Camera Raw in my opinion not there yet. Uses should keep their options open.   

[ErikKaffehr]

Hi,

Mark D Segal has taken the view that the artefacts I demonstrated would not be visible in large prints, so it may matter less than I believe in real world practise.

On the other hand, the Sony A7rII is relatively well behaved, possibly due to the gapless microlenses. A decent fill factor reduces aliasing quite a bit. I feel the the P45+ of mine is much more affected. The screen dump below is from the P45+, processed in RawTherapee/Amaze and LR62. The RT image on the left is much cleaner the LR62 image has more sharpening and contrast but it quite clearly a bit messy. Please notice that the screen dump is at 200%.

Great pity Adobe doesn't use Amaze.

Just to say, I really hoped Adobe would improve on their demosaic algorithms, specially as we now have many cameras without OLP filtering with a simultaneous development of high performing lenses from Zeiss, Sigma and probably some others.

Best regards
Erik

@Erik and Bart, thanks for your informative posts and the time you have taken to look into the merits of different software applications for the rendering of raw data from digital cameras. This reassures me that when I say that the process of rendering and processing raw files is still in the early stages of development that my thinking is not far off. While Adobe Photoshop may be considered and rightly so the premier photo editing software on the market, Lightroom/ Adobe Camera Raw in my opinion not there yet. Uses should keep their options open.

[bjanes]

Attached, a 400% zoomed crop of a shot with my EOS-1Ds Mark III, with the EF 135mm L lens at its optimum aperture. The same Raw file was converted with a number of Raw converters, No sharpening, No noise reduction, No False Color moiré suppression, of which 3 are shown in an animation.

Bart,

Your animation is very instructive and complements Erik's excellent work. When viewing the extinction radii of the star chart, I see that the radii are approximately the same and I interpret the radii as the resolution near the Rayleigh limit. But how do the MTFs compare at lower frequencies? To my eye, contrast at higher frequencies appears slightly better in the renderings showing more Moire.

Regards,

Bill