ACR 4.2 vs Raw Therapee 2.4b1 (pt 1 results)

[brianchapman]

I just posted a bare bones comparison between ACR and Raw Therapee (which hasn't been mentioned here in a while) on my site.  It's certainly not a dpreview quality review nor is it exhaustive by any stretch of the imagination but still interesting results nonetheless.  I normally use ACR in to Photoshop but I like to try different things once in a while just to see how the competition fares.    

http://www.brianchapmanphotography.com/blo...verters-part-1/

GLuijk has posted some promising results with the AFD algorithm they're implementing for PerfectRaw (when's that new version coming out?) and my first impressions of the Raw Therapee HPHD algorithm seems like it does a similar job with the noise (although it has some other issues which I'll show in part 2 hopefully this weekend).  The images I'm using are both night exposures, one on the Oregon coast under moonlight and the other in the city which poses some different challenges.

In any case, I hope this is at least informative.  For what it's worth this was all prompted by two things - the fact that I have a 5D Mark II on order and I'm debating whether or not to upgrade to CS4 (or at least wait a bit to save the $$) and also that I'm still hoping Adobe will provide the option to turn off noise reduction all the way in ACR.

Brian

[Schewe]

The fact you ran Camera Raw's sharpening at an amount of 2 (unless I'm misreading your screenshot) and no luminance noise reduction you pretty much hosed the results...in RT you used a .74 radius (instead of the default 1 in CR 4.2) and an amount of what, 52? Don't know what the scale is in RT, but considering t sure seems HIGHER than Camera Raw I really don't understand what you think you've proven (other than perhaps you haven't figured out how to use Camera Raw's sharpening controls).

Also, since CS4 is shipping and the current version of Camera Raw is 5.1, doing a comparison of 4.2 is maybe a year or so late? While the sharpening was introduced in CR 4.1, substantial adjustments were made to the luminance noise reduction even as late as 4.41 (not to mention the CURRENT version for CS3 is CR 4.6).

It would be interesting to see side-by-sides of CR vs RT, but your test is pretty much flawed...it would be useful to see what the BEST you can get out of both, not less than the best.

[brianchapman]

The fact you ran Camera Raw's sharpening at an amount of 2 (unless I'm misreading your screenshot) and no luminance noise reduction you pretty much hosed the results...in RT you used a .74 radius (instead of the default 1 in CR 4.2) and an amount of what, 52? Don't know what the scale is in RT, but considering t sure seems HIGHER than Camera Raw I really don't understand what you think you've proven (other than perhaps you haven't figured out how to use Camera Raw's sharpening controls).

Also, since CS4 is shipping and the current version of Camera Raw is 5.1, doing a comparison of 4.2 is maybe a year or so late? While the sharpening was introduced in CR 4.1, substantial adjustments were made to the luminance noise reduction even as late as 4.41 (not to mention the CURRENT version for CS3 is CR 4.6).

It would be interesting to see side-by-sides of CR vs RT, but your test is pretty much flawed...it would be useful to see what the BEST you can get out of both, not less than the best.

Jeff,

Thanks for pointing out the error in the screenshot - I must have bumped it accidentally - it has been updated in the post.  I have confirmed the image is correct and that the sharpening was set at 0.  Regarding the RT settings, unchecking the 'Enabled' box turns off the feature so I didn't bother setting the sliders to the 0 position as well but in both cases sharpening was turned off.  

I am using version 4.4.1.85 (March 08) - so I have not upgraded to 4.6 but I didn't see anything in the release notes that would indicate any need for me to do so.  The title of this forum post says (until I see if I can change it) 4.2 but in my blog post I stated that the noise control has been adjusted since 4.2.  Version 5.1 was just released and isn't a free upgrade so I'm probably not going to get it right away even though it's got some really appealing features (ACR and Photoshop).  

I put this information out there as information - I'm not trying to bash ACR although I do believe that there is no harm done by Adobe allowing those of us who want the option to turn off noise reduction to have it.  

I generally can't get the "best" for one image with one round of processing through ACR (or any other converter) - improvements in certain areas often result in sacrifices elsewhere, so I thought it would be useful to see a comparison of the "bare bones" output from the tools.  I know the arguments for doing the post processing operations on the linear data but frankly despite the hullabaloo I haven't seen much  difference doing the same operations in post processing in Photoshop.  Maybe all that changed with the localized adjustments in 5.1 but I haven't gotten there yet.  

In any case, when I'm done with the next part of the post you probably won't be surprised to find out that of the two ACR, even the ancient version I'm using, performed significantly better with the next image.  

Brian

[brianchapman]

Anyone know how to update the title of the thread?

It should read:

ACR 4.4.1.85 vs Raw Therapee 2.4b1 (pt 1 results)

[Schewe]

Thanks for pointing out the error in the screenshot - I must have bumped it accidentally - it has been updated in the post.  I have confirmed the image is correct and that the sharpening was set at 0.  Regarding the RT settings, unchecking the 'Enabled' box turns off the feature so I didn't bother setting the sliders to the 0 position as well but in both cases sharpening was turned off.

Unless you ALWAY turn sharpening off in Camera Raw, then turning it off is really only going to show you what two processors can do with their tools turned off. That's kinda like trying to determine what's the best raw processor without ever making and adjustments for tone and color. The tools are there for a purpose. You are dismayed that Camera Raw 4 incorporated improved noise reduction in the demosaicing...yeah, well part of that was to make the images more sharpenable. So, turning off the sharpening results in soft output. Expectable, huh? Unless you actually deploy the tools and set an optimum for each processor, it's really telling you a false story unless you NEVER sharpen in Camera Raw (which would be too bad cause CR's sharpening is really, very good).

BTW, looking at a 100% crop of an image that will have between 240-480PPI after processing is really pretty usless for determining the image quality for print. You can;t really tell much about what the images would look like at web rez either since nobody would be posting full resolution files on the web. While it's useful to determining exactly how to sharpen and image in Camera Raw and Lightroom, trying to determine what is noise and what is micro-detail on a computer display is pretty south of useful. An image at 300PPI viewed on a computer display that is about 100 pixel per inch will be showing your image 3X the actual size and 1/3 of the resolution as a print file would look in print. Keeping super high frequency noise (not image detail) does not improve image quality...what you see at 100% is irrelevant to print output. You "think" you are seeing things, but that stuff will never print.

Really, you should be testing what the "optimal" results you can achieve, not by turning tools off.

[brianchapman]

You are dismayed that Camera Raw 4 incorporated improved noise reduction in the demosaicing...yeah, well part of that was to make the images more sharpenable. So, turning off the sharpening results in soft output. Expectable, huh?

Sure, improved noise reduction = softer output + acr sharpening (correctly applied) = better overall output.  If the tools are perfect it makes sense but it's difficult to build something that works in all circumstances - that's why people like options.  In the particular case of the image I'm using as an example the surface texture removed by the noise reduction actually enhances the appearance of "sharpness" and even adding back the sharpening (to the best of my ability in ACR) still results in nothing better than the equivalent output to the image that was never softened by the noise reduction in the first place (viewing at both 100% and print size).  If Adobe let me decide (seriously, why not? and especially considering the localized adjustments in 5.1) when I wanted to use noise reduction (any at all) and then repair the resulting softness with sharpening I would be perfectly happy - I mean I've been using ACR/Photoshop since it came out so I'm not exactly anti-Adobe.  

As I said before, however, that on an image by image basis there are distinct differences and in the second part of the post I am working on your theory is very well supported because there is not the kind of surface texture that the default noise reduction in ACR might remove.  As a result the ACR image is much better IMO.

...trying to determine what is noise and what is micro-detail on a computer display is pretty south of useful. An image at 300PPI viewed on a computer display that is about 100 pixel per inch will be showing your image 3X the actual size and 1/3 of the resolution as a print file would look in print. Keeping super high frequency noise (not image detail) does not improve image quality...what you see at 100% is irrelevant to print output. You "think" you are seeing things, but that stuff will never print.

I have no disagreement with this at all - if anything I spend too much time "fixing" things that have no chance of ever showing up in a print.  It's hard to let go of things you see in front of you even though logically you know they won't be there in the final output...at least for me.

Brian

[teddillard]

Thanks for this post and the hard work Brian...  it looks like a great site!  ... looking forward to diving into it deeper.

So, I'm guessing you're not happy with the Canon's DPP noise processing either?  I haven't played with it lately, esp. on the noise front...

[brianchapman]

Thanks for this post and the hard work Brian...  it looks like a great site!  ... looking forward to diving into it deeper.

So, I'm guessing you're not happy with the Canon's DPP noise processing either?  I haven't played with it lately, esp. on the noise front...

Thanks Ted.  I don't like DPP really - in fact I haven't used it since I started using ACR years ago.  

[brianchapman]

Part 2 has been posted...  
http://www.brianchapmanphotography.com/blo...verters-part-2/

Be sure that if you are going to draw any conclusions that you read the last line first to keep things in perspective while reading the rest of it.

[teddillard]

Part 2 has been posted...  
http://www.brianchapmanphotography.com/blo...verters-part-2/

Be sure that if you are going to draw any conclusions that you read the last line first to keep things in perspective while reading the rest of it.

Thanks once again...  what a ton of work!  I've got to go back and read both again in detail a few times.

That last line reminded me of a talk I had with Bill Atkinson.  I was with a bunch of friends who were at his studio, and we were comparing prints and all sorts of talk, and it came around to the 1440/2880 Epson printing question, that is, which dot pitch do you print at.  Bill said, all I'm gonna say is the 2880 lays down a LOT of ink...  complete with coy smile.  

That was enough to get me to go back and run the tests...  I printed samples and measured them and the difference in black density was really astounding.  I can't remember the numbers, but it wasn't a little difference, it was huge.  I left the prints out for a few weeks (this was a digital imaging lab with pretty heavy traffic from all types- working photographers, fine art printers to amateurs) and there was one conclusive decision...  not one person could see the difference.  

Thought you'd like a story while nursing your smoking computer back to health!

Ted

[brianchapman]

Thanks once again...  what a ton of work!  I've got to go back and read both again in detail a few times.

That last line reminded me of a talk I had with Bill Atkinson.  I was with a bunch of friends who were at his studio, and we were comparing prints and all sorts of talk, and it came around to the 1440/2880 Epson printing question, that is, which dot pitch do you print at.  Bill said, all I'm gonna say is the 2880 lays down a LOT of ink...  complete with coy smile.  

That was enough to get me to go back and run the tests...  I printed samples and measured them and the difference in black density was really astounding.  I can't remember the numbers, but it wasn't a little difference, it was huge.  I left the prints out for a few weeks (this was a digital imaging lab with pretty heavy traffic from all types- working photographers, fine art printers to amateurs) and there was one conclusive decision...  not one person could see the difference.  

Thought you'd like a story while nursing your smoking computer back to health!

Ted

Ha!  Yeah I'm not surprised.  I've spent *way* too many hours over the last few years working on things that in the end don't matter that much.  (But they still matter at the time!)

Glad I read your blog by the way - some good info there!

Brian

[teddillard]

Ha!  Yeah I'm not surprised.  I've spent *way* too many hours over the last few years working on things that in the end don't matter that much.  (But they still matter at the time!)

Glad I read your blog by the way - some good info there!

Brian

Thanks, Brian, glad you found it helpful...  

Jeff's very helpful comments on your testing also brought me back to testing that I used to do on cameras, when I was working for a retailer.  Basically I came to the conclusion that, until I saw files, I couldn't make valid conclusions about file quality.  I set up a pretty hilarious test board, with stuff on it that could bring any camera to it's knees.  (I always crack up when I see that PDI test target image, it's so pristine...)  

hey, waddaya know...  it's still here: http://www.teddillard.com/2008/05/g9-test-shot.html
(This is why I keep the blog, so I can find my own stuff.    )

Anyway, about half the time I'd set up 2 cameras, side by side, complete with capture systems, and shoot the tests on as level a field as I could...  only to realize 2 days later I overlooked some variable that required me to re-shoot.  Processing strategy is a huge question.  Do you want to process at baseline defaults, or to the individual camera best advantage?  

The "best advantage" strategy is what I ended with, especially for processing systems, the assumption that you have two users who are shooting under the exact same conditions with different cameras, who know how to get the most out of the equipment.  The problem with that is, to do it right you need to know each capture package intimately.  That 's not just knowing what the manufacturer trains you, but what really works best (due respect to the engineers, but that's about the last thing they know in actual shooting conditions).

Anyway, you get the idea...  and then, showing a photographer the files, side-by-side, shot at optimal conditions, (like the lowest ISO, for example), the guy would say, yeah, whatever, how do they stack up at ISO 400?  d'OH!

Anyway, it is very valuable to run the tests, and re-run them, however OCD  it may seem at the tme...  it's gone a long way to my own understanding of processes and answering questions.  Usually it's my first answer to a student asking which way to do stuff, try it and see which is best.  

Thanks again.  
 

PS- that G9 test file?  I have shots from ca. 2001 cameras that cost over 25K (that shall remain unnamed) that my little G9 kicks around the block!  How far we've come...

[Dinarius]

Brian,

This is really interesting and I am intrigued by the results.

I just have a wee suggestion.......

In, for example, your last paragraph in your second article you write:

"To me the details in the ornamentation and small areas where leaves/branches are visible look more natural and less like I tried to add sharpening to bring them out."

All very well, but in the real world of commercial photography, those of us who might be interested in what other raw converters are offering need to see comparisons between things we recognize. The familiar is always more revealing.

A side-by-side comparison of two cans of COKE shot under candle light could be very embarrassing (joke!) for the big boys.

Seriously though, aren't there practical implications involved here? e.g. A file's behaviour under massive enlargement?

I'm not being facetious, I'm serious! The text on a Coke can @ 300% would be very revealing.

That said, ACR is fab. My only regret is that I can't process my Hassie files with it. Only my Canon ones. But, that's not Adobe's fault.

Well done!

D.

[brianchapman]

Anyway, it is very valuable to run the tests, and re-run them, however OCD  it may seem at the tme...  it's gone a long way to my own understanding of processes and answering questions.  Usually it's my first answer to a student asking which way to do stuff, try it and see which is best.  

PS- that G9 test file?  I have shots from ca. 2001 cameras that cost over 25K (that shall remain unnamed) that my little G9 kicks around the block!  How far we've come...

You're exactly right!  It would be completely impossible for me to set up any kind of test that would satisfy all conditions so I didn't even bother trying (which is what I thought I was implying by saying 'bare bones' in the first post  ).  This was definitely a real world (my world at least) test which, as you suggested, turned in to a great learning opportunity...well, after about 10 hours of re-running the scenarios!  

I started with a Fuji 2000 or something like that, then to a G2 - which at the time I thought was the best thing that ever happened, then to a Rebel XT - which I still have, and I'm finally taking the full frame jump with the new 5D - as long as they don't jack up the price on my preorder(s).  It's amazing how far things have come!

[brianchapman]

Brian,

This is really interesting and I am intrigued by the results.

I just have a wee suggestion.......

In, for example, your last paragraph in your second article you write:

"To me the details in the ornamentation and small areas where leaves/branches are visible look more natural and less like I tried to add sharpening to bring them out."

All very well, but in the real world of commercial photography, those of us who might be interested in what other raw converters are offering need to see comparisons between things we recognize. The familiar is always more revealing.

A side-by-side comparison of two cans of COKE shot under candle light could be very embarrassing (joke!) for the big boys.

Seriously though, aren't there practical implications involved here? e.g. A file's behaviour under massive enlargement?

I'm not being facetious, I'm serious! The text on a Coke can @ 300% would be very revealing.

That said, ACR is fab. My only regret is that I can't process my Hassie files with it. Only my Canon ones. But, that's not Adobe's fault.

Well done!

D.

I hadn't really thought of the implications of the results after a massive enlargement so I'll definitely try that out (thanks!)  I suspect that the more natural look in the RT "best" file would behave better in such a situation but of course that's what the test is for!  I am going to do some more experimentation with the RT sharpening so I'll pass those along when I'm done.

Brian