Processin version 2010 vs 2003

[ErikKaffehr]

Hi,

I made some comparisons of the new processing pipeline in Lightroom with the old one. A short write up, with samples,  is here:

http://echophoto.dnsalias.net/ekr/index.ph...htroom-3-beta-2

Best regards
Erik

[John R Smith]

I sort of hesitate to open my big mouth here, but anyway -

As some of you will remember, I am a new convert to Lightroom, but I now have a fully paid-up licensed copy of 2.6 and the Martin Evening book, and I have been using LR intensively for the past several weeks. This week I downloaded the LR3 v2 beta and have taken just a brief look at it so far.

So far, with my Hasselblad CFV 39MP 3FR files I am not seeing this big difference in IQ that everyone else is raving about. Admittedly, I don't have any high ISO shots - the highest I have used is 400. And all my stuff is immediately converted to B/W, so I don't do colour.

But last night I did some tests, processing the same ISO 200 3FR file in 2.6, and then in 3 beta using the new 2010 process engine. In both cases I did NR and sharpening to what I considered optimal when viewing the image at 100%. I exported a crop of each file as a TIFF, then printed that section without up or down sampling to my R2400 without further sharpening (this is a section of what would be a 24 x 18 inch print). Comparing the two prints, I can really see no difference whatsoever. Well, maybe it's just me.

What I do notice is that the sharpening values in 3 beta 2 are much more extreme in their effect than in LR2. In other words, amount and detail at say value 50 are very much more aggressive. So there is a problem if you update a file previously edited in LR2 to the 2010 engine, in that it may be considerably over-sharpened as a result because the old sharpening numbers are simply carried across. The gamma of the file is also altered to the right (lightened), and you can see the histogram move when you update from process 2003 to 2010. So updating a large number of existing LR2 files to the new process is not necessarily straightforward or indeed a good idea, without further editing.

I am not sure that I see the point of the sharpening changes at all (here I know I am risking the wrath of Jeff). A better algorithm is one thing, and fine if this is the case, but the values from 50 onwards in amount and detail seem to me to be extreme and unusable - so why effectively squash the usable range of each slider into the left-hand side? One of the things I really like about LR2 is that the adjustment range for brightness, contrast, sharpening etc is really gradual and very subtle (apart from the shadow slider, which is a bitch). It is very hard to produce an extreme or un-photographic result in LR2, which is a jolly good thing in my view.

But one pat on the back for LR3 - it loads up all your existing user presets and printing templates. Nice one, chaps  

John

[John R Smith]

Actually, while I'm here (I'm bored and it's pouring with rain here in Cornwall) -

Can I have just another little gripe about LR3 b2? It would have been really, really nice (as others have said) to have had perspective correction included. I would much rather have that than grain or vignetting, honestly Eric. If you don't have a shift lens the ability to correct slightly diverging or converging verticals is an absolute life-saver. And PS Elements will only do it in 8-bit.

And the other one lies with the interface. I know that it is brilliantly trendy and super-cool to have everything really subliminal and shades of gray, but - when you get into your mid-sixties like me and your eyesight is perhaps not altogether what it was, trying to enable/disable the develop controls with a tiny switch which is a dark gray colour set against a slightly darker gray colour is a nightmare. Couldn't you just have made them a tiny bit bigger, and perhaps a little colour would have helped as a background - green for on, red for off? C'mon LR Team, not everyone is in their 20s with perfect vision.

John

[DarkPenguin]

Ptlens is cheap and we do not know what the final feature set will be.   I believe they've kept a surprise or two in years past.

As to the not everyone is in their 20s thing I can't believe it is 2010 and I do not have a clone growing in a tank somewhere.  I'll need it (and a good 100 infants worth of stem cells to recondition my brain for transplant) in 20 years.  Of course if it is a perfect clone it'll need glasses right out of the tank.

[feppe]

What I do notice is that the sharpening values in 3 beta 2 are much more extreme in their effect than in LR2. In other words, amount and detail at say value 50 are very much more aggressive. So there is a problem if you update a file previously edited in LR2 to the 2010 engine, in that it may be considerably over-sharpened as a result because the old sharpening numbers are simply carried across. The gamma of the file is also altered to the right (lightened), and you can see the histogram move when you update from process 2003 to 2010. So updating a large number of existing LR2 files to the new process is not necessarily straightforward or indeed a good idea, without further editing.

Oversharpening is just another symptom of overcooked JPEGs coming out of RAW programs, along with vibrance and local contrast enhancement no matter how appropriate or overdone. It seems that's what people these days want and expect.

[Schewe]

I am not sure that I see the point of the sharpening changes at all (here I know I am risking the wrath of Jeff). A better algorithm is one thing, and fine if this is the case, but the values from 50 onwards in amount and detail seem to me to be extreme and unusable - so why effectively squash the usable range of each slider into the left-hand side?

It's not that the sharpening was changed a lot (the changes are subtle but important) but it's the noise reduction component of the demosiacing being removed makes the Process 2010 look sharper. Process 2003 required a much stronger sharpening "bite" in order to over come overcome the built in and non-adjustable noise reduction...

In Process 2010, the noise reduction component has been removed (well, essentially removed) and all the noise reduction has now been put into the 5 noise reductions sliders under the sharpening sliders.

So, if you have images whose sharpening settings were "perfect" in Process 2003 and you switch to Process 2010, they will no longer be "perfect" without additional work.

The two areas to pay very close attention to is the Detail slider settings and the Luminous Noise settings...

If you have adjusted an image to a higher than default Detail setting in Process 2003, the odds are you'll have to back that off. Also, if you like the noise profile of the Process 2003 you WILL need to add noise reduction even if your image is a lower ISO well exposed image.

The reason the engineers decided to do "Process Versions" was to allow people to continue to use the previous processing if the user really, really like the old process. However, the vast majority of camera users will, I believe find the new Process 2010 image quality to be superior to Process 2003 and will want to adopt it. However, to adopt it you WILL need to re-learn the finesse you may have developed with Process 2003–which won't happen overnite. Which is also one of the reasons Lightroom 3 Beta 2 was released...not only to get feedback but to also help users prepare for the changes.

[John R Smith]

Aaaah, I see.

So if you remove the underlying NR, then the same amount of sharpening suddenly looks a whole lot sharper. Yes, understood.

Right, thank you Jeff. I shall go back and do some more testing. I think I am going to have to do this in colour instead of B/W, so I can actually see the noise more clearly. Just when you think you were getting somewhere (with LR 2.6) you find that you weren't and all the rules have changed . . .

John

[pdm]

I've noticed some strangeness with the noise reduction in 2010. I didn't use the new sliders in this example, so the settings between LR2 and LR3 with 2010 are the same values, though from what i've read in this thread here, the underlying meaning for those same settings may indeed be different, and perhaps I've over done something here.

Notice how the bottom one is quite pixelated around the catchlight. Quite bizarre.

This is 100% crop:

http://ruminate.net/tmp/LR_noisereduction.jpg

FWIW, the eye wasn't in perfect focus to begin with.

[ErikKaffehr]

Hi,

My guess is that you have a bit sharpening involved vid significant masking to protect the skin, so the pixelation comes from sharpening differences and the mask.

BR
Erik

I've noticed some strangeness with the noise reduction in 2010. I didn't use the new sliders in this example, so the settings between LR2 and LR3 with 2010 are the same values, though from what i've read in this thread here, the underlying meaning for those same settings may indeed be different, and perhaps I've over done something here.

Notice how the bottom one is quite pixelated around the catchlight. Quite bizarre.

This is 100% crop:

http://ruminate.net/tmp/LR_noisereduction.jpg

FWIW, the eye wasn't in perfect focus to begin with.

[John R Smith]

Notice how the bottom one is quite pixelated around the catchlight. Quite bizarre.

This is exactly what I was trying to explain (although perhaps not very clearly). If you convert files from process 2003 to 2010 it looks as if you will need to revisit the details panel and start from scratch with NR and sharpening, as Jeff makes clear. After I logged off last night, I did a bit more testing. The noise reduction in LR3 b2 is actually very good indeed. You can apply significant amounts of colour NR without apparently losing any detail at all (balancing NR against the detail slider and sharpening level), so reworking Process 2003 files may be well worth while, especially with high ISO shots.

John

[madmanchan]

Correct, you do not want to use numbers that you used for PV 2003 directly with PV 2010. Particularly when using extreme numbers, results will be poor. You gave one example earlier with sharpening. Another one is NR. For example, suppose you shot an ISO 6400 image in the past and previously tried LR 2's NR (equiv of LR 3's PV 2003). To try to minimize color blotches, you cranked the color slider all the way up to 100. Well, due to the new methods and calibrations used for PV 2010's NR, a "Color" noise reduction slider value of 100 is likely to be way too strong. If you have an image at PV 2003 and want to see what it looks like at PV 2010, you may want to reset sharpening & NR set to their default values (maybe we should take care of that automatically when you click on the warning icon).
 
Some sharpening tips:

- Yes, it is true that if you crank sharpening's Amount and Detail both up, things will get crunchy and look bad.
- However, if you crank Amount up but turn down Detail, I believe that you can get very useful sharpening results. Better edge definition without texture emphasis.
- Similarly, if you crank Detail up but turn down Amount, I also believe that you can get useful sharpening results, and different from the previous (i.e., high amount, low detail). The latter in particular will accentuate texture. If you don't want to bring out texture (e.g., in skin!), keep Detail down.

This is why the slider range is there. It's not there so that you can crank up all the sliders and hope you get something useful.  

As an analogy to another set of controls ... I guarantee that if you crank Exposure to +4, Brightness to +150, and Fill Light to +100, you probably won't get anything useful, either. (i.e., the image will become basically all white -- not very useful). But you can do useful things with high Exposure and low Brightness, or vice versa. The former brings out highlight detail and compresses midtones & shadows; the latter does the opposite.

One more tip:

- If you want to fine-tune sharpening in a part of the image, without affecting other areas, try local sharpening (e.g., via the brush). If you've already applied capture sharpening to an image but find it's a little too strong in a specific area (e.g., the catchlight), try using a brush with negative sharpening to "back off" on the sharpening in that area alone.

[feppe]

It's not that the sharpening was changed a lot (the changes are subtle but important) but it's the noise reduction component of the demosiacing being removed makes the Process 2010 look sharper. Process 2003 required a much stronger sharpening "bite" in order to over come overcome the built in and non-adjustable noise reduction...

In Process 2010, the noise reduction component has been removed (well, essentially removed) and all the noise reduction has now been put into the 5 noise reductions sliders under the sharpening sliders.

So, if you have images whose sharpening settings were "perfect" in Process 2003 and you switch to Process 2010, they will no longer be "perfect" without additional work.

The two areas to pay very close attention to is the Detail slider settings and the Luminous Noise settings...

If you have adjusted an image to a higher than default Detail setting in Process 2003, the odds are you'll have to back that off. Also, if you like the noise profile of the Process 2003 you WILL need to add noise reduction even if your image is a lower ISO well exposed image.

The reason the engineers decided to do "Process Versions" was to allow people to continue to use the previous processing if the user really, really like the old process. However, the vast majority of camera users will, I believe find the new Process 2010 image quality to be superior to Process 2003 and will want to adopt it. However, to adopt it you WILL need to re-learn the finesse you may have developed with Process 2003–which won't happen overnite. Which is also one of the reasons Lightroom 3 Beta 2 was released...not only to get feedback but to also help users prepare for the changes.

Thanks, that makes sense although creates extra work and might confuse upgraders. I just hope the overcooked default upgrade results for photos originally processed with version 2003 doesn't become the new standard...

This is exactly the kind of insight and practical information I enjoyed with the LR 2 tutorial. We can all read manuals, so the added benefit of the tutorials comes from your and Michael's commentary to how to make things work better and faster. Looking forward to LR3 tuts!

[Schewe]

Thanks, that makes sense although creates extra work and might confuse upgraders.

Yep, there is that risk. That is why you must actually go in and actively upgrade your images from Process 2003 to Process 2010 manually (or via a preset) so images that have been tuned previously won't get updated by accident.

But I really don't know what else the ACR/LR team could have done. The Process 2010 demosiacing and noise reduction are such a tremendous change that they had to introduce the dual process scheme.

Obviously there will be a period of transition where LR 2.x users need to adapt to the new Process 2010 potential for image quality. The testers were kinda shocked when the new noise reduction went from 3 slider (LR 3 beta 1) to 5 sliders...but that's what it took to accomplish the optimal noise reduction in the Process 2010.

It takes practice and will vary camera by camera and ISO by ISO. But the effort will be worth it! Once you get a better handle on how the new controls interact, I really do suggest the creation and use of Presets and don't forget, the ACR/LR "Default" can be set to be ISO specific so you can let ACR/LR do the heavy lifting and then just go in an fine tune for the specific image.

[Jeremy Payne]

Based on what I've seen so far, I'm not likely to update most of my old images ... I'll probably go back and revisit anything shot at ISO 1600 and above, but anything under that will probably remain process 2003 forever ...

Done is done, and done is good.

[Schewe]

...I'll probably go back and revisit anything shot at ISO 1600 and above, but anything under that will probably remain process 2003 forever ...

That's fine and a good reason the team decided to do the dual process route rather than forcing people to update. Note however, new images will by default, come in with Process 2010 unless you go in and change your ACR/LR "Default" for your camera...

[Jeremy Payne]

That's fine and a good reason the team decided to do the dual process route rather than forcing people to update. Note however, new images will by default, come in with Process 2010 unless you go in and change your ACR/LR "Default" for your camera...

I'm totally cool with using 2010 for everything going forward as I see it as a "pick 'em" for anything under ISO 1600 and a clear win for 2010 at 1600 and up.

In fact, I've thrown caution to the wind and have been using the beta for all my work for a while now ...

I know, I know ... no guarantees ... but I'll take the risk.

[madmanchan]

I just hope the overcooked default upgrade results for photos originally processed with version 2003 doesn't become the new standard...

Point taken, and I understand the concern.

I don't think that will happen, though. Please keep in mind that the default settings for PV 2010 sharpening still have the same numbers as before (e.g., 25/1/25/0) and if you're bringing in new images into LR 3 and just using the defaults, this should not result in "overcooked" sharpening. The only cases where you should have to be careful is if (1) you are using a custom per-camera default or import preset with different sharpening numbers, or (2) if you're going back to work on older images -- which of course is what we've been discussing in this thread. In both cases you may have to do some tweaking, which as Jeff noted is why we made the PV change a manual switch (instead of forcing an automatic PV change to existing images, without asking).

The larger topic being explored in this thread is how does one deal with backwards compatibility in a parametric (non-destructive, if you prefer) editing environment like LR. We struggled with this, with many discussions internally and externally. We didn't want to change the appearances of your existing images behind your backs yet we also wanted to provide you with improvements you've been asking for. It's a tricky balance.

[John R Smith]

OK, chaps, I'm working between two PCs here, this one on the Internet and my other PC with LR on it. Since yesterday I have been testing this stuff like a lunatic and my hard drive is smoking and my track ball is flaccid. Time for a break.

I still have a real problem with this whole issue, in that I cannot see ANY difference between my 3FR Hasselblad files between LR 2.6 and LR3 b2 at the basic import level, that is with all NR and sharpening switched OFF.

Now, according to Jeff and the release notes for beta 2 - LR2 does some basic noise reduction by default. So if I compare the two, then LR 2 should look less noisy but slightly softer, and LR3 should look more noisy but a little crisper. So I may be doing something completely wrong, but here goes -

Taking a 3FR file which I edit in 2.6 (to a sidecar XMP file), and setting all NR and sharpening off in 2.6. Crop it to a very small section which is good and noisy (ISO 400 and the 'blad is poor even at this speed). Open 3 beta 2 and in Develop module convert to PV 2010 with compare, so now both 2003 and 2010 versions are side by side on screen. No difference to my eyes at all, they look identical, exactly the same amount of colour noise at any magnification. To be sure I printed the same blown-up section from LR2 and LR3 and checked the prints with an 8x loupe. Same result, no difference.

This is not a criticism of the new NR and sharpening routines, I am sure that they are very good when you get used to them. But if the whole demosaicing algorithm is different, I should be seeing something, surely? I am really quite disappointed, as everyone else is raving about the greatly improved IQ, and I am missing out for some reason. And its been raining all day.

John

[Jeremy Payne]

But if the whole demosaicing algorithm is different, I should be seeing something, surely? I am really quite disappointed, as everyone else is raving about the greatly improved IQ, and I am missing out for some reason. And its been raining all day.

John

I have a Nikon D700 ... and there is so little noise until you get past ISO 800 that I see very little difference between the two process versions until I hit ISO 1600.

By 6400, it is night and day.

[Jeremy Payne]

We didn't want to change the appearances of your existing images behind your backs yet we also wanted to provide you with improvements you've been asking for. It's a tricky balance.

As a fellow software soldier, I feel your pain ...

I think you handled it well - I especially like the new Before/After comparison view on conversion.