Luminous Landscape Forum

Raw & Post Processing, Printing => Adobe Lightroom Q&A => Topic started by: John R Smith on December 15, 2010, 04:32:31 am

Title: Problems With Auto-Mask
Post by: John R Smith on December 15, 2010, 04:32:31 am
Do other people here have problems with the Auto-Mask feature in LR? When I use a local adjustment brush with auto-mask, I get very strange edge effects, difficult to describe exactly – a sort of crinkly, luminous edge, I suppose. So then I have to spend time afterwards with a very small brush going around cleaning them up, which rather negates the usefulness of this feature.

Do others experience this, or is it just me (yet again)?

John
Title: Re: Problems With Auto-Mask
Post by: dchew on December 15, 2010, 07:01:54 am
I sure do when I'm making large adjustments (large = moving a slider in a big way).  For example a B&W conversion in which I try to darken the sky dramatically. Also happens when I make large changes in the luminance of a specific color after converting to B&W (not sure that has anything to do with auto-mask).  I take it as a sign I'm pushing the image too far, but it would be nice to have some sort of 'feather' or 'blur' feature associated with auto-mask, and with the color sliders as well.

Dave
Title: Re: Problems With Auto-Mask
Post by: donbga on December 17, 2010, 10:43:24 am
I sure do when I'm making large adjustments (large = moving a slider in a big way).  For example a B&W conversion in which I try to darken the sky dramatically. Also happens when I make large changes in the luminance of a specific color after converting to B&W (not sure that has anything to do with auto-mask).  I take it as a sign I'm pushing the image too far, but it would be nice to have some sort of 'feather' or 'blur' feature associated with auto-mask, and with the color sliders as well.

Dave

Ditto what John and Dave have reported. Adjustments made with very large slider settings and self masking result in some nasty looking artifacts.

Don
Title: Re: Problems With Auto-Mask
Post by: Costas on December 18, 2010, 11:16:13 am
and I thought I was doing something seriously wrong when I got results like that using Auto-Mask and the local adjustment brush. At least I am not alone  :'(
Title: Re: Problems With Auto-Mask
Post by: john beardsworth on December 18, 2010, 12:17:01 pm
You can't expect something like Automask to get it right every time, with every kind of edge transition. Much of the time it'll do a good job of detecting the edges, but sometimes the transitions are that bit too subtle for it. In such cases I'd use Automask to get most of the way and then mop up the rest with it switched off.
Title: Re: Problems With Auto-Mask
Post by: donbga on December 25, 2010, 12:13:07 pm
then mop up the rest with it switched off.

Do you have a 'Mop up' Custom brush setting for the Adjustment Brush? :)

Title: Re: Problems With Auto-Mask
Post by: John Caldwell on January 19, 2011, 10:50:25 am
When you examine the actual mask that's being laid down ( O key), are there drop outs at the edge of the mask? If so, it's not ar artifact that you are seeing when imposing your brush settings; it's just that the settings are being discontinuously applied in the mask drop-out areas. Remember that auto mask samples the pixels where the brush first touches the canvas to define what it will include in the mask. Many times the edge of an area being masked will contain values very different from the more central region of those areas. The work around is that after you've made your more central selections, zoom in and touch down where your areas of dissimilarity arise. Certainly turning off the auto mask may be necessary for some brush strokes.

John-
Title: Re: Problems With Auto-Mask
Post by: Slobodan Blagojevic on January 19, 2011, 11:17:04 am
If you ever wondered why we still need Photoshop, this is your answer ;)
Title: Re: Problems With Auto-Mask
Post by: Steve Weldon on January 19, 2011, 04:07:10 pm
– a sort of crinkly, luminous edge, I suppose.

I've noticed this, and imo it's a type of CA you're seeing which is being exposed by the auto-mask and large changes in levels between the two areas.  Normally we only notice CA when one edge is up against a bright background.. But just because we can't see it (easily) doesn't mean it's not there on close inspection.  Artificially create a bright background and now you can see what you didn't see before.  You'll notice it doesn't do this with all your images.  Start noticing which ones and you'll see where certain CA prone lenses at certain apertures are much more of an issue.

There are lots of ways to fix this.. paint over, replace color (in PS), clone edge to edge.. it's time consuming.

But I really don't think it's 'the' fault of the automask feature.  I think its just doing its job and finding that edge of contrast it was designed to do..
Title: Re: Problems With Auto-Mask
Post by: Schewe on January 19, 2011, 11:38:09 pm
Remember that auto mask samples the pixels where the brush first touches the canvas to define what it will include in the mask.

Actually, it's a bit more sophisticated...it's the CENTER of where your brush first hits and that point remains for the duration of that stroke. Next stroke it's again the center of THAT stroke. So the better strategy is to be very accurate where you start a stroke and while the mouse is held down you can really go anywhere and that first sample will be the mask tone/color.

Also note that Auto-Mask is not designed for massive image adjustments but more subtle adjustments. You will always have problems with edges if the amount of the adjustment is strong.

Another way to use Auto-Mask is to paint the original adjustment in without Auto-Mask but use Auto-Mask when subtracting (option/Alt key) or erasing the mask. Many times this actually works better.

Auto-Mask was based on the design and function of Background Eraser in Photoshop. For what it does, it's very good if you keep things subtle. If you need massive adjustments then you really need to go into Photoshop where very accurate selections are possible.
Title: Re: Problems With Auto-Mask
Post by: wolfnowl on January 20, 2011, 12:25:35 am
Quote
Another way to use Auto-Mask is to paint the original adjustment in without Auto-Mask but use Auto-Mask when subtracting (option/Alt key) or erasing the mask. Many times this actually works better.

Never thought of that... thanks!  I use the Alt-brush option to undo, but hadn't thought of that technique...

Mike.
Title: Re: Problems With Auto-Mask
Post by: John Caldwell on January 20, 2011, 07:27:23 am
Actually, it's a bit more sophisticated...it's the CENTER of where your brush first hits and that point remains for the duration of that stroke. Next stroke it's again the center of THAT stroke...

Thanks for the elaboration regarding the redefinition of sample with each stroke.

Another way to use Auto-Mask is to paint the original adjustment in without Auto-Mask but use Auto-Mask when subtracting (option/Alt key) or erasing the mask.

Regarding a back and forth with adding and subtracting to the mask, but toggling Auto Mask as your working, are there key stroke shortcuts that permit selection of Brush A, Brush B, and Auto Mask On vs. Auto Mask Off? As we now enjoy with Option to engage Erasure, it would be great to have a key stroke to select brush A, B and a Auto Mask toggle.

John-
Title: Re: Problems With Auto-Mask
Post by: Robert-Peter Westphal on January 20, 2011, 12:19:36 pm
...
Also note that Auto-Mask is not designed for massive image adjustments but more subtle adjustments. You will always have problems with edges if the amount of the adjustment is strong.
...

"Subtle and delicious" !!! ;D
Title: Re: Problems With Auto-Mask
Post by: David Eichler on January 29, 2011, 06:59:50 pm
Actually, it's a bit more sophisticated...it's the CENTER of where your brush first hits and that point remains for the duration of that stroke. Next stroke it's again the center of THAT stroke. So the better strategy is to be very accurate where you start a stroke and while the mouse is held down you can really go anywhere and that first sample will be the mask tone/color.

Also note that Auto-Mask is not designed for massive image adjustments but more subtle adjustments. You will always have problems with edges if the amount of the adjustment is strong.

Another way to use Auto-Mask is to paint the original adjustment in without Auto-Mask but use Auto-Mask when subtracting (option/Alt key) or erasing the mask. Many times this actually works better.

Auto-Mask was based on the design and function of Background Eraser in Photoshop. For what it does, it's very good if you keep things subtle. If you need massive adjustments then you really need to go into Photoshop where very accurate selections are possible.

I recall an example in Martin Evening's LR2 book, where a solid color background is completely replaced with a very different color, with the Adjustment Brush and automasking. However, this was a case where the border between the subject and background was very distinct, and the background was very even in color and illumination.
Title: Re: Problems With Auto-Mask
Post by: Rory on January 30, 2011, 11:32:29 am
Regarding a back and forth with adding and subtracting to the mask, but toggling Auto Mask as your working, are there key stroke shortcuts that permit selection of Brush A, Brush B, and Auto Mask On vs. Auto Mask Off? As we now enjoy with Option to engage Erasure, it would be great to have a key stroke to select brush A, B and a Auto Mask toggle.

John-

Hi John

Use "A" to toggle the auto mask and use "/" to toggle between the A and B brush.

I certainly agree with the OP.  I understand Jeff's explanation of the way it is, but we are talking about what we want.  Basically the issue comes down to wanting further improvements to control the mask in localized editing.  Here is an example where I could like to see improvement"

(http://members.shaw.ca/hillrg1/temp/automask.jpg)

I tried using Jeff's reverse automask to do a better job of masking around the tree line but either I am doing something wrong or the technique does not work in this instance.

Regards
Rory
Title: Re: Problems With Auto-Mask
Post by: Slobodan Blagojevic on January 30, 2011, 01:58:46 pm
The only way (at least that I know of) to deal with that white line is to use a one-pixel clone brush and spend countless hours cloning it away.
Title: Re: Problems With Auto-Mask
Post by: Schewe on January 30, 2011, 02:21:00 pm
I tried using Jeff's reverse automask to do a better job of masking around the tree line but either I am doing something wrong or the technique does not work in this instance.

What you are trying to do is difficult even in Photoshop where you have complete control of the way the edges can be selected. I would say that the easier way of darkening the sky would be to darken the luminance of the blues in HSL–but I suspect you'll still get some fringing which is the problem of that kind of adjustment. Can Auto Mask be improved? You bet...but I don't think it'll ever replace making an accurate selection and the use of layer blending in Photoshop...
Title: Re: Problems With Auto-Mask
Post by: Rory on January 30, 2011, 08:42:29 pm
Well, I thought I would try the competition as I recalled that NIK viveza control points work pretty well at automasking, but I was wrong.  They do not work any better in this example.  So, I guess Jeff's advice about PS is the way to go (not that I doubted Jeff!).

(http://members.shaw.ca/hillrg1/temp/automask2.jpg)

If the smart lightroom engineers could add a brush that would clean up halos and fringes like this as you paint along that would be pretty cool.

Rory
Title: Re: Problems With Auto-Mask
Post by: Schewe on January 30, 2011, 09:09:15 pm
If the smart lightroom engineers could add a brush that would clean up halos and fringes like this as you paint along that would be pretty cool.

Agreed...but now that you mention it, what are your settings in the Lightroom Lens Corrections under Defringe? If you set it to All Edges (or even Highlight Edges) does that help mitigate the halos?
Title: Re: Problems With Auto-Mask
Post by: Rory on January 30, 2011, 10:04:35 pm
Agreed...but now that you mention it, what are your settings in the Lightroom Lens Corrections under Defringe? If you set it to All Edges (or even Highlight Edges) does that help mitigate the halos?

No, I tried that.  It seems to help with the purple fringing you often see, but not with this type of harsh halo.
Title: Re: Problems With Auto-Mask
Post by: dchew on January 31, 2011, 08:24:10 am
Agreed...but now that you mention it, what are your settings in the Lightroom Lens Corrections under Defringe? If you set it to All Edges (or even Highlight Edges) does that help mitigate the halos?

That's a great point Jeff.  I seem to be able to push the color sliders a bit farther in B&W conversions now because I am using lens profiles and paying much closer attention to fringing before converting.

On the one hand I keep wishing for more features in LR like an adjustable micro/gaussian/feather blur for Automask, while on the other hand not wanting LR to balloon into another PS...

I need to be careful what I wish for.

Dave
Title: Re: Problems With Auto-Mask
Post by: John R Smith on January 31, 2011, 10:22:10 am
While we're at it . . .

There are a couple of other little snags (perhaps bugs) which I have encountered in LR 3, which can catch you out. This is in B/W mind, I haven't tested what happens in colour.

* White objects against a darker sky (such as a lighthouse, flagpole, etc) interact unpleasantly with the detail slider. If you crank up the detail, you can get a large dark halo around the white object.

* There is some sort of bug in the highlight priority vignette. Eric has said that it is a known issue. What happens is that a local brush adjustment using EV also alters the overall vignette density, and the two interact in an unpredictable way.

Otherwise, of course, Lightroom is completely brilliant (and I'm not kidding - I tried using Phocus again the other night, which made me realise how brilliant LR really is).

John
Title: Re: Problems With Auto-Mask
Post by: Bill Carr on May 02, 2011, 01:44:22 am
Darn, I thought you were going to have a really simple question, one that many would benefit from: "The problem with Auto-mask is .... drumroll .... IT"S NOT PHOTOSHOP"

Maybe they could add a new menu item between Develop and Slideshow and call it, what the heck, Photoshop!

Oh, darn, then LR would be $898.   :o

Jeff, thanks for the helpful info.  Interesting about the starting center point.  I'll try the new eraser trick, too. 
Title: Re: Problems With Auto-Mask
Post by: Bill Carr on May 04, 2011, 02:26:33 am
It seems to me that the issue here may have to do with the fact that we are talking about parametric editing as opposed to pixel level editing.  PS actually changes the pixels along the edges and if it misses one, you can go in and actually change the pixel.  In LR, it writes a description of the change in the database, making it much more difficult to describe the color of each pixel along this border that, in this case, has a lot of detail in it.

Calls up the difference between running the car through the car wash and taking it in to get detailed.  Different products for different purposes.
Title: Re: Problems With Auto-Mask
Post by: BFoto on May 04, 2011, 03:57:05 pm
* White objects against a darker sky (such as a lighthouse, flagpole, etc) interact unpleasantly with the detail slider. If you crank up the detail, you can get a large dark halo around the white object.
John

Are you sure this is not the clarity bumped too high?
Title: Re: Problems With Auto-Mask
Post by: John R Smith on May 05, 2011, 03:08:44 am
Are you sure this is not the clarity bumped too high?


Essentially, yes, in this circumstance the acceptable level available from the detail slider is reduced to the point where you can barely use it. I am working in B/W, not colour, so things may be different there, I don't know.

John
Title: Re: Problems With Auto-Mask
Post by: NikoJorj on May 06, 2011, 06:16:40 am
Essentially, yes, in this circumstance the acceptable level available from the detail slider is reduced to the point where you can barely use it.
From what I heard, that's the way it works : detail at 0 means no haloes, detail at 100 means deconvolution sharpening and haloes.
You can choose, at least.
Title: Re: Problems With Auto-Mask
Post by: John R Smith on May 06, 2011, 09:12:51 am
Sorry, chaps. I have led you all completely astray. I meant to type the "clarity" slider, not detail, which of course is in the sharpening department. And then I go and perpetuate my mistake twice over.

I can only plead advancing years, and the early onset of dribbling senility. Probably exacerbated by a lifetime of substance abuse, just to top things off . . .

John
Title: Re: Problems With Auto-Mask
Post by: NikoJorj on May 06, 2011, 03:50:05 pm
Okay, then, direct from our "No Free Lunch" dept... yes, the clarity slider does produce haloes when pushed too far, and yes sometimes too far isn't far enough.