Luminous Landscape Forum
Equipment & Techniques => Mirrorless Cameras => Topic started by: Martin Kristiansen on July 21, 2019, 02:38:35 am
-
I have picked up a issue with face and eye detect on both the Sony A9 and the A7riii.
I resisted auto focus for years. Most of my photography the last while has been tripod bound studio, landscape and architectural work. Much more recently I began doing more sport, street, documentary, fashion and so on. I am even doing a few weddings and some events again, something I haven’t done for years. Obviously AF is a real advantage and so I ended up with the gear I now have. AF is not as simple as I hoped and getting the right camera with the right lens and the correct settings is crucial to its success.
But I have found an odd problem.
In South Africa we have a big variety of skin colours. From west African populations that are extremely dark to blond blue eyed whites of Dutch decent. In commercial photography advertisers try to be as representative as possible and rather than going for mixed race people, we call coloureds over here, I frequently have to contend with a shot that has a white man in a navy shirt next to a black west African woman in a white shirt. Thank goodness for modern high dynamic range sensors. Challenging but so it goes. We have techniques.
The problem is focusing. The cameras will almost always focus on the white face. If there are 5 people in frame and one is white the white person will be critically focused if I shoot an event I end up with all the white people in focus even though we are a minority. In my own family the one black woman is frequently the person that turns up with far fewer images where she is sharply focused.
There are ways around it but it takes work and concentration and an awareness of the problem. I am not for a moment saying it’s a racist thing on the part of camera manufacturers. I think it’s the same issue that causes backlit faces to be harder for the AF to deal with. White faces have more contrast and simply reflect more light so the AF detects them faster and locks on. Black faces being darker are not selected by the AF system if you are using wide or zone focusing. If you allow the camera to choose the face with face detect is consistently chooses the lighter face.
I find I cannot trust zone focus area with face detect or wide with face detect. The AF will always pick the white faces in this mode. It’s better to select a large spot and move it over the face you want sharp and force the AF to select that face. You need to be aware that the AF is likely to quickly abandoned that dark face and hop to a white face if one should move close the the auto focus zone you have selected. It makes it harder in situations where as long as one face is sharp you are good to go, now you need to deliberately decide on every shot which face that should be. When shooting an event for example I need a balance of white and black faces that are sharp. Not just the white faces.
Perhaps no issue in places where most people are white but for those of us in countries where that is not the reality it might be worth taking note of this characteristic of face and eye detect AF systems.
Likely not of much interest to most.
-
Like everything in photography, it's a matter of understanding one's equipment and getting on with it.
Peter
-
Like everything in photography, it's a matter of understanding one's equipment and getting on with it.
Peter
Quite right. And sometimes it throws up an issue you may not have considered.
-
Martin, I can't say I'm surprised about this issue and will be looking out for it when using my Z7 in similar circumstances.
That said I rarely shoot white faces.
-
The cameras will almost always focus on the white face ... I am not for a moment saying it’s a racist thing on the part of camera manufacturers.
Well, as you describe it, the behaviour of the camera is racist.
Is this because the camera manufacturer is racist? I suspect it is.
-
Well, as you describe it, the behaviour of the camera is racist.
Is this because the camera manufacturer is racist? I suspect it is.
As I described it and I thought made quite clear the camera finds it easier to focus on something brighter and with more contrast. This behavior is easily seen without face and eye detect. We all know that as light levels fall off the AF performance also falls off. A west African has less light coming off his face than a white person under the same lighting conditions, that is a fact not a racist statement. It makes sense for the AF system to work with the brighter more contrasty parts of a scene because that will boost performance. I believe the unintended consequence of this is darker skinned people are less likely to be selected as the target for AF systems featuring face and eye detect.
I don’t believe this issue has a racial bias at its foundation. That could be debated in the Coffee Corner but I am not going to get involved. I am bringing up these AF issues in case it will help someone working under similar circumstances to my own. I make adjustments for black skin when setting up studio lights. I see this as a similar practical issue that might need to be addressed.
-
As I described it and I thought made quite clear the camera finds it easier to focus on something brighter and with more contrast. This behavior is easily seen without face and eye detect. We all know that as light levels fall off the AF performance also falls off. A west African has less light coming off his face than a white person under the same lighting conditions, that is a fact not a racist statement. It makes sense for the AF system to work with the brighter more contrasty parts of a scene because that will boost performance. I believe the unintended consequence of this is darker skinned people are less likely to be selected as the target for AF systems featuring face and eye detect.
I don’t believe this issue has a racial bias at its foundation. That could be debated in the Coffee Corner but I am not going to get involved. I am bringing up these AF issues in case it will help someone working under similar circumstances to my own. I make adjustments for black skin when setting up studio lights. I see this as a similar practical issue that might need to be addressed.
As you say, it's probably an illumination problem. Weddings push that problem for those who shoot them too, at least in traditional white dress/dark suit situations.
It's enough to drive people crazy. That said, I guess there will always be the situation where going back to manual makes more sense! Or perhaps using only central focussing point. I've never tried anything else... I still think life is governed by split-image central areas.
:-)
-
Absolutely Rob. Always a way around the problem. As long as you recognize that there is in fact a problem. I suppose that my point in a way.
-
Well, as you describe it, the behaviour of the camera is racist.
Is this because the camera manufacturer is racist? I suspect it is.
This post has been reported to me as being "entirely inappropriate". I don't agree. I think it is too fatuously silly to be elevated to the level of inappropriate. It contributes nothing to the discussion save for importing the buzzword "racist", which of course places the user on an unassailable pedestal and the accused irredeemably and unarguably in the wrong. Rod Liddle, a political commentator in the UK, generally spells with word with multiple "a"s, aptly mimicking the usual pronunciation of those who resort to it.
The post does merit a warning, though. Elliot, if you can contribute nothing but abuse, you will be silenced.
Jeremy
-
I only have a first generation A7. My wife, her family and many of her friends are Asian.
I deliberately avoid using face detect (the A7 doesn't have eye detect) when I think this is going to be an issue for exactly this reason but even when placing the focus point on the face I want to focus on I've had a number of slightly out of focus shots especially when the subject is relatively small in the frame. Shots with the subjects face bigger in the frame are usually much less of an issue in my experience.
I suppose in time focus systems and algorithms will improve and the problem will lessen or even hopefully go away altogether.
To be honest I'd long since twigged what the issue was but hadn't mentioned it to anyone and didn't know that it could still be an issue with the newer cameras. I hope it gets sorted soon and in the meantime in situations when the focus can't be relied upon to lock onto the desired face I'll target the clothing and let dof do the rest or if time allows use manual focus.
-
Well, as you describe it, the behaviour of the camera is racist.
Is this because the camera manufacturer is racist? I suspect it is.
-
There is a kind of corollary to this problem involving paper-media use of photographs with a mix of races, especially with newsprint. Black faces require a heavier application of ink than white faces. Newsprint is absorbent, and black and even brown faces often were reproduced as featureless blobs, even when the original photograph distinctly showed the features. This was a particularly terrible problem in high school basketball, often played in somewhat dimly lit gymnasiums (dimly lit by photography standards, but perfectly okay for the game and spectators.) The solution most papers went for was to deliberately over-expose; you might then be able to pull up black features, at the cost of white faces going excessively contrasty, to the point where the nose usually disappeared and you were left with hair, eyebrows and lips, which was better than losing the black faces altogether.
-
There is a kind of corollary to this problem involving paper-media use of photographs with a mix of races, especially with newsprint. Black faces require a heavier application of ink than white faces. Newsprint is absorbent, and black and even brown faces often were reproduced as featureless blobs, even when the original photograph distinctly showed the features. This was a particularly terrible problem in high school basketball, often played in somewhat dimly lit gymnasiums (dimly lit by photography standards, but perfectly okay for the game and spectators.) The solution most papers went for was to deliberately over-expose; you might then be able to pull up black features, at the cost of white faces going excessively contrasty, to the point where the nose usually disappeared and you were left with hair, eyebrows and lips, which was better than losing the black faces altogether.
In the old days as it were I actually wrote a custom dot gain curve for newsprint to help with this issue.
I wasn’t aware this issue would arise with Asian faces actually. I don’t think the issue is with facial features and the differences between races. I believe it to be a simple issue of contrast and light levels. The Sony system seems to err on the side of finding faces that don’t exist and on occasion will think some other pattern is a face when it is not. Also the system shows the same tendencies to ignore with skinned Indians and they have very similar features to Caucasians. I did a shoot two weeks ago with a Chinese woman and a Japanese woman but both very light skinned and the camera handled it well. The black man who was backlit the camera didn’t pick up at all even when he was the only person in frame.
-
I only have a first generation A7. My wife, her family and many of her friends are Asian.
I deliberately avoid using face detect (the A7 doesn't have eye detect) when I think this is going to be an issue for exactly this reason but even when placing the focus point on the face I want to focus on I've had a number of slightly out of focus shots especially when the subject is relatively small in the frame. Shots with the subjects face bigger in the frame are usually much less of an issue in my experience.
I suppose in time focus systems and algorithms will improve and the problem will lessen or even hopefully go away altogether.
To be honest I'd long since twigged what the issue was but hadn't mentioned it to anyone and didn't know that it could still be an issue with the newer cameras. I hope it gets sorted soon and in the meantime in situations when the focus can't be relied upon to lock onto the desired face I'll target the clothing and let dof do the rest or if time allows use manual focus.
The problem with focus existed, differently, with my 500 Series cameras using studio flash modelling lights that were too low to let me see well enough on faces (in full-length shots) and the answer was to have the girl place her hand flat against her stomach so the highlight on the rings could be used as the focussing area. Fortunately, large junk jewellery was often used! Shooting at around f6.3 or f8 was perfectly good enough to render the face sharp. The camera was tripod mounted, of course.
-
I'll be interested to see how I fare with the Z7 face/eye detect when I next go to India where skin tones have a wide range.
-
I'll be interested to see how I fare with the Z7 face/eye detect when I next go to India where skin tones have a wide range.
It should be easy to do the control test by toggling off the eye detect feature and comparing similar shots to see what the difference, if any, is. I did the firmware update on my Z 6 before we headed off on vacation to Canada last month. 90% of my images were landscapes so I didn't have this issue. I had some shots of street musicians and even one of Leonard Cohen in Montreal (the face/eye detect worked perfectly on that one!!! :D)
-
I'll be interested to see how I fare with the Z7 ...
/* slightly off-topic /*
Keith,
have you tried using any Leica M-series lenses on the Z series, and if you have was there an improvement in the smearing issue that affects the thicker cover glass Sony's ?
M
-
/* slightly off-topic /*
Keith,
have you tried using any Leica M-series lenses on the Z series, and if you have was there an improvement in the smearing issue that affects the thicker cover glass Sony's ?
M
M, not yet.
I don't think there's much point in trying with anything wider than 50mm although I do have plans to test my 50 Zeiss C Sonnar on the Z. I wouldn't expect any M fit lens to perform as well on the Z but in any case that's not what I'm looking for. It might also be fun to try the 7 Artisans 50 f/1.1 but it's not a lens I own.
Best.
-
I don't think there's much point in trying with anything wider than 50mm although I do have plans to test my 50 Zeiss C Sonnar on the Z. I wouldn't expect any M fit lens to perform as well on the Z ....
It's the 50 (and the75's) I'm curious about! It could be a motive for switching bodies, but for the time being it looks as though the only option is still a Kolarvision modified Sony.
Thanks for the feedback.
All best,
M
-
It should be easy to do the control test by toggling off the eye detect feature and comparing similar shots to see what the difference, if any, is. I did the firmware update on my Z 6 before we headed off on vacation to Canada last month. 90% of my images were landscapes so I didn't have this issue. I had some shots of street musicians and even one of Leonard Cohen in Montreal (the face/eye detect worked perfectly on that one!!! :D)
Cohen might as well have been another test wall. Everything looks in focus on my little iPad, from here to eternity - hardly much of a test for af.
-
I don’t think the issue is with facial features and the differences between races. I believe it to be a simple issue of contrast and light levels.
Another possibility: my understanding is that the face- and eye-detection capabilities of recent autofocus systems rely on features in the virtual image that were identified through by neural networks—"artificial intelligence" software—and thus would be influenced by the training set of photographs used by the manufacturer.
-
Another possibility: my understanding is that the face- and eye-detection capabilities of recent autofocus systems rely on features in the virtual image that were identified through by neural networks—"artificial intelligence" software—and thus would be influenced by the training set of photographs used by the manufacturer.
Yes I am aware of that. But with this issue it doesn’t seem to be that it doesn’t recognize the features. Thats why I brought up my experience with the very dark Indian woman where it also seemed to show a preference for a lighter skinned person. While the woman was dark her features were quite European for want of a better word and the camera rather chose to focus on her friend who was a light skinned Japanese person.
The Face detect is also not at all put off by glasses and even sun glasses. The actual recognition of the patterns that make up a face seem to be very robust.
-
Are we going to have a rehash of the Shirley debate?
-
Are we going to have a rehash of the Shirley debate?
I sincerely hope not. I thought about that before beginning the thread and almost bailed on the whole idea. I have family that are various types of not white. I live in a country that is almost defined by race issues. I really do get all that stuff. I just don’t think that is what is behind this particular characteristic of the AF system. It is an issue for me because if I am unaware of it and don’t address it someone will eventually ask me why I only focus on the whiteys. Being a whitey myself that could be a problem for me.
That’s all besides the fact that I don’t want to only focus on the white people. Like that wedding photographer from a while ago that got himself in a spot of bother for producing lots of pictures of cleavage, bums and legs of the pretty girls at the wedding. Unprofessional.
-
Like that wedding photographer from a while ago that got himself in a spot of bother for producing lots of pictures of cleavage, bums and legs of the pretty girls at the wedding. Unprofessional.
I fell partly into that trap when I did a colleague's wedding a few years ago. Happily, I wasn't the only photographer and I wasn't even trying to be professional.
I did get some rather nice shots, though ;)
Jeremy
-
Another possibility: my understanding is that the face- and eye-detection . . . would be influenced by the training set of photographs used by the manufacturer.
Yes I am aware of that. But with this issue it doesn’t seem to be that it doesn’t recognize the features. Thats why I brought up my experience with the very dark Indian woman where it also seemed to show a preference for a lighter skinned person.
I've never built any neural network software myself—and my grasp of the underlying technology is admittedly rudimentary—but my understanding is that even the designers of a particular recognizer are often unable to determine what aspects of the training set (in this case, presumably photographs that included tagged images of people's faces) the software uses to perform the recognition. The "features" used by the recognizer need not be meaningful or even identifiable to human viewers of the same training set. This apparently is especially true when so-called "deep-learning"—neural networks with many layers—is involved.
So if your training set includes a large number of Europeans and East Asians and relatively fewer Africans, West Asians, Mestizos, etc., it wouldn't surprise me if the resulting recognizer would "show a preference for a lighter skinned person." In fact it would surprise me if it didn't. That doesn't mean the tone of the skin is primarily what the neural network is keying on. It certainly may be a factor—probably is—but there may be other characteristics, not meaningful from the perspective of a human observer, that are more influential in causing the behavior you have observed.
I don't claim to have an answer; I'm just offering a conjecture. But it would be interesting to know how diverse a population was used to create the Sony training set, as well as that of other manufacturers that offer facial recognition. One thing about which I am fairly confident is that the more diversified the training set, the more effective the facial recognition.
-
Interesting Chris. Strange as it may sound, I have never built a neural network. I will put it on my to do list. Won’t be tomorrow though, I’m on a shoot.
Seriously though. That is all above my pay grade. I suppose what you say is entirely possible. I have no idea or evidence one way or another. I simply observed this behavior and figured I need to find a way to deal with it. I put it out there in case others also need to deal with it. Also perhaps someone smarter than me has a work around that I am currently unaware of.
-
perhaps someone smarter than me has a work around that I am currently unaware of.
I wouldn't be optimistic about a technical workaround. If your conjecture is correct, namely
White faces have more contrast and simply reflect more light so the AF detects them faster and locks on. Black faces being darker are not selected by the AF system if you are using wide or zone focusing.
then probably the only way to defeat this behavior would be the one you have already identified: select the face of the person you want to focus on manually.
If my conjecture is correct,
So if your training set includes a large number of Europeans and East Asians and relatively fewer Africans, West Asians, Mestizos, etc., it wouldn't surprise me if the resulting recognizer would "show a preference for a lighter skinned person." In fact it would surprise me if it didn't.
resolving the problem would require persuading Sony (or whatever company supplies this software component to Sony) to enlarge the diversity of its neural network training set. While Sony lately seems to trying to gain traction in the professional photography market, unless you have a fairly senior contact in the company, I suspect getting them to make such a change would be a difficult thing to do.
-
I've never built any neural network software myself
It's easier than you might think, Chris.
(http://imgs.xkcd.com/comics/trained_a_neural_net.png)
Jeremy
-
O dear, the company manufacturing my camera with its built-in light meter is racist. When photographing black people against a black background and white people against a white background my images are completely wasted, totally useless. Only the people with skin colors and backgrounds in-between come out decently.
The film companies are also to blame. With their non-linear film curves they discriminate (discrimination!) blacks and whites differently.
Of course this is utterly B.S. I'm sure you'll agree. So are the previously made statements on racism w.r.t. different behavior on eye/face detect. Get a life >:(
Regards,
Jaap.
-
O dear, the company manufacturing my camera with its built-in light meter is racist. When photographing black people against a black background and white people against a white background my images are completely wasted, totally useless. Only the people with skin colors and backgrounds in-between come out decently.
The film companies are also to blame. With their non-linear film curves they discriminate (discrimination!) blacks and whites differently.
Of course this is utterly B.S. I'm sure you'll agree. So are the previously made statements on racism w.r.t. different behavior on eye/face detect. Get a life >:(
Regards,
Jaap.
Jaap,
Agree!
This was basically was what I said ...A photographer learns their gear and gets on with it.
Peter
-
Jaap,
Agree!
This was basically was what I said ...A photographer learns their gear and gets on with it.
Peter
Yes I agree. I was simply trying to bring peoples attention to a quirk of the equipment that some of us might need to be aware of. It was actually an effort to be helpful. This is quite new tech and new issues will come up. I was kind of hoping that a forum such as this could be a place to help others and to learn from others.
Usually it just turn into a pissing contest.
-
Are we going to have a rehash of the Shirley debate?
Clearly not! My allusion to it nearly got me banned. (Thank you Chris Kern for at least entertaining the idea.)
-
Thank you Chris Kern for at least entertaining the idea.
Ahhh, you're welcome, I guess. But I don't have a clue what you are referring to.
-
Ahhh, you're welcome, I guess. But I don't have a clue what you are referring to.
Thank goodness for that: me neither.
Better not let Slobodan find out, or before you know it you'll find yourself swimming out of that mainstream too.
:-)
-
... Better not let Slobodan find out...
No need to find out, he knows already. This is Shirley. “Shirley debate” is just another idiotic lefties’ attempt to accuse Kodak of racism and turn everyone and everything into racist.
-
This is Shirley.
Aha! Got it now. That little dust-up completely passed me by. I just read the posts in the Shirley thread and I presume it's clear to everyone reading this thread that there is no correspondence between the way an analog emulsion responds to brightness and contrast, and the way a neural network recognizes features in a large collection of digital or digitized photographs that have been used to train a camera autofocus subsystem—features which in the case of the latter may not be apparent to human beings, including the programmers who built the software.
My purpose was simply to share a little of what I've learned during the past few months about neural networks because it seemed to be relevant to the issue raised by the original poster. As I pointed out, I don't claim to be an expert, and my grasp of the technology is no doubt considerably less sophisticated than that of some of the other participants on this site whose software development skills are more current than mine. But I suspect I'm somewhat more familiar with how it works than the majority of LuLa posters.
In addition to reading about their theory and design, I've pulled and examined some available free source code to understand how others have implemented neural networks—although, as I noted in an earlier post in this thread, I have not attempted to build any of my own software. I've also been experimenting with various hosted and online tools for using neural networks to make semantic identifications of images; sharpen, enlarge, and otherwise enhance pictures; and even transform photographs into something that is no longer photographic but which doesn't qualify as freehand art: a collaboration between the photographer and the software, you might call it. (Some samples here (https://www.flickr.com/photos/chriskernpix/albums/72157705322226491).)
The point I was trying to make is that the way a neural network will act on its "training set" of data, while obviously influenced by the parameters chosen by the developers to mediate the network's behavior (there isn't much is this world that is more perfectly deterministic than software), often involves extracting features than are not visible to or, if visible, at least not identifiable by human beings, and that those features will be strongly if not always predictably influenced by the scope and scale of the image set.
Now that I've read that earlier thread, I understand why some readers here might have conflated neural network recognition with the use of a "Shirley card" in analog photography, but that's a mistake. If my explanation isn't sufficiently clear, there is plenty of explanatory information about the operation of neural networks available online, as well as some interesting tools (free and commercially licensed) and sample source code.
This machine-learning technique is important to all of us who make photographs, I think, because modern cameras aren't optical machines which instantaneously record light on a sensitive medium; they are dynamic software systems that are attached to optical and light-sensing input devices. Post-processing is also being transformed by this technology: look no further than the ability of Lightroom's Enhance Details to sharpen raw camera sensor data or the the ability of the Topaz suite to take demosaiced images and in effect build new images (bigger, sharper, etc.) based on them.
Interesting stuff. I'm not sure where it will all wind up. Hopefully not here (https://forum.luminous-landscape.com/index.php?topic=129748.msg1102057#msg1102057).
-
Yes I agree. I was simply trying to bring peoples attention to a quirk of the equipment that some of us might need to be aware of. It was actually an effort to be helpful. This is quite new tech and new issues will come up. I was kind of hoping that a forum such as this could be a place to help others and to learn from others.
Usually it just turn into a pissing contest.
Thank you, Martin, for pointing this out, and for the “heads up” to those of us who have not yet been in the position to notice it. Sorry you’ve had to had to endure some of the nonsense that’s found it’s way into this thread.
-
Interesting Chris. Strange as it may sound, I have never built a neural network. I will put it on my to do list. Won’t be tomorrow though, I’m on a shoot.
Seriously though. That is all above my pay grade. I suppose what you say is entirely possible. I have no idea or evidence one way or another. I simply observed this behavior and figured I need to find a way to deal with it. I put it out there in case others also need to deal with it. Also perhaps someone smarter than me has a work around that I am currently unaware of.
Well, here's a long shot: have you tried a b&w creative style (or picture style)? If, by some fluke chance, the face detect is applied after the creative style, it may evaluate differently.
I agree with Chris that it is most likely a NN training bias, not on race as you already quite obviously noted, but on contrast and potentially color complexity. They likely harvest face images from the usual sources, which simply have a disproportionate contribution of dark faces and dark skincolor for the purposes of training a NN. (For obvious reasons, most of those face images will likely not be in the shade, to name just one issue).
That said: the mk4 is said to be a significant improvement, face detecting everything from amphibians, to lizards, to monkeys, to people, and even president Trump. There: two evolutions in a single sentence! One of them may actually be an improvement. I hope for our sake, the mk4 still sucks with dark complexities.
-
Well, here's a long shot: have you tried a b&w creative style (or picture style)? If, by some fluke chance, the face detect is applied after the creative style, it may evaluate differently.
I agree with Chris that it is most likely a NN training bias, not on race as you already quite obviously noted, but on contrast and potentially color complexity. They likely harvest face images from the usual sources, which simply have a disproportionate contribution of dark faces and dark skincolor for the purposes of training a NN. (For obvious reasons, most of those face images will likely not be in the shade, to name just one issue).
That said: the mk4 is said to be a significant improvement, face detecting everything from amphibians, to lizards, to monkeys, to people, and even president Trump. There: two evolutions in a single sentence! One of them may actually be an improvement. I hope for our sake, the mk4 still sucks with dark complexities.
That an interesting idea. I have a shoot on Saturday with a group of people completing their yoga teacher training. I will set up a little trial and see what happens. They are mostly a white group, about 60% I would guess, but enough other races to be able to do some tests.
Up to now I have noticed this behavior we are discussing but haven’t had an opportunity to set up an actual experiment. Perhaps I will get a chance on Saturday.