Pages: 1 [2]   Go Down

Author Topic: Issues with face and eye detect.  (Read 3456 times)

Chris Kern

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 2034
    • Chris Kern's Eponymous Website
Re: Issues with face and eye detect.
« Reply #20 on: July 22, 2019, 01:20:58 pm »

I don’t think the issue is with facial features and the differences between races. I believe it to be a simple issue of contrast and light levels.

Another possibility: my understanding is that the face- and eye-detection capabilities of recent autofocus systems rely on features in the virtual image that were identified through by neural networks—"artificial intelligence" software—and thus would be influenced by the training set of photographs used by the manufacturer.

Martin Kristiansen

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 1527
    • Martin Kristiansen
Re: Issues with face and eye detect.
« Reply #21 on: July 22, 2019, 01:32:57 pm »

Another possibility: my understanding is that the face- and eye-detection capabilities of recent autofocus systems rely on features in the virtual image that were identified through by neural networks—"artificial intelligence" software—and thus would be influenced by the training set of photographs used by the manufacturer.

Yes I am aware of that. But with this issue it doesn’t seem to be that it doesn’t recognize the features. Thats why I brought up my experience with the very dark Indian woman where it also seemed to show a preference for a lighter skinned person. While the woman was dark her features were quite European for want of a better word and the camera rather chose to focus on her friend who was a light skinned Japanese person.

The Face detect is also not at all put off by glasses and even sun glasses. The actual recognition of the patterns that make up a face seem to be very robust.
Logged
Commercial photography is 10% inspiration and 90% moving furniture around.

faberryman

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 4851
Re: Issues with face and eye detect.
« Reply #22 on: July 22, 2019, 01:43:21 pm »

Are we going to have a rehash of the Shirley debate?
Logged

Martin Kristiansen

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 1527
    • Martin Kristiansen
Re: Issues with face and eye detect.
« Reply #23 on: July 22, 2019, 02:29:28 pm »

Are we going to have a rehash of the Shirley debate?

I sincerely hope not. I thought about that before beginning the thread and almost bailed on the whole idea. I have family that are various types of not white. I live in a country that is almost defined by race issues. I really do get all that stuff. I just don’t think that is what is behind this particular characteristic of the AF system. It is an issue for me because if I am unaware of it and don’t address it someone will eventually ask me why I only focus on the whiteys. Being a whitey myself that could be a problem for me.

That’s all besides the fact that I don’t want to only focus on the white people. Like that wedding photographer from a while ago that got himself in a spot of bother for producing lots of pictures of cleavage, bums and legs of the pretty girls at the wedding. Unprofessional.
Logged
Commercial photography is 10% inspiration and 90% moving furniture around.

Jeremy Roussak

  • Administrator
  • Sr. Member
  • *****
  • Offline Offline
  • Posts: 8961
    • site
Re: Issues with face and eye detect.
« Reply #24 on: July 22, 2019, 03:01:38 pm »

Like that wedding photographer from a while ago that got himself in a spot of bother for producing lots of pictures of cleavage, bums and legs of the pretty girls at the wedding. Unprofessional.

I fell partly into that trap when I did a colleague's wedding a few years ago. Happily, I wasn't the only photographer and I wasn't even trying to be professional.

I did get some rather nice shots, though  ;)

Jeremy
Logged

Chris Kern

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 2034
    • Chris Kern's Eponymous Website
Re: Issues with face and eye detect.
« Reply #25 on: July 22, 2019, 04:52:04 pm »

Another possibility: my understanding is that the face- and eye-detection . . . would be influenced by the training set of photographs used by the manufacturer.

Yes I am aware of that. But with this issue it doesn’t seem to be that it doesn’t recognize the features. Thats why I brought up my experience with the very dark Indian woman where it also seemed to show a preference for a lighter skinned person.

I've never built any neural network software myself—and my grasp of the underlying technology is admittedly rudimentary—but my understanding is that even the designers of a particular recognizer are often unable to determine what aspects of the training set (in this case, presumably photographs that included tagged images of people's faces) the software uses to perform the recognition.  The "features" used by the recognizer need not be meaningful or even identifiable to human viewers of the same training set.  This apparently is especially true when so-called "deep-learning"—neural networks with many layers—is involved.

So if your training set includes a large number of Europeans and East Asians and relatively fewer Africans, West Asians, Mestizos, etc., it wouldn't surprise me if the resulting recognizer would "show a preference for a lighter skinned person."  In fact it would surprise me if it didn't.  That doesn't mean the tone of the skin is primarily what the neural network is keying on.  It certainly may be a factor—probably is—but there may be other characteristics, not meaningful from the perspective of a human observer, that are more influential in causing the behavior you have observed.

I don't claim to have an answer; I'm just offering a conjecture.  But it would be interesting to know how diverse a population was used to create the Sony training set, as well as that of other manufacturers that offer facial recognition.  One thing about which I am fairly confident is that the more diversified the training set, the more effective the facial recognition.

Martin Kristiansen

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 1527
    • Martin Kristiansen
Re: Issues with face and eye detect.
« Reply #26 on: July 22, 2019, 05:39:21 pm »

Interesting Chris. Strange as it may sound, I have never built a neural network. I will put it on my to do list. Won’t be tomorrow though, I’m on a shoot.

Seriously though. That is all above my pay grade. I suppose what you say is entirely possible. I have no idea or evidence one way or another. I simply observed this behavior and figured I need to find a way to deal with it. I put it out there in case others also need to deal with it. Also perhaps someone smarter than me has a work around that I am currently unaware of.
Logged
Commercial photography is 10% inspiration and 90% moving furniture around.

Chris Kern

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 2034
    • Chris Kern's Eponymous Website
Re: Issues with face and eye detect.
« Reply #27 on: July 22, 2019, 06:37:30 pm »

perhaps someone smarter than me has a work around that I am currently unaware of.

I wouldn't be optimistic about a technical workaround.  If your conjecture is correct, namely

White faces have more contrast and simply reflect more light so the AF detects them faster and locks on. Black faces being darker are not selected by the AF system if you are using wide or zone focusing.

then probably the only way to defeat this behavior would be the one you have already identified: select the face of the person you want to focus on manually.

If my conjecture is correct,

So if your training set includes a large number of Europeans and East Asians and relatively fewer Africans, West Asians, Mestizos, etc., it wouldn't surprise me if the resulting recognizer would "show a preference for a lighter skinned person."  In fact it would surprise me if it didn't.

resolving the problem would require persuading Sony (or whatever company supplies this software component to Sony) to enlarge the diversity of its neural network training set.  While Sony lately seems to trying to gain traction in the professional photography market, unless you have a fairly senior contact in the company, I suspect getting them to make such a change would be a difficult thing to do.

Jeremy Roussak

  • Administrator
  • Sr. Member
  • *****
  • Offline Offline
  • Posts: 8961
    • site
Re: Issues with face and eye detect.
« Reply #28 on: July 23, 2019, 03:35:56 am »

I've never built any neural network software myself

It's easier than you might think, Chris.



Jeremy
Logged

JaapD

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 303
Re: Issues with face and eye detect.
« Reply #29 on: July 23, 2019, 05:44:25 am »

O dear, the company manufacturing my camera with its built-in light meter is racist. When photographing black people against a black background and white people against a white background my images are completely wasted, totally useless. Only the people with skin colors and backgrounds in-between come out decently.

The film companies are also to blame. With their non-linear film curves they discriminate (discrimination!) blacks and whites differently.

Of course this is utterly B.S. I'm sure you'll agree. So are the previously made statements on racism w.r.t. different behavior on eye/face detect. Get a life  >:(

Regards,
Jaap.


Logged

petermfiore

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 2705
    • Peter Fiore Fine Art
Re: Issues with face and eye detect.
« Reply #30 on: July 23, 2019, 08:43:53 am »

O dear, the company manufacturing my camera with its built-in light meter is racist. When photographing black people against a black background and white people against a white background my images are completely wasted, totally useless. Only the people with skin colors and backgrounds in-between come out decently.

The film companies are also to blame. With their non-linear film curves they discriminate (discrimination!) blacks and whites differently.

Of course this is utterly B.S. I'm sure you'll agree. So are the previously made statements on racism w.r.t. different behavior on eye/face detect. Get a life  >:(

Regards,
Jaap.

Jaap,
Agree!

This was basically was what I said ...A photographer learns their gear and gets on with it.

Peter

Martin Kristiansen

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 1527
    • Martin Kristiansen
Re: Issues with face and eye detect.
« Reply #31 on: July 23, 2019, 12:16:20 pm »

Jaap,
Agree!

This was basically was what I said ...A photographer learns their gear and gets on with it.

Peter

Yes I agree. I was simply trying to bring peoples attention to a quirk of the equipment that some of us might need to be aware of. It was actually an effort to be helpful. This is quite new tech and new issues will come up. I was kind of hoping that a forum such as this could be a place to help others and to learn from others.

Usually it just turn into a pissing contest.
Logged
Commercial photography is 10% inspiration and 90% moving furniture around.

elliot_n

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 1219
Re: Issues with face and eye detect.
« Reply #32 on: July 23, 2019, 05:17:52 pm »

Are we going to have a rehash of the Shirley debate?

Clearly not! My allusion to it nearly got me banned. (Thank you Chris Kern for at least entertaining the idea.)


Logged

Chris Kern

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 2034
    • Chris Kern's Eponymous Website
Re: Issues with face and eye detect.
« Reply #33 on: July 23, 2019, 05:19:36 pm »

Thank you Chris Kern for at least entertaining the idea.

Ahhh, you're welcome, I guess.  But I don't have a clue what you are referring to.

Rob C

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 24074
Re: Issues with face and eye detect.
« Reply #34 on: July 23, 2019, 05:37:52 pm »

Ahhh, you're welcome, I guess.  But I don't have a clue what you are referring to.


Thank goodness for that: me neither.

Better not let Slobodan find out, or before you know it you'll find yourself swimming out of that mainstream too.

:-)

Slobodan Blagojevic

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 18090
  • When everyone thinks the same, nobody thinks
    • My website
Re: Issues with face and eye detect.
« Reply #35 on: July 23, 2019, 06:38:17 pm »

... Better not let Slobodan find out...

No need to find out, he knows already. This is Shirley. “Shirley debate” is just another idiotic lefties’ attempt to accuse Kodak of racism and turn everyone and everything into racist.

Chris Kern

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 2034
    • Chris Kern's Eponymous Website
Re: Issues with face and eye detect.
« Reply #36 on: July 23, 2019, 08:24:14 pm »

This is Shirley.

Aha!  Got it now.  That little dust-up completely passed me by.  I just read the posts in the Shirley thread and I presume it's clear to everyone reading this thread that there is no correspondence between the way an analog emulsion responds to brightness and contrast, and the way a neural network recognizes features in a large collection of digital or digitized photographs that have been used to train a camera autofocus subsystem—features which in the case of the latter may not be apparent to human beings, including the programmers who built the software.

My purpose was simply to share a little of what I've learned during the past few months about neural networks because it seemed to be relevant to the issue raised by the original poster.  As I pointed out, I don't claim to be an expert, and my grasp of the technology is no doubt considerably less sophisticated than that of some of the other participants on this site whose software development skills are more current than mine.  But I suspect I'm somewhat more familiar with how it works than the majority of LuLa posters.

In addition to reading about their theory and design, I've pulled and examined some available free source code to understand how others have implemented neural networks—although, as I noted in an earlier post in this thread, I have not attempted to build any of my own software.  I've also been experimenting with various hosted and online tools for using neural networks to make semantic identifications of images; sharpen, enlarge, and otherwise enhance pictures; and even transform photographs into something that is no longer photographic but which doesn't qualify as freehand art: a collaboration between the photographer and the software, you might call it.  (Some samples here.)

The point I was trying to make is that the way a neural network will act on its "training set" of data, while obviously influenced by the parameters chosen by the developers to mediate the network's behavior (there isn't much is this world that is more perfectly deterministic than software), often involves extracting features than are not visible to or, if visible, at least not identifiable by human beings, and that those features will be strongly if not always predictably influenced by the scope and scale of the image set.

Now that I've read that earlier thread, I understand why some readers here might have conflated neural network recognition with the use of a "Shirley card" in analog photography, but that's a mistake.  If my explanation isn't sufficiently clear, there is plenty of explanatory information about the operation of neural networks available online, as well as some interesting tools (free and commercially licensed) and sample source code.

This machine-learning technique is important to all of us who make photographs, I think, because modern cameras aren't optical machines which instantaneously record light on a sensitive medium; they are dynamic software systems that are attached to optical and light-sensing input devices.  Post-processing is also being transformed by this technology: look no further than the ability of Lightroom's Enhance Details to sharpen raw camera sensor data or the the ability of the Topaz suite to take demosaiced images and in effect build new images (bigger, sharper, etc.) based on them.

Interesting stuff.  I'm not sure where it will all wind up.  Hopefully not here.
« Last Edit: July 23, 2019, 08:58:28 pm by Chris Kern »
Logged

D Fuller

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 608
    • AirStream Pictures
Re: Issues with face and eye detect.
« Reply #37 on: July 23, 2019, 09:10:13 pm »

Yes I agree. I was simply trying to bring peoples attention to a quirk of the equipment that some of us might need to be aware of. It was actually an effort to be helpful. This is quite new tech and new issues will come up. I was kind of hoping that a forum such as this could be a place to help others and to learn from others.

Usually it just turn into a pissing contest.

Thank you, Martin, for pointing this out, and for the “heads up” to those of us who have not yet been in the position to notice it. Sorry you’ve had to had to endure some of the nonsense that’s found it’s way into this thread.
Logged
business website: www.airstream.pictures
blog: thirtynineframes.com/blog

32BT

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 3095
    • Pictures
Re: Issues with face and eye detect.
« Reply #38 on: July 24, 2019, 01:35:38 am »

Interesting Chris. Strange as it may sound, I have never built a neural network. I will put it on my to do list. Won’t be tomorrow though, I’m on a shoot.

Seriously though. That is all above my pay grade. I suppose what you say is entirely possible. I have no idea or evidence one way or another. I simply observed this behavior and figured I need to find a way to deal with it. I put it out there in case others also need to deal with it. Also perhaps someone smarter than me has a work around that I am currently unaware of.

Well, here's a long shot: have you tried a b&w creative style (or picture style)? If, by some fluke chance, the face detect is applied after the creative style, it may evaluate differently.

I agree with Chris that it is most likely a NN training bias, not on race as you already quite obviously noted, but on contrast and potentially color complexity. They likely harvest face images from the usual sources, which simply have a disproportionate contribution of dark faces and dark skincolor for the purposes of training a NN. (For obvious reasons, most of those face images will likely not be in the shade, to name just one issue).

That said: the mk4 is said to be a significant improvement, face detecting everything from amphibians, to lizards, to monkeys, to people, and even president Trump. There: two evolutions in a single sentence! One of them may actually be an improvement. I hope for our sake, the mk4 still sucks with dark complexities.


Logged
Regards,
~ O ~
If you can stomach it: pictures

Martin Kristiansen

  • Sr. Member
  • ****
  • Offline Offline
  • Posts: 1527
    • Martin Kristiansen
Re: Issues with face and eye detect.
« Reply #39 on: July 24, 2019, 01:48:09 am »

Well, here's a long shot: have you tried a b&w creative style (or picture style)? If, by some fluke chance, the face detect is applied after the creative style, it may evaluate differently.

I agree with Chris that it is most likely a NN training bias, not on race as you already quite obviously noted, but on contrast and potentially color complexity. They likely harvest face images from the usual sources, which simply have a disproportionate contribution of dark faces and dark skincolor for the purposes of training a NN. (For obvious reasons, most of those face images will likely not be in the shade, to name just one issue).

That said: the mk4 is said to be a significant improvement, face detecting everything from amphibians, to lizards, to monkeys, to people, and even president Trump. There: two evolutions in a single sentence! One of them may actually be an improvement. I hope for our sake, the mk4 still sucks with dark complexities.

That an interesting idea. I have a shoot on Saturday with a group of people completing their yoga teacher training. I will set up a little trial and see what happens. They are mostly a white group, about 60% I would guess, but enough other races to be able to do some tests.

Up to now I have noticed this behavior we are discussing but haven’t had an opportunity to set up an actual experiment. Perhaps I will get a chance on Saturday.
Logged
Commercial photography is 10% inspiration and 90% moving furniture around.
Pages: 1 [2]   Go Up