“We found that facial analysis services performed consistently worse on transgender individuals, and were universally unable to classify non-binary genders,” said study lead author Morgan Klaus Scheuerman from the University of Colorado Boulder.
“While there are many different types of people out there, these systems have an extremely limited view of what gender looks like,” Scheuerman added.
Previous research suggests they tend to be most accurate when assessing the gender of white men but misidentify women of colour as much as one-third of the time.
“We knew there were inherent biases in these systems around race and ethnicity and we suspected there would also be problems around gender,” said senior author Jed Brubaker.
“We set out to test this in the real world,” Brubaker said.
For the findings, researchers collected 2,450 images of faces from Instagram, each of which had been labeled by its owner with a hashtag indicating their gender identity.
The pictures were then divided into seven groups of 350 images (#women, #man, #transwoman, #transman, #agender, #agenderqueer, #nonbinary) and analyzed by four of the largest providers of facial analysis services (IBM, Amazon, Microsoft, and Clarifai).
Notably, Google was not included because it does not offer gender recognition services.
On average, the systems were most accurate with photos of cisgender women (those born females and identifying as female), getting their gender right 98.3 percent of the time. They categorized cisgender men accurately 97.6 percent of the time.
But trans men were wrongly identified as women up to 38 percent of the time.
And those who identified as agender, genderqueer or nonbinary – indicating that they identify as neither male or female – were mischaracterized 100 percent of the time.
The study also suggests that such services identify gender based on outdated stereotypes.
Also read: Make Sure To Upgrade Your IPhone 5 To IOS 10.3.4 By Nov 3, Else You Might Suffer
When researcher Scheuerman, who is male and has long hair, submitted his own picture, half categorized him as female.
“These systems run the risk of reinforcing stereotypes of what you should look like if you want to be recognized as a man or a woman. And that impacts everyone,” said Scheuerman.
The authors said that they would like to see tech companies move away from gender classification entirely and stick to more specific labels like ‘long hair’ or ‘make-up’ when assessing images.
The research is scheduled to be presented in November at the ACM Conference on Computer Supported Cooperative Work in the US.
Prasanta Patnaik