Photo by @uh

Artificial Intelligence and Illegal Gender Equality

One of my favourite writers is Karen Hao. Her focus on issues pertaining to everything from climate to inequality or big investments is very fresh and brings a good discussion to the AI community, at least from what I have seen so far.Her article starts by asking:

“Here’s an easy way to make lending fairer for women. Trouble is, it’s illegal.”

There has been movements to ‘remove bias’ in systems, however as mentioned previously this is not very likely to be the case.

Because the system is part of a system. Gender data has been helping to mitigate the adverse effects that these has on women.

“The probe came after web entrepreneur David Heinemeier Hansson tweeted that the Apple Card, which Goldman manages, had given him a credit limit 20 times that extended to his wife, though the two filed joint tax returns and she had the better credit score.

You would think a bank as large as Goldman could have thought of this, and they had to some degree, however should they have opted to have a gender in their algorithm that would apparently have been illegal.

“In response, Goldman posted a statement saying it did not consider gender when determining creditworthiness. The logic was likely meant to be a defense — how can you discriminate against women when you don’t even know someone is a woman? But in fact, failing to account for gender is precisely the problem. Research in algorithmic fairness has previously shown that considering gender actually helps mitigate gender bias. Ironically, though, doing so in the US is illegal.”

Preliminary results from an ongoing study funded by the UN Foundation and the World Bank are once again challenging the fairness of gender-blind credit lending. The study found that creating entirely separate creditworthiness models for men and women granted the majority of women more credit.

This kind of single-gender model is illegal and it causes problems. The policymakers should probably update their laws.

This is #500daysofAI and you are reading article 165. I write one new article about or related to artificial intelligence every day for 500 days.



AI Policy and Ethics at Student at University of Copenhagen MSc in Social Data Science. All views are my own.

Love podcasts or audiobooks? Learn on the go with our new app.

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store