Are Algorithms Sexist? – The New York Times

Sign up here to get In Her Words delivered to your inbox. Let us know what you think at [email protected].

“We cannot bow down to the algorithms.”

— Jamie Heinemeier Hansson, the woman who inexplicably received a lower credit limit than her husband.

When a prominent software developer, David Heinemeier Hansson, accused the Apple Card of being a “sexist program” last week, he set off a tweetstorm.

He detailed (with more expletives than I’ll include here) that he had been given a credit line 20 times higher than his wife’s despite the fact that her credit score was better than his.

Others jumped in to share similar experiences with the Apple Card, including, embarrassingly, the Apple co-founder Steve Wozniak, who said his credit limit was 10 times the one his wife received.

New York State regulators opened an investigation. Apple blamed Goldman Sachs, its banking partner for the credit card. And Goldman blamed an algorithm.

At the root of it all might be what the New York University professor Meredith Broussard calls “technochauvinism” — the flawed assumption that technology is always the superior, more effective solution to any problem, like determining credit worthiness.

“Automated systems discriminate by default,” said Broussard, author of “Artificial Unintelligence: How Computers Misunderstand the World.” They are built using data from historically unequal systems and therefore spit out the same skewed outputs. Or as Senator Elizabeth Warren put it in an interview Thursday, “algorithms are only as good as the data that gets packed into them.”

In the case of Apple Card, although it’s not yet clear if a sexist algorithm was at fault (all we have is anecdotal evidence), technology, so often heralded as a great equalizer, seems to have just perpetuated unequal access to the banking system.

Until 1975, single, divorced or widowed women in the U.S. needed a man to co-sign their credit card applications. In some cases, women had to present a doctor’s note proving that they were using contraceptives before they could qualify for a mortgage loan (the logic was that if they didn’t have babies, they wouldn’t quit their jobs). And it wasn’t until 2013 that the Consumer Financial Protection Bureau urged banks and credit card issuers to consider shared incomes in order to make it easier for stay-at-home spouses to qualify for cards.

“We should just be really thoughtful about when we do and don’t use technology,” said Broussard. And more inclusive development teams would also help pick up on and alleviate the kind of blind spots that landed Apple in this situation in the first place, she added.

Apple raised the credit limit for Hansson’s wife, Jamie, after his tweet went viral.

“I felt the weight and guilt of my ridiculous privilege,” she wrote in a powerfully worded statement. “Justice for another rich white woman is not justice at all.”

Have you noticed designs or products that seem to exclude certain groups? Tell me about it by writing in or on Twitter.

What else is happening

Here are five articles from The Times you might have missed.

• “My case wasn’t really about a nonexistent abortion, but it was the result of the arbitrary politics of the state.” Hajar Raissouni was wrongly accused of having an abortion — a crime in Morocco — sparking a heated national discussion over the country’s antiquated laws. [Read the story]

• “If women wanted to be appreciated for their brains, they’d go to the library instead of to Bloomingdale’s.” Michael Bloomberg has long been known in New York City circles for making crude, sexist remarks that are likely to come under scrutiny again as he considers running for president. [Read the story]

• “Feminism, homosexuality and atheism.” Saudi Arabia’s official government Twitter account listed those three as forms of “extremism.” And then quickly took down the post. [Read the story]

• “The higher tax rate on these products amounts to fiscal discrimination of women.” Germany reclassified menstrual products from being a “luxury” item to a “necessary” one, which is taxed at a lower rate. [Read the story]

• “I don’t think there’s any evidence that it reduces harassment and there’s some evidence that it’s likely to increase harassment.” A nationwide awakening to workplace mistreatment has fostered a market for sexual harassment training videos — even though their effectiveness at changing office culture is questionable. [Read the story]

Sign up here to get future installments of In Her Words delivered to your inbox.

ICYMI: The E.R.A. Is Coming