Amazon Shut Down Recruiting AI for Sexist Results

Donna Miller
October 13, 2018

The company had been building the programme since 2014 as automation has been key to Amazon's e-commerce dominance for both, inside the warehouses and the driving pricing decisions.

"Everyone wanted this holy grail", said one of the insiders quoted in the Reuters story. However, by 2015, it was realised that the system was not rating applicants in a gender-neutral way. Amazon's AI was fed data or resumes submitted to the company over a 10-year period and most of these resumes had unsurprisingly come from men.

Engineers found the AI was unfavourable towards female candidates because it had combed through male-dominated resumes to accrue its data.

"The technology thought, "Oh, Amazon doesn't like any resume that has the word "women's" in it - captain of a women's chess club, women's soccer team, some all-women's universities", said Reuters correspondent Jeffrey Dastin". According to Reuters, it also downgraded graduates of two women's colleges. They did not specify the names of the schools. The engineers told Reuters the idea was scrapped a year ago once it was determined the gender bias could not be corrected. Having machines and AI play some role in the recruitment process would seemingly ease the pressure on Amazon's human work force. The company did not elaborate further.

According to the report, the recruiters thankfully didn't take the AI's recommendations as gospel, but we suspect a few potentially great female workers slipped through the net thanks to the AIs sexism.

The company's experiment, which Reuters is first to report, offers a case study in the limitations of machine learning.

49ers apologize for leaving Kaepernick photos out of gallery
Kaepernick led the California team to playoff victories over Green Bay after the 2013 and 2014 seasons. Kaepernick also threw for over 400 yards during a regular-season game against the Packers.

Around 55 percent of USA human resources managers said artificial intelligence, or AI, would be a regular part of their work within the next five years, according to a 2017 survey by talent software firm CareerBuilder.

The AI was trained to use a method known as machine learning. To do this, the team created 500 computer models and taught each of them to recognize 50,000 terms from previous applicants' resumes. The algorithms learned to assign little significance to skills that were common across IT applicants, such as the ability to write various computer codes, the people said.

Stevie Buckley, the co-founder of United Kingdom job website Honest Work, which is used by companies such as Snapchat to recruit for technology roles, said that "the basic premise of expecting a machine to identify strong job applicants based on historic hiring practices at your company is a surefire method to rapidly scale inherent bias and discriminatory recruitment practices".

The model used in the AI system had other problems that led to unqualified candidates being recommended for a variety of unsuitable jobs.

The project was only ever used in a trial phase, not independently, and Amazon closed it in early 2017. (GS.N) that are looking to automate portions of the hiring process. His firm analyzes candidates' speech and facial expressions in video interviews to reduce reliance on resumes. There was just one problem: It overwhelmingly spit back men. His company's customers include Unilever PLC and Hilton.

Other reports by

Discuss This Article

FOLLOW OUR NEWSPAPER