• Amazon tried building an artificial-intelligence tool to help with recruiting, but it showed a bias against women, Reuters reports.
  • Engineers reportedly found the AI was unfavorable toward female candidates because it had combed through male-dominated résumés to accrue its data.
  • Amazon reportedly abandoned the project at the beginning of 2017.

Amazon worked on building an artificial-intelligence tool to help with hiring, but the plans backfired when the company discovered the system discriminated against women, Reuters reports.

Citing five sources, Reuters said Amazon set up an engineering team in Edinburgh, Scotland, in 2014 to find a way to automate its recruitment.

The company created 500 computer models to trawl through past candidates’ résumés and pick up on about 50,000 key terms. The system would crawl the web to recommend candidates.

“They literally wanted it to be an engine where I’m going to give you 100 résumés, it will spit out the top five, and we’ll hire those,” one source told Reuters.

A year later, however, the engineers reportedly noticed something troubling about their engine - it didn't like women. This was apparently because the AI combed through predominantly male résumés submitted to Amazon over a 10-year period to accrue data about whom to hire.

Consequently, the AI concluded that men were preferable. It reportedly downgraded résumés containing the words "women's" and filtered out candidates who had attended two women-only colleges.

Amazon's engineers apparently tweaked the system to remedy these particular forms of bias but couldn't be sure the AI wouldn't find new ways to unfairly discriminate against candidates.

Gender bias was not the only problem, Reuters' sources said. The computer programs also spat out candidates who were unqualified for the position.

Remedying algorithmic bias is a thorny issue, as algorithms can pick up on subconscious human bias. In 2016, ProPublica found that risk-assessment software used to forecast which criminals were most likely to reoffend exhibited racial bias against black people. Overreliance on AI for things like recruitment, credit-scoring, and parole judgments have also created issues in the past.

Amazon reportedly abandoned the AI recruitment project by the beginning of last year after executives lost faith in it. Reuters' sources said Amazon recruiters looked at recommendations generated by the AI but never relied solely on its judgment.

Amazon told Business Insider it was committed to workplace diversity and equality.  "This was never used by Amazon recruiters to evaluate candidates."