AWS' new tool is designed to mitigate AI bias

SageMaker Clarify can discover potential bias during data preparation and after training, says AWS

AWS has released SageMaker Clarify, a new tool designed to reduce bias in machine learning (ML) models.

Announcing the tool at AWS re:Invent 2020, Swami Sivasubramanian, VP of Amazon AI, said that Clarify will provide developers with greater visibility into their training data, to mitigate bias and explain predictions.

Amazon AWS ML scientist Dr. Nashlie Sephus, who specialises in issues of bias in ML, explained the software to delegates.

Biases are imbalances or disparities in the accuracy of predictions across different groups, such as age, gender, or income bracket. A wide variety of biases can enter a model due to the nature of the data and the background of the data scientists. Bias can also emerge depending on how scientists interpret the data through the model they build, leading to, e.g. racial stereotypes being extended to algorithms.

For example, facial recognition systems have been found to be quite accurate at recognising white faces, but show much less accuracy when identifying people of colour.

According to AWS, SageMaker Clarify can discover potential bias during data preparation, after training, and in a deployed model by analysing attributes specified by the user.

SageMaker Clarify will work within SageMaker Studio - AWS's web-based development environment for ML - to detect bias across the machine learning workflow, enabling developers to build fairness into their ML models. It will also help developers to increase transparency by explaining the behaviour of an AI model to customers and stakeholders. The issue of so-called 'black box' AI has been a perennial one, and governments and companies are only just now starting to address it.

SageMaker Clarify will also integrate with other SageMaker capabilities like SageMaker Experiments, SageMaker Data Wrangler, and SageMaker Model Monitor.

SageMaker Clarify is available in all regions where Amazon SageMaker is available. The tool will come free for all current users of Amazon SageMaker.

During AWS re:Invent 2020, Sivasubramanian also announced many other new SageMaker capabilities, including SageMaker Data Wrangler; SageMaker Feature Store, SageMaker Pipelines, SageMaker Debugger, Distributed Training on Amazon SageMaker, SageMaker Edge Manager, and SageMaker JumpStart.

An industry-wide challenge

The launch of SageMaker Clarify has come at the time when an intense debate is ongoing about AI ethics and the role of bias in machine learning models.

Just last week, Google was at the centre of the debate as former Google AI researcher Timnit Gebru claimed that the company abruptly terminated her for sending an internal email that accused Google of "silencing marginalised voices".

Recently, Gebru had been working on a paper that examined threats posed by computer systems that can analyse human language databases and use them to create their own human-like text. The paper argues that such systems will over-rely on data from rich countries, where people have better access to internet facilities, and so be inherently biased. It also mentions Google's own technology, which Google is using in its search business.

Gebru says she submitted the paper for internal review on 7th October, but it was rejected the next day.

Thousands of Google employees, academics and civil society supporters have now signed an open letter demanding the company to show transparency and to explain the process by which Dr Gebru's paper was unilaterally rejected.

The letter also criticises the company for racism and defensiveness.

Google is far from the only tech giant to face criticism of its use of AI. AWS itself was subject to condemnation two years ago, when it came out that an AI tool it had built to help with recruitment was biased against women.