KAIST Develops Model Training Technology to Correct AI 'Bias'

As the social domains affected by artificial intelligence expand, various bias issues in AI have recently come to the forefront, increasing the need for machine learning training that takes these biases into account.

For example, the COMPAS system, a machine learning-based model used to predict recidivism among criminals, exhibited bias by assigning different recidivism probabilities based on race.

Additionally, bias problems in AI have emerged in socially important areas such as recruitment and loan systems, highlighting the necessity for machine learning training that considers "fairness."

In this regard, KAIST announced on the 30th that a research team led by Professor Hwang Euijong from the Department of Electrical Engineering at KAIST has developed a new AI model training technique that enables unbiased judgments even on test data with new distributions different from the training conditions.

(From left) Yooji Noh, PhD candidate, Department of Electrical Engineering, KAIST, and Professor Eui-Jong Hwang, Department of Electrical Engineering, KAIST. Provided by KAIST

Researchers worldwide have previously proposed various training methodologies to enhance AI fairness. Most of these proposals share the assumption that the data used to train AI models and the data used in actual test situations have the same distribution.

However, in real-world scenarios, this assumption generally does not hold, revealing limitations where bias is observed between training data and test data across various applications.

If the bias pattern between the true labels of data and specific group information changes in the test environment, the fairness of an AI model that was fairly trained in advance can be compromised, resulting in worsened bias.

For instance, if an organization that previously hired mainly a specific race now hires regardless of race, an AI recruitment model fairly trained on past data could be judged as unfair when applied to current data.

This context underscores the significance of Professor Hwang’s research team’s findings.

First, to address the problem, the research team introduced the concept of "correlation shifts" and theoretically analyzed the fundamental limitations of existing fairness training algorithms in terms of accuracy and fairness performance.

For example, the team noted that in past data from companies that primarily hired a specific race, the correlation between race and hiring was strong, making it difficult for AI to accurately and fairly predict hiring under the current weaker correlation, even if the model is trained fairly.

Based on this theoretical analysis, they proposed a new training data sampling technique and a new training framework that allows the model to be trained fairly even when the bias pattern of the data changes during testing.

This structure reduces the proportion of data from the previously dominant race, thereby lowering the correlation with hiring.

The main advantage of the proposed technique is that it only requires data preprocessing, allowing existing algorithm-based training methods (for fairness) to be improved without modification.

The key point is that when existing training algorithms are vulnerable to correlation shifts, the problem can be resolved by applying the technique proposed by the research team in parallel.

Yuji Noh, a first author and doctoral student in the Department of Electrical Engineering at KAIST, said, "We expect our proposal to play a role in making AI models more trustworthy and enabling fairer judgments in real-world AI application environments."

Meanwhile, the research team’s findings were also presented at the International Conference on Machine Learning (ICML), the world’s leading conference on machine learning, held in Hawaii, USA, last July.

The research involved doctoral student Yuji Noh as the first author, Professor Hwang Euijong of KAIST as the corresponding author, and Professors Seo Changho of KAIST and Lee Kangwook of the University of Wisconsin?Madison as co-authors.