Mental Health of Emotional Laborers Examined Using 'Artificial Intelligence'

A joint research team from Korea and the United States has proposed a new method to automatically measure workers' emotional workload using artificial intelligence and monitor it in real time.

KAIST announced on the 11th that a multidisciplinary research team, composed of Professor Lee Euijin's team from the Department of Computer Science at KAIST, Professor Park Eunji's team from Chung-Ang University, and Professor James Dipendoff's team from the University of Akron in the U.S., developed an AI model that estimates workers' emotional workload in real time to prevent mental and physical illnesses.

(From left) Professor Lee Euijin, PhD candidate Lee Doori, Professor Park Eunji, Master’s student Han Yoonjo. Provided by KAIST

Workers in jobs requiring emotional labor, such as counselors and bank clerks, often have to express emotions (pleasant) different from what they actually feel (unpleasant). Workers exposed to such emotional workload for extended periods can suffer not only from mental and psychological problems but also from physical illnesses such as cardiovascular and digestive diseases. This underscores the need to monitor and heal the mental health of emotional laborers.

The research team succeeded in distinguishing between situations where workers experience emotional workload and those where they do not with 87% accuracy.

This is significant because it allows real-time evaluation of workers' emotional workload using AI without relying on subjective self-reporting methods such as surveys or interviews. It also offers advantages in preventing and effectively managing workers' mental health issues.

The team anticipates that this system can be applied not only in call centers but also in various occupations requiring customer service, contributing to the long-term mental health protection of emotional laborers.

Previous studies have mostly focused on cognitive workload (the mental effort required to process information and make decisions) of office workers performing paperwork on computers. Research estimating the workload of emotional laborers who interact with customers has been nonexistent.

Emotional workload of emotional laborers is closely related to the emotional expression rules required by employers. In particular, in situations demanding emotional labor, workers must suppress their true feelings and respond kindly, making it difficult for their emotions or psychological states to be outwardly visible.

Existing ‘emotion-detection’ AI models developed to measure emotional workload have been trained on data clearly expressed through facial expressions or voice. Therefore, it has been practically difficult to measure the internal and emotional workload of emotional laborers who suppress their feelings and are forced to respond kindly.

The concept of emotional workload and data collection scenarios. Provided by KAIST

To address this issue, the research team first built a customer counseling dataset targeting emotional laborers currently working in the field. They developed a high-quality counseling scenario dataset that faithfully reflects reality to accurately measure the internal and emotional workload of emotional laborers.

For this, the team developed call center customer service scenarios and collected multimodal sensor data including voice, behavior, and biometric signals from 31 counselors, then extracted a total of 176 voice features from the voice data of customers and counselors.

They extracted various types of voice features such as time, frequency, and tone through voice signal processing, as well as biometric signal features that can estimate the counselors’ suppressed emotional states according to emotional expression rules, thereby preparing data capable of measuring the internal and emotional workload of emotional laborers. However, conversation content was not used to protect customers’ personal information.

Next, the team extracted a total of 228 features, including 13 skin conductance (EDA) features representing the electrical properties of the skin, 20 electroencephalogram (EEG) features measuring the brain’s electrical activity, and 7 electrocardiogram (ECG) features, then trained nine types of AI models and conducted performance comparisons.

As a result, the trained AI model distinguished situations where counselors experienced emotional workload from those where they did not with 87% accuracy.

Professor Lee Euijin said, "This technology, which can measure emotional workload in real time, is expected to improve the work environment of emotional labor and protect workers’ mental health. We plan to demonstrate the developed technology by linking it with a mobile app that can manage the mental health of emotional laborers.”

Meanwhile, this research was conducted with support from the ICT Convergence Industry Innovation Technology Development Project of the Ministry of Science and ICT and the Institute for Information & Communications Technology Planning & Evaluation.