Overcoming AI 'Limits'? Teaching 'Flexibility' Modeled on the Human Brain

Professor Sangwan Lee's KAIST Team Develops Meta Reinforcement Learning Model to Resolve Overfitting-Underfitting Trade-off Problem

[Asia Economy Reporter Kim Bong-su] Artificial intelligence (AI) has recently evolved rapidly and is being utilized for various functions and problem-solving. However, its critical weakness lies in its inability to respond flexibly to changing situations. A domestic research team has attracted attention by developing technology that can overcome AI's limitations by mimicking the functions and characteristics of the human brain.

KAIST announced on the 5th that a research team led by Professor Lee Sang-wan of the Department of Bio and Brain Engineering (Director of the Neuroscience AI Convergence Research Center) succeeded in elucidating the principle to solve one of the challenging problems in AI?the overfitting-underfitting tradeoff?using brain-based AI technology. The research results were published online on May 28 in 'Cell Reports,' an open-access journal of the international academic journal Cell.

Recently, AI models provide optimal solutions to various real-world problems, but they still face difficulties in responding flexibly to changing situations. In machine learning, this is referred to as the underfitting-overfitting risk or bias-variance tradeoff, and although it has been studied for a long time, no clear solution has yet been proposed for situations where conflicting conditions continuously change, as in the real world.

On the other hand, humans focus on the current given problem (solving the underfitting issue) while not excessively fixating on the immediate problem (solving the overfitting issue), and respond flexibly according to changing situations. The research team established a theoretical framework on how the human brain solves this problem using brain data, probabilistic process inference models, and reinforcement learning algorithms, from which they derived a flexible meta-reinforcement learning model.

The human brain solves this problem using a single piece of information called the 'prediction error lower bound,' processed in the midbrain dopamine circuit and the prefrontal cortex. Our prefrontal cortex, especially the dorsolateral prefrontal cortex, estimates the limit of the expected performance of the current problem-solving method (e.g., "If I solve it this way, I can get up to 90 points"). It then minimizes the risk of underfitting and overfitting by flexibly selecting the optimal problem-solving strategy according to changing situations (e.g., "If I solve it this way, I can only get about 70 points, so let me try a different approach").

The research team previously discovered in 2014 that this prefrontal cortex area is involved in flexibly adjusting reinforcement learning strategies based on environmental uncertainty and published a paper on this. In 2015, they also found that it is involved in causal inference processes. In 2019, they revealed that this brain area can even consider the complexity of the problem.

A meta-reinforcement learning model that mimics the human flexible problem-solving approach. Image provided by KAIST

The research team also reported that this series of research results provide evidence of humans' metacognitive ability to self-assess their learning and reasoning capabilities. Based on this ability, they established the 'Prefrontal Cortex Meta-Learning Theory,' which suggests that AI can solve various conflicting situations in the real world that are difficult to solve otherwise.

This study is evaluated as the first case to actually solve the long-standing underfitting-overfitting tradeoff problem in AI based on this theory. Using the meta-reinforcement learning model developed through the research, it is possible to indirectly measure human flexible problem-solving ability through simple games. Furthermore, when applied to smart education or cognitive behavioral therapy related to addiction, it is expected to enhance humans' problem-solving abilities to respond flexibly to changing situations. This foundational technology, which has significant ripple effects in various fields such as next-generation AI, smart education, and cognitive behavioral therapy, has recently completed patent applications domestically and internationally.

Dr. Kim Dong-jae, who led the research, said, "This is one of the studies that show how important it is to understand the unique strengths of human intelligence."

Professor Lee Sang-wan, the principal investigator, explained, "There are many problems that AI solves better than us, but conversely, there are many problems that feel really easy to us but are difficult for AI to solve. Through research formalizing various high-level human abilities from the perspective of AI theory, we expect to unravel the secrets of human intelligence one by one. This brain-based AI research can be seen as an engineering exploration of human intelligence, and it will establish a clear benchmark where humans and AI can help each other and grow together."