Is AI a Genius or a Fool... The Answer So Far Is 'This' [Tech Talk]

Basic Cognitive Abilities of Latest AI
Less Than One-Third of Human Level
Advanced Cognitive Skills 'Surpass' Humans

Artificial intelligence (AI) solves math problems that even the greatest geniuses struggle with, predicts protein structures, and makes complex decisions. On the other hand, AI sometimes makes basic arithmetic errors or mistakes an orange traffic light for the sun.

So, what is the intellectual capacity of the latest generation AI? Is it a superintelligence that surpasses humans, or is it simply foolish? Interestingly, recent research suggests that it could be "both."

Human Brain vs. Latest AI: Competing in Spatial Cognition

[Image source=Reuters Yonhap News]

The latest large models developed aiming for Artificial General Intelligence (AGI) are called 'Frontier AI.' On the 9th of last month (local time), Apple collaborated with several AI scientists to conduct an intriguing experiment measuring the 'cognitive abilities' of these Frontier AI models. The AI models used in the experiment included GPT-4o, Llama 3 70B, Mistral 123B, among the latest generation models at this time.

Cognitive ability is one of the key indicators to gauge the intellectual capacity of machine intelligence. Animals, including humans, possess basic cognitive abilities such as navigation and spatial awareness, as well as abstract cognitive abilities like logical reasoning and decision-making. The latter is considered a trait of more 'advanced' animals.

The researchers created a proprietary cognitive ability test called 'SPACE.' They designed various missions such as maze navigation for AI, spatial recognition, object recognition, memory tests, and shortest path finding to measure performance differences compared to the control group of human brains.

As the test progressed, the missions became increasingly abstract and complex. Later tasks included predicting the direction water would flow when a container is tilted, requiring the AI to provide answers based on predictions.

Human Intelligence Dominates... Except for 'High-Level Attention'

The innate lack of spatial awareness in artificial intelligence (AI) is also a reason why developing autonomous driving is challenging.

What were the results? In most tests, human cognitive abilities overwhelmingly outperformed AI. Especially in simple tasks like navigation or object recognition, AI found it much more difficult. Some basic test scores were only about one-third of human levels, making them almost unusable.

However, in one area of the SPACE test, AI surpassed humans: the 'selective attention' task performed solely with text. Selective attention is a characteristic of the human brain, the ability to focus only on immediately relevant information while ignoring others. It tests how well one can concentrate while excluding distracting elements in the environment.

Selective attention requires a higher level of concentration and patience than simple cognitive abilities. Moreover, for humans, performing such tasks with text rather than images is even more challenging. Yet, while AI struggled with simpler tasks from a human perspective, it achieved excellent results in this high-level cognitive domain.

From a Human Perspective, AI’s Mental State is Close to a 'Disability'

The intelligence of AI cannot be interpreted from a human perspective.

The SPACE test results highlight that AI and human 'intelligence' are fundamentally different. The authors of the report conducting the test stated, "Our research shows that advanced AI intelligence differs from human or other animal intelligence," explaining that "biological brains cannot develop higher cognitive abilities without basic cognitive abilities."

In other words, humans and AI may possess opposite characteristics. Functions requiring high mental effort and concentration for humans may be natural for AI. The authors also pointed out that "in biological cases, a lack of basic cognitive abilities is a symptom of disability."

In short, current AI is (by human standards) in a state close to intellectual disability. It completely lacks essential abilities for living but possesses extraordinary concentration in highly abstracted specific areas that surpass ordinary people. Therefore, describing AI solving math Olympiad problems or deriving complex equation theorems as "AI surpassing human intelligence" is inappropriate.

The Difference Between Humans Evolved in Nature and AI Growing Through Data Learning?

Ancestors of humans, Australopithecus. Humans and other animals acquire intelligence by receiving and analyzing external stimuli such as vision, hearing, and smell.
[Image source=Yonhap News]

This test also shows how far apart animals and machines are. Animal intelligence, including humans, is the result of evolution optimized for survival. We have prioritized abilities to respond to various external stimuli such as vision, hearing, and smell, and to perceive space and distance, which led to the cognitive abilities we have today.

In contrast, AI, which is just beginning to emerge, has grown by ingesting 'data,' especially text materials through large language models (LLM) from the start. This means the development paths and environments for AI and humans are completely different.

This may explain why AI shows dazzling achievements in some fields while still unable to move beyond beginner levels in others. Particularly, difficulties in tasks requiring 'response to external stimuli,' such as autonomous driving, may stem from this reason.