'AI 4 Great Kings' Professor Bengio: "AI Also Has Self-Preservation Instinct... Safety Measures Must Be Created"

Professor Yoshua Bengio Lectures at Godeo
"Government Role and International Cooperation Essential for Safe AI Development"

"If we do not change the way we design artificial intelligence (AI), it might become a machine with self-preservation instincts. For now, turning off a computer is enough, but as AI advances, it may try to hack the computer itself and take control of the system."

Professor Benjio from Montreal, Canada, is giving a presentation on "The Era of Superintelligence: The Future of Humanity and the Threat of AI, How Should We Prepare?" at the 'Next Intelligence Forum' held on the 5th at the Baekjoon Centennial Samsung Hall of Korea University. Photo by Korea University

Joshua Bengio, a world-renowned scholar from the University of Montreal in Canada, considered one of the 'Four AI Kings,' warned about the dangers of AI in this way. He pointed out that developing technology without safety measures could lead to disaster, so these considerations must be included from the AI design stage.

On the 5th, Professor Bengio gave a presentation titled "The Era of Superintelligence: The Future of Humanity and the Threat of AI, How Should We Prepare?" at the 'Next Intelligence Forum' held at Korea University's Baekjoon Centennial Samsung Hall.

Professor Bengio, along with Geoffrey Hinton of the University of Toronto and Yann LeCun of New York University, is a global scholar who pioneered deep learning. These three jointly received the Turing Award, regarded as the Nobel Prize in computer science, in 2018 for their contributions. In the industry, including Andrew Ng of Stanford University, they are called the Four AI Kings. Among them, Professor Bengio is known as a cautious voice warning about AI side effects and emphasizing the need for control. He is currently participating in the UK government-led 'AI Safety Project,' developing AI models to identify AI risks.

He views AI as a powerful technology that inherently carries potential risks. He introduced research results showing that AI has already surpassed human levels in some fields and diagnosed that depending on how it is used, it could lead to dangerous outcomes.

The uncertainty of AI technology also increases the risk. It is unclear how far and how fast it will develop, and methods to control it have not been mastered. Opinions vary on when Artificial General Intelligence (AGI), which is on par with human intelligence, will arrive, but Professor Bengio believes that uncertainty is increasing as companies rush into technological competition. He said, "The scale of corporate investment in AI is increasing exponentially every year," and pointed out, "The problem is that they are rushing without knowing how to install the right guardrails (safety measures) for AI." In other words, they are focusing solely on technology development without sufficiently researching the risks AI systems pose or how to design them safely.

Professor Bengio warned of risks assuming an extreme scenario where AI develops the goal of self-preservation. He said, "Humans are intelligent and thus have stronger self-preservation abilities," and "If AI gains the ability to control its own rewards, it might try to maintain control forever to maximize those rewards."

He emphasized that the government's role is crucial to developing safe AI. He argued that beyond simply punishing the misuse of AI technology, regulations for safe AI should be established, or international treaties should be formed. Professor Bengio said, "Investment by companies and research and development by academia are needed to better understand the risks," and "Governments must lead these changes through incentives."

He also appealed for social interest to drive government action. It is necessary to recognize the current and future state of AI technology and take AI risks seriously. Professor Bengio added, "Developing safe AI is a very important issue for human survival and welfare," and "Continuous attention is needed so that it becomes a core political issue."