1400 Cases of 'Fake Pornographic' Deepfakes... Kim Sang-hee Calls for Technology Regulation

Chairperson Kim Sang-hee of the Science and ICT Committee
"If AI technology causes serious harm to humans,
the government must intervene directly"

Kim Sang-hee, Deputy Speaker of the National Assembly, Science, Technology, Information and Broadcasting and Communications Committee

[Asia Economy Reporter Cha Min-young] From January to March this year, Mr. A, who received production requests from buyers via messenger, was arrested for producing and selling illegal synthetic materials of about 70 celebrities and ordinary people using deepfake technology. Deepfake is a new type of crime that abuses artificial intelligence (AI) technology to create fake pornographic videos for trade or to blackmail victims.

Although the "Special Act on the Punishment of Sexual Violence Crimes," which prohibits the production and distribution of deepfake sexual false videos, was established last year, it has been confirmed that more than 1,400 deepfake crimes have occurred this year alone. Since practical means to stop the technology are limited, there are calls for improvements in laws and systems related to intelligent information technology.

According to data submitted by the Korea Communications Standards Commission to Kim Sang-hee, Deputy Speaker of the National Assembly's Science, Technology, Information and Broadcasting Committee, the number of blocked and deleted "deepfake sexual false videos" increased more than 2.5 times from 548 cases between June 25 and December last year to 1,408 cases from January to September 24 this year. The monthly average also rose 1.7 times from about 91 cases to 156 cases. Although the Sexual Violence Crime Punishment Act, which prohibits the production and distribution of deepfake sexual false videos, has been in effect since June 25 last year, the number of crimes has continued to increase.

Examining the deepfake crime details submitted by the police to Deputy Speaker Kim, it was revealed that photos and personal information of victims, including ordinary people and celebrities, were obtained via social networking services (SNS) and chat messengers to produce and distribute deepfake sexual false videos for sale. Since AI creates videos without a sense of incongruity, if strict control is not enforced, these videos could spread through the dark web and potentially become a "second N-bunbang incident."

Therefore, there are calls for stronger technological regulations to prevent crimes abusing intelligent information technology, including AI. According to Articles 31 and 60 of the current "Basic Act on Intelligent Informatization," if intelligent information technology threatens human life or physical safety, it can be restricted, and if necessary, the state can suspend the technology in an emergency. However, so far, there have been no cases where AI or other intelligent information technologies have caused urgent harm to human life or body, so this law has never been applied.

Deputy Speaker Kim Sang-hee stated, "Current laws do not have provisions regarding serious mental or financial damage caused by abusing AI," and added, "To prevent crimes abusing AI and to stop significant harm, there should be measures to restrict or suspend the use of technology in cases of serious abuse of intelligent information technology."

She continued, "AI development is essential for entering the future industrial society, but if it causes serious harm to humans, the government must step in directly," emphasizing, "We will define AI crimes as new types of crimes and prepare measures to prevent illegal acts and abuse of AI so that no citizens suffer from AI crimes anymore."