A technology capable of detecting and identifying Korean-language comments generated by artificial intelligence (AI) has been developed in South Korea. Given growing concerns over online opinion manipulation using generative AI technologies, this new development is expected to contribute to preventing such manipulation in the future.
On June 23, KAIST announced that a research team led by Professor Yongdae Kim from the Department of Electrical Engineering, in collaboration with the National Security Research Institute (NSRI), has developed a technology called XDAC to detect AI-generated Korean comments.
(From left) Yongdae Kim, Professor, Department of Electrical Engineering, KAIST; Wooyoung Ko, Senior Researcher, National Security Research Institute (PhD candidate at KAIST). Courtesy of KAIST
According to the joint research team, recent advances in generative AI have made it possible to adjust not only the context but also the sentiment and tone of news article comments. In particular, the ability to automatically generate hundreds of thousands of comments within a few hours has significantly increased the potential for misuse in opinion manipulation.
For example, based on OpenAI's GPT-4o API, the cost to generate a single comment is about 1 KRW, meaning that generating the average daily number of comments (200,000) on major domestic news platforms would cost only 200,000 KRW. Moreover, by using open LLMs and in-house GPU infrastructure, it is effectively possible to generate massive numbers of comments at virtually no cost.
While the mass generation of comments using AI has become easy, distinguishing them from human-written comments remains difficult.
In an experiment involving 210 comments, the joint research team found that humans mistook 67% of AI-generated comments for human-written ones, and could accurately identify only 73% of actual human-written comments. This demonstrates the challenge of filtering out AI-generated comments.
AI-generated comments sometimes received higher evaluations than human-written comments in terms of contextual relevance to the article, sentence fluency, and bias awareness.
Efforts to develop technologies to detect such maliciously used AI-generated comments have continued. However, most existing technologies were developed based on long, standardized English texts, which limits their effectiveness when applied to the short, informal nature of Korean comments.
Short comments lack sufficient statistical features and often contain informal spoken expressions such as emojis, slang, and repeated characters, making it difficult for existing detection models to work effectively. Additionally, the lack of datasets for Korean AI-generated comments and the use of simple prompting methods have also made detection more challenging.
In response, the joint research team developed an AI comment generation framework using four strategies: leveraging 14 different LLMs, enhancing naturalness, fine-grained emotion control, and augmented generation using reference materials. They built a dataset of Korean AI-generated comments that mimic real user styles and released part of it as a benchmark dataset.
They also applied explainable AI (XAI) techniques to analyze linguistic expressions in detail, confirming that AI-generated comments have unique patterns (speech styles) that differ from those of humans.
For example, AI tended to use formal expressions such as "~geot gatda" (it seems that...) and "~e daehae" (regarding...), as well as a high frequency of conjunctions, while humans preferred informal spoken expressions such as repeated characters ("kkkkk"), emotional expressions, line breaks, and special symbols.
In terms of special character usage, AI mainly used globally standardized emojis, whereas humans utilized a variety of culturally specific characters, such as Korean consonants ("ㅋ", "ㅠ", "ㅜ") and special symbols ("♡", "★"), making it possible to distinguish between AI-generated and human-written texts.
Notably, 26% of human-written comments included formatting characters such as line breaks and multiple spaces, while only 1% of AI-generated comments did so. The use of repeated characters was also four times higher in human-written comments (52%) compared to AI-generated comments (12%).
The joint research team incorporated these differences to enhance XDAC's detection performance. They implemented methods to convert formatting characters such as line breaks and spaces, and to transform repeated character patterns into machine-readable formats. The system was also designed to identify which AI model generated a comment by recognizing unique features (speech styles) of each LLM.
Based on these advancements, the joint research team emphasized that XDAC's detection technology can serve not only to distinguish AI-generated comments but also as a psychological deterrent. Much like sobriety checkpoints, drug tests, or CCTV installations deter crime, the very existence of precise detection technology can reduce attempts to misuse AI.
In particular, the joint research team anticipates that platform operators will be able to use XDAC to closely monitor and respond to suspicious accounts or organized opinion manipulation attempts. XDAC is also considered highly likely to be expanded into real-time monitoring systems or automated response algorithms.
Wooyoung Ko, Senior Researcher at NSRI (PhD candidate at KAIST), said, "This study represents the world's first technology capable of detecting short comments written by generative AI with high accuracy and identifying the generation model. The significance of this research lies in establishing a technological foundation to counter AI-based opinion manipulation."
© The Asia Business Daily(www.asiae.co.kr). All rights reserved.
