Mandatory Age Verification Features for Users
Negative Incidents Involving Teen Suicides and More
The state of California has become the first in the United States to enact legislation restricting the use of artificial intelligence (AI) chatbots by children and adolescents. As the use of chatbots among young people has increased, along with a rise in negative incidents, California appears to be taking regulatory action to protect minors.
On October 13 (local time), California Governor Gavin Newsom announced that he had signed a bill aimed at strengthening online child protection by establishing safety measures for new technologies such as AI.
According to the Associated Press, this bill is one of several AI-related legislative proposals introduced in the California State Legislature this year, and is intended to put the brakes on the rapidly expanding U.S. AI industry, which has been growing largely unchecked. In response, technology companies and industry coalitions have spent at least 2.5 million dollars over the past six months on lobbying efforts to block the legislation, according to the tech watchdog group Tech Oversight California.
Taking effect next year, the law requires AI chatbot operators to implement age verification features for platform users and to clearly indicate that all responses generated by the AI chatbot are artificially produced.
In particular, chatbot platforms known as "companion chatbots," which provide intimate conversational functions, are now required to establish protocols to identify and respond to expressions of suicidal ideation or self-harm by users. These incidents must also be reported to the California Department of Public Health.
Additionally, the law prohibits chatbots from impersonating medical professionals and requires that minors receive notifications every three hours during use, reminding them to take a break. The legislation also mandates safeguards to prevent minors from being exposed to sexually explicit images generated by chatbots.
The bill also strengthens penalties for profiting from illegal deepfakes, imposing fines of up to 250,000 dollars (approximately 360 million Korean won) per offense.
Governor Newsom stated, "New technologies such as chatbots and social media can inspire and connect people, but without real safeguards, they can exploit, mislead, and endanger our children."
He added, "We have witnessed horrific and tragic cases of young people being harmed by unregulated technology. We will not stand by and allow companies to operate without necessary restrictions and accountability."
Previously in the United States, there was a case in which a teenage minor engaged in prolonged conversations about suicide with OpenAI's chatbot ChatGPT and ultimately took their own life. As negative incidents have continued to occur alongside the surge in chatbot usage, states such as Illinois, Nevada, and Utah have passed laws restricting or outright banning the use of AI chatbots as tools for mental health counseling and treatment.
© The Asia Business Daily(www.asiae.co.kr). All rights reserved.
