Announcement of Digital Service Disruption Investigation Results on the 6th
"No Detailed Response Plan or Simulation Drill for SK C&C Fire"
SK C&C Required to Establish 'Fire Detection System'; Kakao Asked for 'Service Redundancy'
[Asia Economy Reporter Minyoung Cha] On the 6th, the government announced in a briefing on the investigation results of the 'Pangyo Data Center Fire' incident that the number of damage reports due to Kakao service disruptions reached approximately 105,000 cases. This is part of the follow-up measures after the Pangyo Data Center fire in October. The government requested SK C&C, Kakao, and Naver to establish and report improvement measures for the main causes of the accident and future plans within one month.
"Fire response manual exists, but no simulation drills"
The Ministry of Science and ICT, the Korea Communications Commission, and the National Fire Agency jointly announced the investigation results on the SK C&C Pangyo Data Center fire and service disruptions of additional communication services such as Kakao and Naver during the briefing.
On October 15, a fire occurred at the SK C&C Pangyo Data Center, causing service disruptions for tenant companies including Kakao and Naver. According to the investigation, Kakao's services were normalized around 11 PM on the 20th, taking 127 hours and 33 minutes to recover from the disruption. Naver experienced some functional errors (such as some news article comments being unavailable). Most major services and functions were restored within approximately 20 minutes to 12 hours. The government immediately formed the 'Broadcasting and Communications Disaster Countermeasures Headquarters' led by the Minister of Science and ICT.
According to the government's accident investigation and analysis results, the fire was extinguished at 11:45 PM on the day of the incident. Power was restored around 5 AM on the 19th. The SK C&C Pangyo Data Center was equipped with a Battery Monitoring System (BMS) to monitor battery temperature and other factors, but it failed to detect signs of fire. The fire started in the battery room on the third basement floor. Some UPS units located in a partitioned space where power lines and ceiling space were not separated above the battery room were damaged and stopped functioning due to heat and other factors (estimated) caused by the fire. To prevent secondary damage such as electric leakage during water spraying, a power cut was decided.
In particular, although gas extinguishing equipment operated after the fire, initial suppression was limited due to the characteristics of lithium-ion battery fires, which are difficult to extinguish with gas. The lithium-ion batteries were not physically completely separated from some uninterruptible power supplies (UPS), causing the UPS to stop functioning and power supply to be interrupted. Although a fire response manual existed, it was revealed that there were no detailed response plans or simulation drills reflecting the actual fire situation level.
Damage scale from Kakao service disruption is 105,116 cases
The number of damage reports due to Kakao service disruptions totaled 105,116 cases. Damage related to paid services accounted for 14,918 cases, and monetary damages mentioned for free services accounted for 13,198 cases. Naver implemented redundancy measures between data centers, so there was no service interruption. However, some damages occurred, such as product reviews not appearing in the shopping mall, some news comments being unavailable, and errors in the Papago pronunciation listening function.
The government urged the three companies to take preemptive actions on matters that can be addressed in the short term. For issues requiring medium- to long-term measures or specific plans, the government requested that action plans be reported within one month. The results of the measures submitted by each company, future plans, and opinions on disaster prevention and recovery will be reflected in future policy measures.
The Ministry of Science and ICT urged SK C&C to strengthen management of battery monitoring system measurement information and to establish various fire detection systems beyond the current battery monitoring system. They requested the installation of fire extinguishing facilities necessary for lithium-ion battery fires or, if impossible, to prepare alternative measures. They also demanded physical separation of batteries and other electrical equipment, relocation of power lines located in the battery room to secure structural stability, and consideration of alternative measures. They asked for individual power cut-off in disaster zones and the ability to cut power in the affected area without direct entry into the disaster site. Separately, they requested the development of disaster response scenarios, establishment of detailed training plans, and reporting of training results.
Kakao was required to implement service diversification. They were urged to apply a very high level of redundancy between data centers for the 'operation and management tools,' which were the core cause of delayed recovery from the disruption. For key functions such as authentication, they were asked to establish plans to apply higher levels of distribution and redundancy than currently in place. They were also urged to conduct disaster simulation drills, establish rapid recovery plans for each scenario, build a system to promptly notify users in various ways in case of service disruptions, and set principles and standards for national damage relief and compensation plans.
Naver was requested to re-examine recovery targets by service, recovery plans for each disruption scenario, and prepare improvement measures, as well as conduct simulation drills assuming situations such as the complete destruction of the main data center.
Comprehensive improvement plan to be prepared by the first quarter of next year
The Ministry of Science and ICT plans to establish a comprehensive improvement plan to ensure digital service stability after one month, based on the results of measures submitted by the three companies, future action plans, and expert and operator feedback for strengthening disaster management systems. The Korea Communications Commission required Kakao and others to open dedicated damage report channels and establish damage compensation consultative bodies to develop and implement practical damage relief measures. They also plan to prepare improvement measures for laws, regulations, and terms of use to ensure user notification and effective damage relief in case of future communication disruptions.
Minister Jongho Lee of the Ministry of Science and ICT stated, "The government will take this incident as an opportunity to fundamentally review the disaster response system for major digital services and comprehensively prepare improvement plans, inspection results, and institutional improvements tailored to the digital era. We will build a robust digital crisis management system that can provide uninterrupted digital services."
© The Asia Business Daily(www.asiae.co.kr). All rights reserved.
