[News Terms] Korea's AI Weapon Leading the 3 Major Powers: 'sLLM'

The government has identified the ‘Small Large Language Model (sLLM),’ one of the nine key AI and semiconductor technologies to leap forward as a top three AI powerhouse. The sLLM refers to a language model with relatively fewer parameters compared to Large Language Models (LLMs). Generally, models with over 100 billion parameters are classified as LLMs, while those below this threshold are considered sLLMs.

President Yoon Suk-yeol is speaking at the semiconductor issue review meeting held on the 9th at the Yongsan Presidential Office building in Seoul. Photo by the Presidential Office Press Photographers Group [Image source=Yonhap News]

Parameters here refer to various variables considered during the AI’s computation process, which can be understood as synapses?the connection points where neurons exchange information in the brain. Just as more synapses allow for processing more information, a greater number of parameters enhances AI performance. For example, the representative LLM-based generative AI, ChatGPT, has 1 trillion parameters (based on GPT-4). NVIDIA’s next-generation AI semiconductor ‘Blackwell (B200),’ regarded as a potential ultra-large AI model, is known to support training and learning with up to 10 trillion parameters. The more parameters there are, the more advanced LLMs can be utilized.

However, a downside is that the more parameters there are, the longer and more costly the training and operation become. Even ChatGPT-4 required several months of training time and over 10 million dollars in development costs.

Recently, global big tech companies such as Meta, MS, and Google have entered the sLLM technology competition, attracted by the advantage that sLLMs require significantly less training time and cost compared to LLMs due to fewer parameters. Another unique advantage of sLLMs is their ability to be optimized for specific industries. By fine-tuning parameters for particular tasks, companies can achieve optimal results, making sLLMs more efficient for business applications than LLMs, according to industry assessments.

Meanwhile, on the 9th, related ministries including the Ministry of Trade, Industry and Energy and the Ministry of Science and ICT held a ‘Semiconductor Issue Review Meeting’ at the Presidential Office in Yongsan, Seoul, announcing plans to promote the ‘AI Semiconductor Initiative,’ which focuses on investing in R&D budgets for nine AI and semiconductor-related technologies to advance as a top three AI powerhouse.