Large Language Models with Over 100 Billion Parameters
SLM Achieves Low Cost and High Efficiency with Fewer Than 10 Billion Parameters
Representatives Include Paai3, Llama3, and Gemma2B
Companies venturing into generative artificial intelligence (AI) development are recently turning their attention to small language models (SLMs). This movement aims to maximize cost-effectiveness by delivering similar performance at a lower cost compared to large language models (LLMs).
According to major foreign media on the 18th (local time), big tech companies such as Microsoft (MS), Meta Platforms, and Google are consecutively unveiling 'small but strong' AI models that achieve decent performance with fewer parameters than LLMs. Parameters refer to various variables considered during AI's computational process, analogous to synapses in the human brain. The more parameters, the better the AI's performance.
Last month, MS unveiled the SLM model 'Pi-3 Mini.' Pi-3 Mini has 3.8 billion parameters, about 1/50th of OpenAI's GPT-3.5 (175 billion parameters). Luis Vargas, MS's Vice President of AI, explained, "The inference cost of Pi-3 Mini is about one-tenth that of other models with similar functionality."
Earlier, Meta also launched 'LLaMA-3' with 8 billion parameters, following the trend of downsizing generative AI. Since LLaMA-2, Meta has open-sourced its models, spawning numerous variant LLaMA models. Google also demonstrated a focus on high efficiency by releasing Gemma 2B in February, which has only 2 billion parameters.
The reason SLMs have recently attracted attention from big tech is their low operating costs. OpenAI's latest AI model GPT-4o (Omni) and Google's Gemini 1.5 Pro, both unveiled last week, are representative LLMs estimated to have over one trillion parameters. The unit cost per one million tokens for these two models is $5 and $7 respectively, more than 25 times higher than LLaMA-3's $0.2. Tokens refer to the units of character data recognized by language models.
In addition to low costs, the ability to use on-device processing is also cited as an advantage of SLMs. Instead of sending data to the cloud, SLMs can process tasks locally on devices, which is excellent for privacy protection. Notably, Apple is reportedly planning to unveil an iPhone equipped with OpenAI's ChatGPT at its annual developer event next month.
Charlotte Marshall, a lawyer at the banking advisory law firm Adleshogodad, said, "One of the important considerations for many clients adopting generative AI products at our firm was regulatory compliance regarding data processing and security," adding, "With the rise of SLMs, companies now have an opportunity to escape legal issues and cost burdens."
© The Asia Business Daily(www.asiae.co.kr). All rights reserved.
