Samjong KPMG "Rise of On-Device AI... Expansion of Low-Power, High-Performance AI Semiconductor Market"

‘A New Stage Unfolded for Generative AI, On-Device AI’ Report
“On-Device AI Emerging to Run AI Services on Devices”
“On-Device AI Strengths in Speed, Cost, and Security”

As 'on-device AI' emerges to complement the shortcomings of existing cloud-based artificial intelligence (AI), a strategy has been proposed to successfully implement on-device AI technology and lead the diversification of services.

Strategies for Scaling Up in the On-Device AI Era [Provided by Samjong KPMG]

Samjong KPMG published a report titled “A New Stage Unfolded for Generative AI, On-Device AI” on the 25th, analyzing utilization strategies for on-device AI technology, which is emerging as a new platform.

Existing AI services based on generative AI, such as OpenAI’s ChatGPT, Google’s Gemini, and Microsoft’s Copilot, operate based on large-scale language models and utilize cloud systems. As models become increasingly large, issues related to operating large AI models have been raised, particularly the massive consumption of data leading to significant use of computing resources such as power and semiconductors.

In contrast, on-device AI operates when a user inputs a request into the device, activating the AI model and semiconductor-based internal data processing system embedded within the device, enabling AI services without external data transfer. Innovative products and features equipped with on-device AI, such as Samsung Electronics’ Galaxy S24 series, Apple Intelligence, and Microsoft’s Copilot+PC, have been unveiled one after another.

Generative AI services based on this have faster AI model analysis speeds and relatively lower system operating costs. Since the information entered by users is not sent outside the device, it is highly suitable for handling sensitive data such as personal information. It is also highly regarded for reducing the burden of securing infrastructure for external network and cloud transmission for data processing.

The report proposed strategies to drive business innovation and scale-up for companies based on on-device AI. First, since on-device AI operates AI models using power within the device, power consumption must be kept low. Semiconductor technologies capable of operating at low power are expected to grow significantly at an average annual rate of over 40% until 2028, while high-performance semiconductors are expected to expand mainly in markets such as cloud and data centers where power constraints are less severe.

Opportunities should be sought within the ‘ambient computing’ market, where IT devices around users autonomously learn user patterns and provide necessary services on their own. The excellent security performance of on-device AI has great value in building and utilizing an ambient computing environment that secures and analyzes personal data.

Since services operate based on small AI models, the small AI model market is also expected to expand significantly. Development of small AI models will proceed to improve the quality of results required by individual on-device AI devices equipped with AI models, such as smartphones and home appliances.

Additionally, utilizing on-device AI brings users and AI models closer by providing AI analysis results to users instantly. This underscores the need to establish control measures for ‘explainable AI’ technology to reduce user confusion and strengthen the operational stability of AI models.

Lee Jun-gi, Executive Director of the AI Center at Samjong KPMG, stated, “Focusing on on-device AI, which is emerging as a new trend in generative AI utilization, we must concentrate on expanding the AI ecosystem and securing use cases. By developing services that enhance user experience based on strategies that effectively leverage the advantages of on-device AI, it will be an opportunity to secure new business potential by gaining differentiated competitiveness in the AI market and increasing customer value.”