[Reading Science] Global AI Chip Development Competition, The Real Battle Is 'Software'

Active Discussions Worldwide on Developing Alternatives to Nvidia Chips
Without Focus on Software, Development May Be Futile
Experts Say "AI Chips and Software Are One"
Breaking Nvidia's 'CUDA'-Based Ecosystem Is Not Easy

The shortage of NVIDIA's graphics processing unit (GPU) chips for artificial intelligence (AI) training is worsening. NVIDIA, which virtually monopolizes the market, has seen its stock price soar continuously. As the chip shortage persists, reports that Sam Altman, CEO of OpenAI, is planning to raise $7 trillion to design and manufacture chips directly are fueling competition in AI chip development.

Besides Altman, governments and companies worldwide are accelerating AI chip development, but analysis suggests that simply developing chips alone will not overcome the current situation. This is because AI chips contain secrets that semiconductor technology alone cannot solve: the software supporting the chips. This is why the AI chip promotion plan pursued by our government must encompass software development to establish an AI chip ecosystem.

Professor Yoo Hoe-jun of the KAIST Graduate School of AI Semiconductor diagnosed that a key element is missing in the global AI chip development competition. Professor Yoo emphasized, "The role of software is very important in AI chips." He warned that even if chips replacing GPUs are developed from the perspective of conventional system semiconductors, they could be pushed out before competing with NVIDIA.

Professor Yoo's warning is already confirmed in the field. The downfall of Nervana Systems technology, acquired by Intel for AI, is a case where obsession with chips actually ruined the effort. Naveen Rao, who founded Nervana and later served as an Intel vice president after selling the company to Intel, aimed to develop chips to replace NVIDIA's GPUs. In an interview with The New York Times (NYT), Rao said, "While Intel hesitated, NVIDIA quickly improved the AI features I was trying to develop and responded." What determined the fate of Intel and NVIDIA was not simply chip performance. After leaving Intel, Rao compared NVIDIA's chips with competitors' from an objective perspective and found a significant difference. He judged that it would be difficult for NVIDIA's competitors to surpass the community of software engineers developing AI using NVIDIA's chips. The power of those using NVIDIA's development language CUDA is the core pillar supporting NVIDIA.

Rao said, "All AI developers inevitably use NVIDIA's chips first." He explained that developers already familiar with software running on NVIDIA's AI chips have no reason to choose other chips. The path developers can take is to wait for NVIDIA's chip supply rather than buying competitors' chips. This is the background behind most attempts to replace NVIDIA's GPUs failing and NVIDIA's performance continuously exceeding market expectations. Daniel Newman, an analyst at Futurum Group, predicted, "Surprisingly, NVIDIA's customers will wait even 18 months."

Major big tech companies such as Microsoft, Google, Amazon, Meta, and Tesla have also developed their own AI chips, but these have limitations as well. Market research firm Omdia diagnosed that over 70% of NVIDIA AI chip sales come from big tech. This reflects the reality that simply developing chips does not easily promote AI. Engineers already adapted to NVIDIA's CUDA have no reason to learn new languages to use other companies' chips.

The reality that even if a chip with superior performance is manufactured, it will inevitably fall behind in competition if it does not receive consumer choice can also be found in the market for calculators used for learning. The performance of Texas Instruments (TI) calculators, mainly used by American high school students, is inferior compared to competitors' products. Yet, math teachers require students to purchase TI chips for classes. This is the result of TI continuously providing education on using its calculators to teachers. American teachers choose TI calculators familiar to their hands no matter how good the performance of other calculators is. This phenomenon also occurs with NVIDIA's GPUs.

Most competing companies aim to replace NVIDIA chips cheaply. However, developing software that matches chip performance requires more investment and manpower than chip development. CUDA transformed NVIDIA's GPU, which might have only handled flashy graphics for PC games, into a tool usable for simulations in scientific fields such as physics and chemistry.

NVIDIA CEO Jensen Huang's bold investment in software cannot be overlooked. After confirming the new application field of GPUs in AI, he actively supported CUDA. An investment of $30 billion was made over ten years. NVIDIA also created and supported various software libraries to ease developers' efforts beyond CUDA. NVIDIA's deep consideration of what AI developers need has united NVIDIA and AI developers. A person in the computing industry who has used CUDA since its early days warned, "AI chips without software support are likely to be ignored even if released to the market."

After President Yoon Suk-yeol emphasized the importance of AI semiconductors at a semiconductor public discussion held last January, the government has taken active steps to foster the AI semiconductor industry. Park Sung-wook, the science and technology chief at the presidential office, stated, "We are preparing a big framework by linking AI semiconductors."

Moon Song-cheon, KAIST emeritus professor who first proposed the term 'cloud' worldwide, emphasized, "AI chips do not work at all without system software." Professor Moon advised, "The government and industry must also focus on software core technologies such as operating systems (OS) and databases (DB) to respond to AI-centered changes." He added, "With Windows 12 expected to open the curtain for AI OS soon, we must no longer rely on others' technologies but open the chapter of independent OS development."

▲Terminology explanation: CUDA is a programming tool that operates on NVIDIA's GPUs. It utilizes the GPU cores to perform parallel tasks and supports rapid processing of simple large-scale data compared to CPUs. It is used in various fields such as science and engineering, deep learning and artificial intelligence, medical imaging, and financial modeling. Since its introduction in 2006, it has developed through continuous support from NVIDIA and increased participation from developers. It is also called the core of the NVIDIA ecosystem.