Training Cost Increased 18 Times from GPT-3 to GPT-4
Data Centers, Semiconductors, Workforce, and Cloud Investment Upward
The stock market plummeted due to concerns over the earnings of major U.S. big tech companies, spreading the 'Artificial Intelligence (AI) bubble theory.' Although the costs of AI development have increased exponentially compared to the initial stages, the industry inside and outside is beginning to realize that generating sufficient revenue to offset these costs is challenging.
At the core of doubts about the profitability of AI companies lies astronomical costs. According to the 'Stanford AI Index 2024' report from the Human-Centered AI Institute (HAI) at Stanford University on the 26th, the estimated training costs for AI models have significantly increased. For example, OpenAI spent $4.32 million (approximately 600 million KRW) on the generative AI GPT-3 model in 2020, but training the GPT-4 model last year cost $78.35 million (approximately 108.5 billion KRW). This represents an 18-fold increase in just three years.
The AI model with the highest training cost compiled by HAI was Google's Gemini Ultra, which cost $191.4 million (approximately 265 billion KRW). In 2017, the transformer model, a basic form of generative AI, required only $930 (approximately 1.3 million KRW) in costs.
The burden of costs associated with AI models is not limited to capital input in training alone. Investments are also rapidly increasing in infrastructure-related areas such as data center construction, semiconductor purchases, hiring personnel, and cloud support. This trend can be seen even by looking at cloud expenditure. According to market research firm IDC, global total spending on cloud infrastructure in the first quarter of this year increased by about 37% compared to the same period last year, reaching $33 billion (approximately 45.7 trillion KRW).
Private investment was particularly prominent in the U.S., which leads AI development, with total investment in AI from 2013 to last year reaching $335.24 billion (approximately 464.6 trillion KRW). This was followed by China ($103.65 billion), the United Kingdom ($22.25 billion), Israel ($12.83 billion), and Canada ($10.56 billion). South Korea ranked 9th with $7.25 billion, following France in 8th place with $8.31 billion.
Although enormous costs are being invested in AI development, only a limited number of companies are showing results. Google recorded capital expenditures of $13.2 billion in the second quarter of this year, exceeding expert expectations by 8%. This was due to increased infrastructure investment required to support generative AI, raising concerns that the spending might be too high compared to future revenue. Daniel Morgan, senior portfolio manager at U.S. asset management firm Synovus Trust, said, "Investors are looking for clear evidence of return on investment for the billions of dollars invested in AI," and criticized, "How much profit is Google actually making from AI? According to reports, Google is still earning revenue primarily through advertising and search."
Even among big tech companies capable of massive capital investment, results have not materialized, making the situation even more difficult for smaller companies or startups. For example, Upstage, a domestic AI technology company gaining attention, recorded sales of 4.6 billion KRW and an operating loss of 18.9 billion KRW as of last year.
There are still many technical aspects that need improvement. HAI stated, "AI surpasses humans in several benchmarks such as image classification, visual reasoning, and understanding English," but also diagnosed that "it lags behind in common-sense reasoning in vision, planning, and higher-level mathematics."
Doubts about AI profitability have spread from the U.S. stock market to South Korea. SK Hynix, a leader in high-bandwidth memory (HBM), a core memory for AI semiconductors, achieved its best performance in six years but saw its stock price plunge 8.87% as of the 25th.
However, there is hope that profitability can improve if technological efficiency increases. An industry insider said, "If the burden of capital expenditures continues to grow, big tech companies will take longer to reach breakeven points, cash flow will become difficult, and startups will be affected," but added, "Only companies and startups with solid fundamentals and strong visions will survive until they reach a stable trajectory." They also predicted, "Research to reduce costs is already underway, and the cost required to train AI models of the same size has decreased nearly tenfold compared to the initial stages, so profitability is expected to improve in the long term."
© The Asia Business Daily(www.asiae.co.kr). All rights reserved.
