WSJ "GPT-5 Development Delayed Due to Data Limitations"
Market's Concerned AI Development Slowdown Becomes Reality
It has been reported that the development of GPT-5, the next-generation artificial intelligence (AI) model by OpenAI, the developer of ChatGPT, is indefinitely delayed due to data limitations. Experts predict that the speed of AI technology development by tech companies will significantly slow down due to a shortage of data, which is crucial for AI model training and inference.
When Will GPT-5 Finally Be Released? “Development Challenges”
The Wall Street Journal (WSJ) reported on the 22nd (local time), citing multiple sources, that OpenAI’s next-generation AI model GPT-5, known by the codename ‘Orion,’ is experiencing much greater delays than initially planned despite enormous development costs.
OpenAI reportedly began developing GPT-5 immediately after the release of GPT-4 in March last year. Microsoft (MS), OpenAI’s largest investor, expected the model to be released around mid-year. However, during at least two large-scale training processes involving massive amounts of data, new problems arose each time, preventing the achievement of the anticipated results.
WSJ evaluated, “The new model showed better performance than the existing GPT-4, but it did not demonstrate enough improvement to justify the enormous costs invested.” Industry estimates suggest that such large-scale AI training could cost about $500 million (approximately 720 billion KRW) in computing expenses alone over six months.
Sam Altman, CEO of OpenAI, unveiled the advanced reasoning AI model ‘o3’ on the 20th but did not mention when a new flagship model worthy of being called GPT-5 would be released. WSJ predicted, “GPT-5 will open doors not only to everyday human tasks like booking flights but also to new scientific discoveries.” A former OpenAI executive confidently stated that if GPT-4 was at the level of a smart high school student, GPT-5 would perform at a doctoral degree level in some tasks.
However, the biggest challenge causing the current development delays is the insufficient amount of data needed for more intelligent training. For previous models, OpenAI used data collected from the internet, including news articles, social media posts, and scientific papers, for AI training, but now this is considered inadequate.
Accordingly, OpenAI decided to generate data directly. To this end, they hired software engineers and mathematicians to create new software code or solve complex mathematical problems, allowing the AI to learn from these.
However, the process is critically slow. WSJ explained, “GPT-4 was trained on about 13 trillion tokens (the basic input unit for GPT models). (For GPT-5 development) if 1,000 people write 5,000 words a day, it would still take several months just to generate 1 billion tokens.”
Meanwhile, competitors have been attempting to poach OpenAI’s top researchers by offering millions of dollars. This year alone, more than 20 people, including OpenAI co-founder Ilya Sutskever, have left the company. WSJ noted, “In the meantime, competitors have released AI products comparable to OpenAI’s GPT-4,” citing Anthropic’s Claude 3 and Google’s NotebookLM as examples.
Is the Slowdown in AI Development Becoming a Reality?
With even OpenAI, the leader in generative AI, hitting limits in AI model development, concerns are growing that improvements in AI model performance may enter a prolonged stagnation phase.
Former OpenAI co-founder Ilya Sutskever pointed out in a lecture on the 13th, “While computer processing power is improving, data is not increasing,” adding, “There is only one internet.” He likened the data needed for AI model training to a finite resource like fossil fuels and predicted, “The pre-training of generative AI models as we know it will be depleted and come to an end.” Sundar Pichai, CEO of Google, emphasized at the New York Times DealBook Summit on the 4th that “all the low-hanging fruit has been harvested,” highlighting the slowdown in AI technology development.
Bloomberg reported, “The emergence of ChatGPT raised concerns that AI would disrupt all kinds of businesses, but two years later, the anticipated catastrophe has not occurred to the extent expected.”
© The Asia Business Daily(www.asiae.co.kr). All rights reserved.
