Meta Buys Nvidia Chips in Bulk to Launch Latest 'Llama 3.1' AI Model
Emphasizes Parity with OpenAI, Google, and Anthropic
A path has opened to use the latest generative AI at the ChatGPT-4.0 level for free. Meta, the parent company of Facebook, has released the latest large language model (LLM) Llama 3.1 as open source, making it available to everyone. Analysts say that Mark Zuckerberg, Meta's CEO, is shaking up the existing AI competition landscape by showcasing a close relationship with Jensen Huang, CEO of Nvidia, which dominates the AI training GPU market.
Mark Zuckerberg, CEO of Meta, and Jensen Huang, CEO of NVIDIA, are taking a commemorative photo wearing each other's shirts. Photo by Mark Zuckerberg Instagram
On the 23rd (local time), Meta announced the release of 'Llama 3.1'. Since unveiling 'Llama 3' last April, Meta has demonstrated a remarkable enhancement in LLM performance in just over three months.
The largest version of Llama 3.1 is Llama 3.1 405B. The 405B has 405 billion parameters. Although ChatGPT-4 has not disclosed its number of parameters, this greatly surpasses GPT-3's 175 billion. Meta also released smaller models. These include the small model Llama 3.1 8B with 7 billion parameters and the medium model 3.1 70B with 70 billion parameters. Meta claims that performance has also significantly improved. Meta emphasized that Llama 3.1 outperforms OpenAI's latest model GPT-4o (Four-O) and Anthropic's Claude 3.5 Sonnet.
The secret to Meta's rapid performance boost for Llama lies in the latest Nvidia chips. Meta explained that it used 16,000 of Nvidia's latest GPUs, the 'H100', to train Llama 3.1. Earlier this year, Zuckerberg set a goal to purchase 350,000 H100s by the end of the year. Zuckerberg attracted attention by sharing a photo swapping jackets with Huang, and an event where they will have a conversation is scheduled for the 28th. It is expected that Zuckerberg will reveal behind-the-scenes stories about the development of Llama 3.1 at this event.
Llama is open source. This means other companies can use Llama to conduct AI business. This contrasts with AI models from OpenAI, Google, and Anthropic, which remain closed source.
Zuckerberg emphasized, "'Llama 3 is a product that can compete with the most advanced (frontier) models while being open source and accessible to everyone," citing the example of open-source Unix-based computers becoming mainstream and predicting that "the future path of AI is also open source." He said, "Starting next year, I expect Llama to be the most advanced model in the industry."
Although the pace of AI model development by overseas companies is accelerating, it is not easy for domestic companies to catch up. According to the Center for Research on Foundation Models (CRFM) at Stanford University in the U.S., Naver's first-generation 'HyperCLOVA', introduced in May 2021, had 82 billion parameters. Naver has not disclosed the number of parameters for the latest HyperCLOVA X. Kim Jong-won, head of the AI Graduate School at GIST, explained, "It is difficult to keep up with the development speed of overseas big tech companies because there is insufficient capacity to purchase large-scale Nvidia GPUs for training."
© The Asia Business Daily(www.asiae.co.kr). All rights reserved.
