Meta unveils its latest large language model Llama 3
Comes in two variants - Llama 3 8B and Llama 3 70B
Meta on Thursday released Llama 3, the latest iteration of its popular large language model (LLM) series, claiming significant performance improvements over its predecessors.
The newly unveiled model will be integrated into Meta AI, Meta's free virtual assistant, which the company boasts is the most advanced among its free-to-use counterparts.
"Thanks to our latest advances with Llama 3, Meta AI is smarter, faster, and more fun than ever before," the company said in a blog post.
The move underscores the Meta's desire to remaining at the forefront of AI development. While CEO Mark Zuckerberg has emphasised the "free-to-use" aspect of Meta AI, it's clear that the company aims to challenge OpenAI for dominance in the rapidly evolving landscape of AI.
Llama 3 arrives in two variants: Llama 3 8B with 8 billion parameters and Llama 3 70B with 70 billion parameters. These parameters represent the "knowledge" the model acquires during training, essentially influencing its capabilities.
Generally, the higher the parameter count, the more capable an LLM is at tasks like analysing and generating text.
Both models are claimed to be a significant leap forward compared to Llama 2.
Meta says that Llama 3 8B outperforms other open models like Mistral 7B and Google's Gemma 7B on several benchmarks, demonstrating its abilities in areas like knowledge acquisition, reasoning and code generation.
The larger Llama 3 70B takes things a step further, according to Meta. It supposedly competes with leading proprietary models such as Google's latest Gemini 1.5 Pro.
While not surpassing Anthropic's most powerful offering, Claude 3 Opus, Llama 3 70B reportedly outshines Claude 3 Sonnet, a less powerful model in the Claude 3 series, again according to Meta.
Meta says it conducted its own tests encompassing tasks like coding, writing, reasoning and summarisation, in which Llama 3 70B emerged victorious when pitted against competitors like OpenAI's GPT-3.5.
It's important to note that Meta designed this test set, so the results should be viewed with some caution.
Beyond benchmarks, Meta highlights qualitative improvements in user experience. Users can expect Llama 3 to be more "steerable," meaning it's less likely to refuse answering questions. Additionally, the model should offer higher accuracy on trivia, STEM-related queries and basic coding recommendations.
Llama has been trained on a massive dataset – 15 trillion tokens, seven times larger than Llama 2's. The data includes code (four times more than Llama 2) and non-English text in 30 languages to bolster performance beyond English. Meta also used synthetic data, a controversial approach due to potential drawbacks, to train on longer documents.
The models are downloadable now and power Meta's AI assistant across its platforms. Soon, they will be available through various cloud providers and optimised for specific hardware from major companies.
Meta is training even larger models exceeding 400 billion parameters. These future iterations aim to be multilingual, handle various data formats beyond text, and offer improved reasoning and coding capabilities.
Open weights
Llama 3 is an open-weights model like Llama 2, released under an open licence, meaning it can be used for free (with some caveats), including for commercial purposes. Unlike closed models such as Claude and ChatGPT it is freely available and can be modified by other developers. However, it does not technically qualify as "open source" since it is not released under a licence recognised by the Open Source Institute. Nevertheless its more open nature should enable its more rapid development, as third party developers build on Meta's work.
Rodrigo Liang, CEO and co-founder of SambaNova Systems, said: "Enterprises don't need to build their own models, because the open-source community is more innovative than any single organisation. A community working together will always be more powerful than an individual company, and leveraging this community is the best way for any organisation to keep up with a rapidly evolving AI landscape."
Victor Botev, CTO and co-founder of Iris.ai, said: "With the global shift towards AI regulation, the launch of Meta's Llama 3 model is notable.
"By embracing transparency through open-sourcing, Meta aligns with the growing emphasis on responsible AI practices and ethical development. Moreover, this grants the opportunity for wider community education as open models facilitate insights into development and the ability to scrutinise various approaches, with this transparency feeding back into the drafting and enforcement of regulation.
"However, it's important to maintain perspective – a model simply being open-source does not automatically equate to ethical AI."