Meta has announced the latest release of its open source AI model, Llama. According to Meta, with the release of Llama 3.1 405B, the company is trying to prove that open models can be just as capable as their closed counterparts, if not better.
“Llama 3.1 405B is the first openly available model that rivals the top AI models when it comes to state-of-the-art capabilities in general knowledge, steerability, math, tool use, and multilingual translation,” Meta wrote in a blog post. “With the release of the 405B model, we’re poised to supercharge innovation—with unprecedented opportunities for growth and exploration. We believe the latest generation of Llama will ignite new applications and modeling paradigms, including synthetic data generation to enable the improvement and training of smaller models, as well as model distillation—a capability that has never been achieved at this scale in open source.
The company evaluated Llama 3.1 against GPT-4, GPT-4o, and Claude 3.5 Sonnet. It outperformed or was on par with the models across a number of evaluations, such as math, reasoning, and coding.
The model was trained on over 15 trillion tokens, which required Meta to optimize its training stack and use over 16K H100 GPUs.
In addition to the 405B version, Llama 3.1 also comes in 8B and 70B options. The company also announced that with this release, it is also changing the license for Llama to allow developers to use its outputs to improve other models.
“While many may argue that closed models are more cost effective, Llama models offer some of the lowest cost per token in the industry, according to testing by Artificial Analysis. And as Mark Zuckerberg noted, open source will ensure that more people around the world have access to the benefits and opportunities of AI, that power isn’t concentrated in the hands of a small few, and that the technology can be deployed more evenly and safely across society. That’s why we continue to take steps on the path for open access AI to become the industry standard,” the company wrote.
The models are now available for download on Meta’s website or on Hugging Face.
You may also like…
Meta releases the first two Llama 3 models
Open Source Initiative is close to coming up with a definition for Open Source AI