Meta’s Llama 4 models Maverick and Scout are out now, but they might not be the best models on the market.

There are two new AI models joining the slew of options, thanks to new releases from Meta. The company unveiled its latest family of generative AI models on Saturday, named Llama 4. You can give the Llama 4 models a test drive now through Meta AI’s website, and Llama 4 will soon power the many Meta AI features on the company’s Instagram, WhatsApp and Messenger services.

The competition between Meta and other AI companies is becoming increasingly intense. Companies are working to build and release AI models capable of more complex tasks and advanced reasoning without requiring vast amounts of computing power and cash to run. It’s a tricky sweet spot to hit, and Meta hopes its newest models will put it ahead of competitors like ChatGPT and Gemini. 

Llama 4 benchmarking, explained

There has been some confusion about how the new Llama models stack up compared to other models. There are a couple of ways to test chatbots, and one of the biggest is through a crowdsource AI benchmarking platform called LMArena, created by UC Berkeley SkyLab. In Meta’s announcement, the company claimed its Maverick model had outperformed ChatGPT-4o. 

But the model that Meta actually submitted to the LMArena tests is not the model that is available for people to use now. The model submitted for testing is called “llama-4-maverick-03-26-experimental.” In a footnote on a chart on Llama’s website (not the announcement), in tiny font in the final bullet point, Meta clarifies that the model submitted to LMArena was ‘optimized for conversationality.” 

LMArena put out a statement on X/Twitter on Monday saying Meta’s policy interpretation did not match LMArena’s expectations, and that Meta should have been clearer that the submitted model was a “customized model to optimize for human preference.” In other words, it’s possible that Meta submitted a better, more human-friendly model to try and juice its scores. One way to do that could be training a model on test sets, which a set of data and tests typically run during the post-training process, not before.

Meta’s VP of generative AI Ahmad Al Dahle tweeted that claims the company trained on test sets are “simply not true.” He said that the differences in performance people are seeing “is due to needing to stabilize implementations.” As of publication, Meta’s Llama Maverick experimental (the original model submitted) is ranked in second place on LMArena, tied with GPT-4o and preview of Grok 3. Google’s Gemini 2.5 Pro is first.

Meta did not immediately return a request for comment.

Meet Scout and Maverick

There are two models in the Llama 4 family available now: Scout and Maverick. They’re open-weights models and multimodal, which means they can generate text, images and code. Open models like Meta’s mean that developers can get some insight into how the models are built. The Llama 4 models are open-weights models, which means you can see how the model makes connections and how certain characteristics are given more weight as it learns. OpenAI announced earlier this month that it is developing an open-weights model for the first time.

Scout is the smallest model of the family, designed to run on a single Nvidia H100 GPU. Scout has a 10 million token context window and is a 17 billion parameter model featuring 16 experts (subnetworks within the model, allowing it to run tasks more efficiently). Scout has more than twice the firepower of Llama 3, which has 8 billion parameters. Generally, the more parameters a model has, the more capable it is of delivering better results faster. Maverick is a midsized model, the big brother to Scout, featuring 17 billion parameters with 128 experts.

More information on the rest of the Llama 4 family, including a base model named Behemoth and a Llama 4 reasoning model, is expected to come later this month, according to a video posted by CEO Mark Zuckerberg. We’ll likely learn more about these models at LlamaCon, the company’s first annual AI developers conference beginning on April 29.

For more, check out what we know about a potential standalone app for Meta AI and our review of the best AI chatbots.