Meta-Llama-3–8B-Instruct the New King of LLMs

ALAgrApHY
2 min readApr 20, 2024

A new open source Large Language Model (LLM) dawns upon us with impressive performance and immediate availability. Ecce LLAMA 3!

llama-3

According to some reddit reviews, it seems to be uncesored out of the box: https://www.reddit.com/r/LocalLLaMA/comments/1c7dmau/so_llama_3_seems_somewhat_uncensored_out_of_the/

Llama-3 Open Access and Availability:

The model can be downloaded using pipeline:

import transformers
import torch
model_id = "meta-llama/Meta-Llama-3-8B-Instruct"pipeline = transformers.pipeline(
"text-generation",
model="meta-llama/Meta-Llama-3-8B-Instruct",
model_kwargs={"torch_dtype": torch.bfloat16},
device="cuda",
)

Or it can be downloaded locally via ollama: https://ollama.com/library/llama3

Or it can be directly accessed via https://ai.meta.com that is available in some countries, not France:

Llama-3 Evaluation:

I have collected a couple of evaluations made by humans and by benchmarks, starting with the one made by humans:

the Llama 3 models’ scores on popular AI benchmarks like MMLU (which attempts to measure knowledge), ARC (which attempts to measure skill acquisition) and DROP (which tests a model’s reasoning over chunks of text). As we’ve written about before, the usefulness — and validity — of these benchmarks is up for debate. But for better or worse, they remain one of the few standardized ways by which AI players like Meta evaluate their models.

llama 3 8b

And followed by the model’s performance according to Benchmarks. Note that Llama-3 is being compared to models of the same size. Larger llama-3 models like 80b and 400b are available however the latter is still training.

llama 3 400b

Image credit: https://ai.meta.com/

--

--

ALAgrApHY

Heptaglot Artist, Data Scientist, Filmmaker exploring Creative AI. Started the GAN AI Art Movement (2016). Former Postdoc @CNRS PhD @INFORMATICS. 3xTEDx Speaker