A new open source Large Language Model (LLM) dawns upon us with impressive performance and immediate availability. Ecce LLAMA 3!
According to some reddit reviews, it seems to be uncesored out of the box: https://www.reddit.com/r/LocalLLaMA/comments/1c7dmau/so_llama_3_seems_somewhat_uncensored_out_of_the/
Llama-3 Open Access and Availability:
The model can be downloaded using pipeline:
import transformers
import torchmodel_id = "meta-llama/Meta-Llama-3-8B-Instruct"pipeline = transformers.pipeline(
"text-generation",
model="meta-llama/Meta-Llama-3-8B-Instruct",
model_kwargs={"torch_dtype": torch.bfloat16},
device="cuda",
)
Or it can be downloaded locally via ollama: https://ollama.com/library/llama3
Or it can be directly accessed via https://ai.meta.com that is available in some countries, not France:
Llama-3 Evaluation:
I have collected a couple of evaluations made by humans and by benchmarks, starting with the one made by humans:
the Llama 3 models’ scores on popular AI benchmarks like MMLU (which attempts to measure knowledge), ARC (which attempts to measure skill acquisition) and DROP (which tests a model’s reasoning over chunks of text). As we’ve written about before, the usefulness — and validity — of these benchmarks is up for debate. But for better or worse, they remain one of the few standardized ways by which AI players like Meta evaluate their models.
And followed by the model’s performance according to Benchmarks. Note that Llama-3 is being compared to models of the same size. Larger llama-3 models like 80b and 400b are available however the latter is still training.
Image credit: https://ai.meta.com/
