smol llama
Collection
🚧"raw" pretrained smol_llama checkpoints - WIP 🚧
•
4 items
•
Updated
•
6
A small 220M param (total) decoder model. This is the first version of the model.
Here are some fine-tunes we did, but there are many more possibilities out there!
Detailed results can be found here
Metric | Value |
---|---|
Avg. | 29.44 |
AI2 Reasoning Challenge (25-Shot) | 24.83 |
HellaSwag (10-Shot) | 29.76 |
MMLU (5-Shot) | 25.85 |
TruthfulQA (0-shot) | 44.55 |
Winogrande (5-shot) | 50.99 |
GSM8k (5-shot) | 0.68 |
Detailed results can be found here
Metric | Value |
---|---|
Avg. | 6.62 |
IFEval (0-Shot) | 23.86 |
BBH (3-Shot) | 3.04 |
MATH Lvl 5 (4-Shot) | 0.00 |
GPQA (0-shot) | 0.78 |
MuSR (0-shot) | 9.07 |
MMLU-PRO (5-shot) | 1.66 |