tangledgroup
/

tangled-llama-109m-32k-base-v0.1

Text Generation

text-generation-inference

Inference Endpoints

Model card Files Files and versions Community

mtasic85 commited on 14 days ago

Commit

1450610

•

1 Parent(s): 2585cb8

eval

Files changed (1) hide show

README.md +1 -1

README.md CHANGED Viewed

@@ -100,7 +100,7 @@ litgpt evaluate --tasks 'leaderboard' --out_dir 'evaluate-0/' --batch_size 4 --d
 litgpt evaluate --tasks 'hellaswag,gsm8k,truthfulqa_mc2,mmlu,winogrande,arc_challenge' --out_dir 'evaluate-1/' --batch_size 4 --dtype 'bfloat16' out/pretrain/final/
 ```
-                 Tasks                 |Version|     Filter     |n-shot|  Metric   |   |Value |   |Stderr|
 |---------------------------------------|------:|----------------|-----:|-----------|---|-----:|---|-----:|
 |arc_challenge                          |      1|none            |     0|acc        |↑  |0.2082|±  |0.0119|
 |                                       |       |none            |     0|acc_norm   |↑  |0.2474|±  |0.0126|

 litgpt evaluate --tasks 'hellaswag,gsm8k,truthfulqa_mc2,mmlu,winogrande,arc_challenge' --out_dir 'evaluate-1/' --batch_size 4 --dtype 'bfloat16' out/pretrain/final/
 ```
+|                 Tasks                 |Version|     Filter     |n-shot|  Metric   |   |Value |   |Stderr|
 |---------------------------------------|------:|----------------|-----:|-----------|---|-----:|---|-----:|
 |arc_challenge                          |      1|none            |     0|acc        |↑  |0.2082|±  |0.0119|
 |                                       |       |none            |     0|acc_norm   |↑  |0.2474|±  |0.0126|