dahara1
/

weblab-10b-instruction-sft-GPTQ

Text Generation

text-generation-inference

Model card Files Files and versions Community

dahara1 commited on Aug 24, 2023

Commit

fe212b7

•

1 Parent(s): 4065b4d

Update README.md

Files changed (1) hide show

README.md +2 -1

README.md CHANGED Viewed

@@ -67,7 +67,8 @@ Also, the score may change as a result of more tuning.
 * **Japanese benchmark**
-    - *We used [Stability-AI/lm-evaluation-harness + gptq patch](https://github.com/webbigdata-jp/lm-evaluation-harness) for evaluation.*
     - *The 4-task average accuracy is based on results of JCommonsenseQA-1.1, JNLI-1.1, MARC-ja-1.1, and JSQuAD-1.1.*
     - *model loading is performed with gptq_use_triton=True, and evaluation is performed with template version 0.3 using the few-shot in-context learning.*
     - *The number of few-shots is 3,3,3,2.*

 * **Japanese benchmark**
+    - *We used [patched Stability-AI/lm-evaluation-harness](https://github.com/webbigdata-jp/lm-evaluation-harness) for evaluation.*
+    - [Stability-AI/lm-evaluation-harness](https://github.com/webbigdata-jp/lm-evaluation-harness) +  [gakada's AutoGPTQ path](https://github.com/EleutherAI/lm-evaluation-harness/pull/519)*
     - *The 4-task average accuracy is based on results of JCommonsenseQA-1.1, JNLI-1.1, MARC-ja-1.1, and JSQuAD-1.1.*
     - *model loading is performed with gptq_use_triton=True, and evaluation is performed with template version 0.3 using the few-shot in-context learning.*
     - *The number of few-shots is 3,3,3,2.*