abhinavnmagic commited on
Commit
91a872b
1 Parent(s): a8c9e50

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +18 -14
README.md CHANGED
@@ -28,7 +28,7 @@ license: llama3.1
28
  - **Model Developers:** Neural Magic
29
 
30
  Quantized version of [Meta-Llama-3.1-405B-Instruct](https://huggingface.co/meta-llama/Meta-Llama-3.1-405B-Instruct).
31
- It achieves an average score of x.x on the [OpenLLM](https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard) benchmark (version 1), whereas the unquantized model achieves x.x.
32
 
33
  ### Model Optimizations
34
 
@@ -146,11 +146,11 @@ This version of the lm-evaluation-harness includes versions of ARC-Challenge, GS
146
  <tr>
147
  <td>MMLU (5-shot)
148
  </td>
149
- <td>xx.xx
150
  </td>
151
- <td>xx.xx
152
  </td>
153
- <td>xx.xx%
154
  </td>
155
  </tr>
156
  <tr>
@@ -176,37 +176,41 @@ This version of the lm-evaluation-harness includes versions of ARC-Challenge, GS
176
  <tr>
177
  <td>Hellaswag (10-shot)
178
  </td>
179
- <td>xx.xx
180
  </td>
181
- <td>xx.xx%
 
 
182
  </td>
183
  </tr>
184
  <tr>
185
  <td>Winogrande (5-shot)
186
  </td>
187
- <td>xx.xx
 
 
188
  </td>
189
- <td>xx.xx%
190
  </td>
191
  </tr>
192
  <tr>
193
  <td>TruthfulQA (0-shot)
194
  </td>
195
- <td>xx.xx
196
  </td>
197
- <td>xx.xx
198
  </td>
199
- <td>xx.xx%
200
  </td>
201
  </tr>
202
  <tr>
203
  <td><strong>Average</strong>
204
  </td>
205
- <td><strong>xx.xx</strong>
206
  </td>
207
- <td><strong>xx.xx</strong>
208
  </td>
209
- <td><strong>xx.xx%</strong>
210
  </td>
211
  </tr>
212
  </table>
 
28
  - **Model Developers:** Neural Magic
29
 
30
  Quantized version of [Meta-Llama-3.1-405B-Instruct](https://huggingface.co/meta-llama/Meta-Llama-3.1-405B-Instruct).
31
+ It achieves an average score of 86.01 on the [OpenLLM](https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard) benchmark (version 1), whereas the unquantized model achieves 86.63.
32
 
33
  ### Model Optimizations
34
 
 
146
  <tr>
147
  <td>MMLU (5-shot)
148
  </td>
149
+ <td>86.25
150
  </td>
151
+ <td>85.97
152
  </td>
153
+ <td>99.67%
154
  </td>
155
  </tr>
156
  <tr>
 
176
  <tr>
177
  <td>Hellaswag (10-shot)
178
  </td>
179
+ <td>88.33
180
  </td>
181
+ <td>88.16
182
+ </td>
183
+ <td>99.80%
184
  </td>
185
  </tr>
186
  <tr>
187
  <td>Winogrande (5-shot)
188
  </td>
189
+ <td>87.21
190
+ </td>
191
+ <td>85.95
192
  </td>
193
+ <td>98.55%
194
  </td>
195
  </tr>
196
  <tr>
197
  <td>TruthfulQA (0-shot)
198
  </td>
199
+ <td>64.64
200
  </td>
201
+ <td>64.75
202
  </td>
203
+ <td>100.17%
204
  </td>
205
  </tr>
206
  <tr>
207
  <td><strong>Average</strong>
208
  </td>
209
+ <td><strong>86.63</strong>
210
  </td>
211
+ <td><strong>86.01</strong>
212
  </td>
213
+ <td><strong>99.28%</strong>
214
  </td>
215
  </tr>
216
  </table>