mobiuslabsgmbh
/

Llama-2-7b-chat-hf_1bitgs8_hqq

Text Generation

Model card Files Files and versions Community

mobicham commited on Mar 26

Commit

4b20f21

•

1 Parent(s): f0c7a8b

Update README.md

Files changed (1) hide show

README.md +9 -9

README.md CHANGED Viewed

@@ -28,19 +28,19 @@ The adapter was trained via SFT on random subsets of the following:
 ## Performance
 | Models            | Llama2-7B (fp16)| Llama2-7B (HQQ-1bit)| Llama2-7B (HQQ+-1bit)| Quip# (2bit)|
 |-------------------|------------------|------------------|------------------|------------------|
-| Wiki Perpexlity   | 5.18             |             9866 |      <b>8.53</b> |         8.54     |
-| VRAM (GB)         |    13.5          |      <b>1.76</b>  |    1.85          |         2.72     |
-| forward time (sec)|   <b>0.1<b>      |    0.231         |     0.257         |      0.353       |
 | Models            | Llama2-7B-chat (fp16)| Llama2-7B-chat (HQQ-1bit)| Llama2-7B-chat (HQQ+-1bit)|
 |-------------------|------------------|------------------|------------------|
-| ARC (25-shot)     |    53.67         | 21.59            |  31.14  |
 | HellaSwag (10-shot)|   78.56         |  25.66           |  52.96  |
-| MMLU (5-shot)     |    48.16         |                  |  26.54  |
-| TruthfulQA-MC2    |    45.32         | 47.81            | 43.16   |
-| Winogrande (5-shot)|   72.53         | 49.72            | 60.54   |
-| GSM8K (5-shot)    |    23.12         |                  | 11   |
-| Average           |    53.56         |                  | 37.56   |
 ## Usage
 First, install the latest version of <a href="https://github.com/mobiusml/hqq/">HQQ</a>:

 ## Performance
 | Models            | Llama2-7B (fp16)| Llama2-7B (HQQ-1bit)| Llama2-7B (HQQ+-1bit)| Quip# (2bit)|
 |-------------------|------------------|------------------|------------------|------------------|
+| Wiki Perpexlity   |    5.18          |      9866        |    <b>8.53</b>   |      8.54        |
+| VRAM (GB)         |    13.5          |   <b>1.76</b>    |    1.85          |      2.72        |
+| forward time (sec)|   <b>0.1<b>      |    0.231         |     0.257        |      0.353       |
 | Models            | Llama2-7B-chat (fp16)| Llama2-7B-chat (HQQ-1bit)| Llama2-7B-chat (HQQ+-1bit)|
 |-------------------|------------------|------------------|------------------|
+| ARC (25-shot)     |    53.67         |  21.59           |  31.14  |
 | HellaSwag (10-shot)|   78.56         |  25.66           |  52.96  |
+| MMLU (5-shot)     |    48.16         |  25.08           |  26.54  |
+| TruthfulQA-MC2    |    45.32         |  47.81           |  43.16  |
+| Winogrande (5-shot)|   72.53         |  49.72           |  60.54  |
+| GSM8K (5-shot)    |    23.12         |  0               |  11     |
+| Average           |    53.56         |  28.31           |  37.56  |
 ## Usage
 First, install the latest version of <a href="https://github.com/mobiusml/hqq/">HQQ</a>: