Update README.md
Browse files
README.md
CHANGED
@@ -18,14 +18,12 @@ pipeline_tag: text-generation
|
|
18 |
* **Library**: [HuggingFace Transformers](https://github.com/huggingface/transformers)
|
19 |
* **License**: Fine-tuned checkpoints is licensed under the Non-Commercial Creative Commons license ([CC BY-NC-4.0](https://creativecommons.org/licenses/by-nc/4.0/))
|
20 |
* **Where to send comments**: Instructions on how to provide feedback or comments on a model can be found by opening an issue in the [Hugging Face community's model repository](https://huggingface.co/upstage/Llama-2-70b-instruct-1024/discussions)
|
21 |
-
* **Contact**: For questions and comments about the model, please email
|
22 |
|
23 |
## Dataset Details
|
24 |
|
25 |
### Used Datasets
|
26 |
-
|
27 |
-
|
28 |
-
> No other data was used except for the dataset mentioned above
|
29 |
|
30 |
### Prompt Template
|
31 |
```
|
@@ -53,17 +51,18 @@ We used the [lm-evaluation-harness repository](https://github.com/EleutherAI/lm-
|
|
53 |
|
54 |
|
55 |
### Main Results
|
56 |
-
| Model | Average | ARC | HellaSwag | MMLU | TruthfulQA |
|
57 |
-
|
58 |
-
| Llama-2-70b-instruct-v2 (Ours, Local Reproduction) | 72.7 | 71.6 | 87.7 | 69.7 | 61.6 |
|
59 |
-
|
|
60 |
-
|
|
61 |
-
|
|
62 |
-
|
|
63 |
-
| llama-30b-instruct-2048 (Ours,
|
64 |
-
| llama-30b-instruct (Ours,
|
65 |
-
| llama-
|
66 |
-
|
|
|
|
67 |
|
68 |
### Scripts
|
69 |
- Prepare evaluation environments:
|
|
|
18 |
* **Library**: [HuggingFace Transformers](https://github.com/huggingface/transformers)
|
19 |
* **License**: Fine-tuned checkpoints is licensed under the Non-Commercial Creative Commons license ([CC BY-NC-4.0](https://creativecommons.org/licenses/by-nc/4.0/))
|
20 |
* **Where to send comments**: Instructions on how to provide feedback or comments on a model can be found by opening an issue in the [Hugging Face community's model repository](https://huggingface.co/upstage/Llama-2-70b-instruct-1024/discussions)
|
21 |
+
* **Contact**: For questions and comments about the model, please email [[email protected]](mailto:[email protected])
|
22 |
|
23 |
## Dataset Details
|
24 |
|
25 |
### Used Datasets
|
26 |
+
- Orca-style dataset
|
|
|
|
|
27 |
|
28 |
### Prompt Template
|
29 |
```
|
|
|
51 |
|
52 |
|
53 |
### Main Results
|
54 |
+
| Model | H4 Average | ARC | HellaSwag | MMLU | TruthfulQA | | MT_Bench |
|
55 |
+
|-----------------------------------------------|---------|-------|-----------|-------|------------|-------|----------|
|
56 |
+
| [Llama-2-70b-instruct-v2 (Ours, Local Reproduction)](https://huggingface.co/upstage/Llama-2-70b-instruct-v2) | 72.7 | 71.6 | 87.7 | 69.7 | 61.6 | | 7.440625 |
|
57 |
+
| **Llama-2-70b-instruct (Ours, Open LLM Leaderboard)** | **72.3** | **70.9** | **87.5** | **69.8** | **61.0** | | |
|
58 |
+
| **Llama-2-70b-instruct (Ours, Local Reproduction)** | **72.0** | **70.7** | **87.4** | **69.3** | **60.7** | | **7.24375** |
|
59 |
+
| llama-65b-instruct (Ours, Local Reproduction) | 69.4 | 67.6 | 86.5 | 64.9 | 58.8 | | |
|
60 |
+
| Llama-2-70b-hf | 67.3 | 67.3 | 87.3 | 69.8 | 44.9 | | |
|
61 |
+
| [llama-30b-instruct-2048 (Ours, Open LLM Leaderboard)](https://huggingface.co/upstage/llama-30b-instruct) | 67.0 | 64.9 | 84.9 | 61.9 | 56.3 | | |
|
62 |
+
| llama-30b-instruct-2048 (Ours, Local Reproduction) | 67.0 | 64.9 | 85.0 | 61.9 | 56.0 | | 6.88125 |
|
63 |
+
| llama-30b-instruct (Ours, Open LLM Leaderboard) | 65.2 | 62.5 | 86.2 | 59.4 | 52.8 | | |
|
64 |
+
| [llama-65b](https://huggingface.co/upstage/llama-65b-instruct) | 64.2 | 63.5 | 86.1 | 63.9 | 43.4 | | |
|
65 |
+
| falcon-40b-instruct | 63.4 | 61.6 | 84.3 | 55.4 | 52.5 | | |
|
66 |
|
67 |
### Scripts
|
68 |
- Prepare evaluation environments:
|