152334H commited on
Commit
e247265
1 Parent(s): 1b17110

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +17 -10
README.md CHANGED
@@ -18,7 +18,8 @@ model-index:
18
  value: 73.04
19
  name: normalized accuracy
20
  source:
21
- url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=152334H/miqu-1-70b-sf
 
22
  name: Open LLM Leaderboard
23
  - task:
24
  type: text-generation
@@ -34,7 +35,8 @@ model-index:
34
  value: 88.61
35
  name: normalized accuracy
36
  source:
37
- url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=152334H/miqu-1-70b-sf
 
38
  name: Open LLM Leaderboard
39
  - task:
40
  type: text-generation
@@ -51,7 +53,8 @@ model-index:
51
  value: 75.49
52
  name: accuracy
53
  source:
54
- url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=152334H/miqu-1-70b-sf
 
55
  name: Open LLM Leaderboard
56
  - task:
57
  type: text-generation
@@ -67,7 +70,8 @@ model-index:
67
  - type: mc2
68
  value: 69.38
69
  source:
70
- url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=152334H/miqu-1-70b-sf
 
71
  name: Open LLM Leaderboard
72
  - task:
73
  type: text-generation
@@ -84,7 +88,8 @@ model-index:
84
  value: 85.32
85
  name: accuracy
86
  source:
87
- url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=152334H/miqu-1-70b-sf
 
88
  name: Open LLM Leaderboard
89
  - task:
90
  type: text-generation
@@ -101,8 +106,11 @@ model-index:
101
  value: 67.7
102
  name: accuracy
103
  source:
104
- url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=152334H/miqu-1-70b-sf
 
105
  name: Open LLM Leaderboard
 
 
106
  ---
107
  this is [miqu-1-70b](https://huggingface.co/miqudev/miqu-1-70b), dequantised from q5 to f16 && transposed to pytorch. shapes have been rotated less wrongly than in [alpindale/miqu-1-70b-pytorch](https://huggingface.co/alpindale/miqu-1-70b-pytorch/tree/main)
108
 
@@ -110,7 +118,7 @@ usage
110
  ```python
111
  from transformers import LlamaForCausalLM as LLM, LlamaTokenizer as LT
112
 
113
- lt = LT.from_pretrained("NousResearch/Llama-2-7b-hf")
114
  t = lt("[INST] eloquent high camp prose about a cute catgirl [/INST]", return_tensors='pt').input_ids.cuda()
115
 
116
  llm = LLM.from_pretrained("152334H/miqu-1-70b-sf", device_map='auto') # note: you may need many gpus for this
@@ -144,7 +152,7 @@ So let us raise our teacups in honor of this fabulous feline, this queen of camp
144
 
145
  ![](https://thicc-af.mywaifulist.moe/waifus/miku-nakano-the-quintessential-quintuplets/phUEiEhPOL75GTDLncGy2dUbkDVMfYExZ2A1RBeQ.png?class=thumbnail)
146
 
147
- some benchmarks
148
 
149
  ```
150
  | Tasks |Version|Filter|n-shot| Metric |Value | |Stderr|
@@ -245,5 +253,4 @@ Detailed results can be found [here](https://huggingface.co/datasets/open-llm-le
245
  |MMLU (5-Shot) |75.49|
246
  |TruthfulQA (0-shot) |69.38|
247
  |Winogrande (5-shot) |85.32|
248
- |GSM8k (5-shot) |67.70|
249
-
 
18
  value: 73.04
19
  name: normalized accuracy
20
  source:
21
+ url: >-
22
+ https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=152334H/miqu-1-70b-sf
23
  name: Open LLM Leaderboard
24
  - task:
25
  type: text-generation
 
35
  value: 88.61
36
  name: normalized accuracy
37
  source:
38
+ url: >-
39
+ https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=152334H/miqu-1-70b-sf
40
  name: Open LLM Leaderboard
41
  - task:
42
  type: text-generation
 
53
  value: 75.49
54
  name: accuracy
55
  source:
56
+ url: >-
57
+ https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=152334H/miqu-1-70b-sf
58
  name: Open LLM Leaderboard
59
  - task:
60
  type: text-generation
 
70
  - type: mc2
71
  value: 69.38
72
  source:
73
+ url: >-
74
+ https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=152334H/miqu-1-70b-sf
75
  name: Open LLM Leaderboard
76
  - task:
77
  type: text-generation
 
88
  value: 85.32
89
  name: accuracy
90
  source:
91
+ url: >-
92
+ https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=152334H/miqu-1-70b-sf
93
  name: Open LLM Leaderboard
94
  - task:
95
  type: text-generation
 
106
  value: 67.7
107
  name: accuracy
108
  source:
109
+ url: >-
110
+ https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=152334H/miqu-1-70b-sf
111
  name: Open LLM Leaderboard
112
+ language:
113
+ - en
114
  ---
115
  this is [miqu-1-70b](https://huggingface.co/miqudev/miqu-1-70b), dequantised from q5 to f16 && transposed to pytorch. shapes have been rotated less wrongly than in [alpindale/miqu-1-70b-pytorch](https://huggingface.co/alpindale/miqu-1-70b-pytorch/tree/main)
116
 
 
118
  ```python
119
  from transformers import LlamaForCausalLM as LLM, LlamaTokenizer as LT
120
 
121
+ lt = LT.from_pretrained("152334H/miqu-1-70b-sf")
122
  t = lt("[INST] eloquent high camp prose about a cute catgirl [/INST]", return_tensors='pt').input_ids.cuda()
123
 
124
  llm = LLM.from_pretrained("152334H/miqu-1-70b-sf", device_map='auto') # note: you may need many gpus for this
 
152
 
153
  ![](https://thicc-af.mywaifulist.moe/waifus/miku-nakano-the-quintessential-quintuplets/phUEiEhPOL75GTDLncGy2dUbkDVMfYExZ2A1RBeQ.png?class=thumbnail)
154
 
155
+ ## some benchmarks
156
 
157
  ```
158
  | Tasks |Version|Filter|n-shot| Metric |Value | |Stderr|
 
253
  |MMLU (5-Shot) |75.49|
254
  |TruthfulQA (0-shot) |69.38|
255
  |Winogrande (5-shot) |85.32|
256
+ |GSM8k (5-shot) |67.70|