chargoddard commited on
Commit
322a2e1
1 Parent(s): d74ae6c

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +18 -2
README.md CHANGED
@@ -14,6 +14,22 @@ tags:
14
 
15
  Experimenting with dataset ratios. Intended to be a roleplay-focused model with some smarts and good long-context recall.
16
 
17
- Not recommended for use yet! WIP, untested, questionable quality. This place is not a place of honor. No highly esteemed deed is commemorated here.
18
 
19
- Uses alpaca prompt format.
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
14
 
15
  Experimenting with dataset ratios. Intended to be a roleplay-focused model with some smarts and good long-context recall.
16
 
17
+ Not sure if I've succeeded on the roleplay front, but something sure went right! Currently the #4 7B model on the leaderboard as of 11/30/2023. Going to riff on this and see where it goes.
18
 
19
+ | model | Average | ARC | HellaSwag | MMLU | TruthfulQA | Winogrande | GSM8K | DROP |
20
+ | --- | --- | --- | --- | --- | --- | --- | --- | --- |
21
+ | fblgit/juanako-7b-UNA | 59.91 | 68.17 | 85.34 | 62.47 | 65.13 | 78.85 | 20.7 | 38.74 |
22
+ | Intel/neural-chat-7b-v3-1 | 59.06 | 66.21 | 83.64 | 62.37 | 59.65 | 78.14 | 19.56 | 43.84 |
23
+ | Weyaxi/OpenHermes-2.5-neural-chat-7b-v3-1-7B | 58.6 | 66.55 | 84.47 | 63.34 | 61.22 | 78.37 | 23.58 | 32.66 |
24
+ | **chargoddard/loyal-piano-m7** | 58.42 | 66.72 | 85.03 | 64.43 | 60.03 | 79.08 | 25.7 | 27.92 |
25
+ | Gryphe/MythoMist7b | 58.26 | 65.87 | 83.55 | 62.32 | 59.98 | 78.06 | 20.24 | 37.82 |
26
+
27
+
28
+ Dataset composition:
29
+ | dataset | rows used | percent of total |
30
+ | --- | --- | --- |
31
+ | PIPPA | 14.6k | 43% |
32
+ | summarize_from_feedback | 9k | 26% |
33
+ | orca_mini_v1_dataset | 5.6k | 17% |
34
+ | rpguild | 2.86k | 8% |
35
+ | LimaRP | 2k | 6% |