chargoddard
commited on
Commit
•
c0cd0b1
1
Parent(s):
e4a68c6
Update README.md
Browse files
README.md
CHANGED
@@ -13,4 +13,11 @@ tags:
|
|
13 |
Trained on a flavorful melange of the WizardLM, Airoboros, and Wizard Vicuna datasets.
|
14 |
This model was trained using both linear and NTK-aware RoPE scaling in tandem. When loading, ensure that `compress_pos_emb` (or `scale`) is set to 2, and `alpha_value` is set to 4. *Both* values must be set.
|
15 |
|
16 |
-
Expect context length of up to 8192 to work for sure. It will probably maintain coherence into the ~12k range, but I have not tested that.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
13 |
Trained on a flavorful melange of the WizardLM, Airoboros, and Wizard Vicuna datasets.
|
14 |
This model was trained using both linear and NTK-aware RoPE scaling in tandem. When loading, ensure that `compress_pos_emb` (or `scale`) is set to 2, and `alpha_value` is set to 4. *Both* values must be set.
|
15 |
|
16 |
+
Expect context length of up to 8192 to work for sure. It will probably maintain coherence into the ~12k range, but I have not tested that.
|
17 |
+
|
18 |
+
Prompt format is vicuna 1.1:
|
19 |
+
```
|
20 |
+
<whatever nonsense system prompt you want>
|
21 |
+
USER: ...
|
22 |
+
ASSISTANT: ...
|
23 |
+
```
|