Update README.md
Browse files
README.md
CHANGED
@@ -187,16 +187,19 @@ We think there is too much secrecy around what data is being used, and different
|
|
187 |
|
188 |
### The Good
|
189 |
We found that increasing the amount of ranks from 64 to 256 has reduced repetition but also led to the language used resembling Claude more than the 64 rank version. No worries, it's still far enough from Claude.
|
190 |
-
<br
|
191 |
-
<br
|
192 |
<br>We found that increasing the amount of data from 1K to 6.5K reduced repetition aswell.
|
193 |
-
|
194 |
-
<br
|
|
|
|
|
195 |
|
196 |
### Improvements for Next Run
|
197 |
|
198 |
Formatting can break sometimes.
|
199 |
-
Repetition can become an issue with certain types of prompts. Removing system helps.
|
|
|
200 |
|
201 |
### Comments about training
|
202 |
|
|
|
187 |
|
188 |
### The Good
|
189 |
We found that increasing the amount of ranks from 64 to 256 has reduced repetition but also led to the language used resembling Claude more than the 64 rank version. No worries, it's still far enough from Claude.
|
190 |
+
<br>**Model follows "OOC:" prompts religiously. Exceptional!**
|
191 |
+
<br>It also led to **increased coherency but reduced system prompt following (when not OOC)**, likely because the model started diverging more away from L3 8B Instruct.
|
192 |
<br>We found that increasing the amount of data from 1K to 6.5K reduced repetition aswell.
|
193 |
+
|
194 |
+
<br>The model is uncensored for RP. For Instruct it needs 2-3 words of prefill for the first message.
|
195 |
+
<br>The **prose is much better** and **the style range is huge** than other synthetic data generations. The model also demonstrates increased **style copying abilities** (from fewshot) likely a result of human longform data and varying writing styles found in WritingPrompts.
|
196 |
+
<br>The model is **exceptional at being creative in roleplaying**, knows different persona's and even a single character will change persona in different contexts, persona is tied to last few messages rather than system message or character card. **This is great as it often means the model can do impressive things without you needing to explicitly specify.**
|
197 |
|
198 |
### Improvements for Next Run
|
199 |
|
200 |
Formatting can break sometimes.
|
201 |
+
<br>Repetition can become an issue with certain types of prompts. Removing system helps.
|
202 |
+
<br>In some contexts the model is "all over the place" and doesn't stick to a coherent narrative. I need to study this further as its a complex trait which manifests in different quantities and can be good or bad depending on what the user wants to get out of the model.
|
203 |
|
204 |
### Comments about training
|
205 |
|