Update README.md
Browse files
README.md
CHANGED
|
@@ -2,7 +2,8 @@ GPT-R [Ronin]
|
|
| 2 |
|
| 3 |
This is an experimental model containing a parameter-wise 60/40 blend (weighted average) of the weights of ppo_hh_gpt-j and GPT-JT-6B-v1.
|
| 4 |
|
| 5 |
-
-Intended Merge Value
|
|
|
|
| 6 |
As with fine-tuning, merging weights does not add information but transforms it, therefore it is important to consider trade-offs.
|
| 7 |
GPT-Ronin combines ppo_hh_gpt-j and GPT-JT; both technical
|
| 8 |
achievements are blended with the intent to elevate the strengths of
|
|
@@ -27,7 +28,7 @@ by instruct.
|
|
| 27 |
Merge tested using KoboldAI with Nucleus Sampling Top-P set to 0.7, Temperature at 0.5, and Repetition Penalty at 1.14; extra samplers
|
| 28 |
disabled.
|
| 29 |
|
| 30 |
-
-Credits
|
| 31 |
|
| 32 |
Core Model:
|
| 33 |
https://huggingface.co/EleutherAI/gpt-j-6B
|
|
|
|
| 2 |
|
| 3 |
This is an experimental model containing a parameter-wise 60/40 blend (weighted average) of the weights of ppo_hh_gpt-j and GPT-JT-6B-v1.
|
| 4 |
|
| 5 |
+
-Intended Merge Value-
|
| 6 |
+
|
| 7 |
As with fine-tuning, merging weights does not add information but transforms it, therefore it is important to consider trade-offs.
|
| 8 |
GPT-Ronin combines ppo_hh_gpt-j and GPT-JT; both technical
|
| 9 |
achievements are blended with the intent to elevate the strengths of
|
|
|
|
| 28 |
Merge tested using KoboldAI with Nucleus Sampling Top-P set to 0.7, Temperature at 0.5, and Repetition Penalty at 1.14; extra samplers
|
| 29 |
disabled.
|
| 30 |
|
| 31 |
+
-Credits To-
|
| 32 |
|
| 33 |
Core Model:
|
| 34 |
https://huggingface.co/EleutherAI/gpt-j-6B
|