BlinkDL
/

rwkv-4-pile-1b5

Text Generation

Model card Files Files and versions Community

BlinkDL commited on Feb 22, 2023

Commit

7dc0579

•

1 Parent(s): 6e38bc9

Update README.md

Files changed (1) hide show

README.md +1 -1

README.md CHANGED Viewed

@@ -24,7 +24,7 @@ ctx_len = 1024
 n_layer = 24
 n_embd = 2048
-New checkpoint: RWKV-4-Pile-1B5-20220929-ctx4096.pth : Fine-tuned to ctx_len = 4096
 Final checkpoint: RWKV-4-Pile-1B5-20220903-8040.pth : Trained on the Pile for 332B tokens.
 * Pile loss 2.0415

 n_layer = 24
 n_embd = 2048
+New checkpoint: RWKV-4-Pile-1B5-20220929-ctx4096.pth : Fine-tuned to ctx_len = 4096. Use it only when your ctxlen is long. Might be slightly weaker for short ctxlens.
 Final checkpoint: RWKV-4-Pile-1B5-20220903-8040.pth : Trained on the Pile for 332B tokens.
 * Pile loss 2.0415