rwkv-4-world

README.md

@@ -29,14 +29,12 @@ datasets:
 
 RWKV-4 trained on 100+ world languages (70% English, 15% multilang, 15% code).
 
-World = Some_Pile + Some_RedPajama + Some_OSCAR + All_Wikipedia + All_ChatGPT_Data_I_can_find
-
-XXXtuned = finetune of World on MC4, OSCAR, wiki, etc.
+Some_Pile + Some_RedPajama + Some_OSCAR + All_Wikipedia + All_ChatGPT_Data_I_can_find
 
 How to use:
-* use https://github.com/josStorer/RWKV-Runner for GUI
-* use latest rwkv pip package (0.8.0+)
-* use https://github.com/BlinkDL/ChatRWKV/blob/main/v2/benchmark_world.py and https://github.com/BlinkDL/ChatRWKV/blob/main/API_DEMO_WORLD.py to test it
+* use latest rwkv pip package (0.7.4+)
+* use https://github.com/BlinkDL/ChatRWKV/blob/main/v2/benchmark_world.py to test it
+* larger models are stronger even though not fully trained yet
 
 The differences between World & Raven:
 * set pipeline = PIPELINE(model, "rwkv_vocab_v20230424") instead of 20B_tokenizer.json (EXACTLY AS WRITTEN HERE. "rwkv_vocab_v20230424" is included in rwkv 0.7.4+)

RWKV-4-World-ARAtuned-7B-v1-20230803-ctx4096.pth → RWKV-4-World-CHNtuned-7B-v1-OnlyForTest_49%_trained-20230703-ctx4096.pth

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:b1ce4cd0bd3b0c9ff6f5ecfcc0a86219aa7f61808f11a53e20ecafb8ca3397ca
+oid sha256:1050d59453c8292bff2a046c43bd20893b8cdb91031fb656c113c6c02131001b
 size 15035393448

RWKV-4-World-JPNtuned-7B-v1-20230718-ctx4096.pth

@@ -1,3 +0,0 @@
-version https://git-lfs.github.com/spec/v1
-oid sha256:3e4c7664ce893ac1f6bb59cd76664fb5c872cb076bb82dbd534db0555b6e9fa5
-size 15035393458

RWKV-4-World-CHNtuned-7B-v1-20230709-ctx4096.pth → RWKV-4-World-JPNtuned-7B-v1-OnlyForTest_10%_trained-20230704-ctx4096.pth

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:52d33e8352a40158d21425fee4f68df1515d6324056f788d2c78a366ef578ffa
-size 15035393458
+oid sha256:4b1debccdb74bec1620764164077709dd5f8514d399e25e0f6824c127dfbdac3
+size 15035393443