readme

README.md

@@ -1,8 +1,40 @@
-### Run Huggingface RWKV6 World Model
+---
+license: apache-2.0
+---
+### Huggingface RWKV Finch 14B Model
 
-> origin pth weight from https://huggingface.co/BlinkDL/rwkv-6-world/blob/main/RWKV-x060-World-7B-v2.1-20240507-ctx4096.pth .
+> HF compatible model for Finch-14B.
 
-#### CPU
+![Finch Bird](./imgs/finch.jpg)
+
+
+> **! Important Note !**
+>
+> The following is the HF transformers implementation of the Finch 14B model. This is meant to be used with the huggingface transformers
+>
+>
+
+
+## Quickstart with the hugging face transformer library
+
+```
+model = AutoModelForCausalLM.from_pretrained("RWKV/v6-Finch-14B-HF", trust_remote_code=True).to(torch.float32)
+tokenizer = AutoTokenizer.from_pretrained("RWKV/v6-Finch-14B-HF", trust_remote_code=True)
+```
+
+## Evaluation
+
+The following demonstrates the improvements from Eagle 7B to Finch 14B
+
+|  | [Eagle 7B](https://huggingface.co/RWKV/v5-Eagle-7B-HF) | [Finch 7B](https://huggingface.co/RWKV/v6-Finch-7B-HF) | [Finch 14B](https://huggingface.co/RWKV/v6-Finch-14B-HF) |
+| --- | --- | --- | --- |
+| [ARC](https://github.com/EleutherAI/lm-evaluation-harness/tree/main/lm_eval/tasks/arc) | 39.59% | 41.47% | 46.33% |
+| [HellaSwag](https://github.com/EleutherAI/lm-evaluation-harness/tree/main/lm_eval/tasks/hellaswag) | 53.09% | 55.96% | 57.69% |
+| [MMLU](https://github.com/EleutherAI/lm-evaluation-harness/tree/main/lm_eval/tasks/mmlu) | 30.86% | 41.70% | 56.05% |
+| [Truthful QA](https://github.com/EleutherAI/lm-evaluation-harness/tree/main/lm_eval/tasks/truthfulqa) | 33.03% | 34.82% | 39.27% |
+| [Winogrande](https://github.com/EleutherAI/lm-evaluation-harness/tree/main/lm_eval/tasks/winogrande) | 67.56% | 71.19% | 74.43% |
+
+#### Running on CPU via HF transformers
 
 ```python
 import torch
@@ -27,8 +59,8 @@ User: {instruction}
 Assistant:"""
 
 
-model = AutoModelForCausalLM.from_pretrained("RWKV/rwkv-6-world-7b", trust_remote_code=True).to(torch.float32)
-tokenizer = AutoTokenizer.from_pretrained("RWKV/rwkv-6-world-7b", trust_remote_code=True, padding_side='left', pad_token="<s>")
+model = AutoModelForCausalLM.from_pretrained("RWKV/v5-Eagle-7B-HF", trust_remote_code=True).to(torch.float32)
+tokenizer = AutoTokenizer.from_pretrained("RWKV/v5-Eagle-7B-HF", trust_remote_code=True)
 
 text = "请介绍北京的旅游景点"
 prompt = generate_prompt(text)
@@ -58,7 +90,7 @@ Assistant: 北京是中国的首都，拥有众多的旅游景点，以下是其
 8. 天坛：是中国古代皇家
 ```
 
-#### GPU
+#### Running on GPU via HF transformers
 
 ```python
 import torch
@@ -83,8 +115,8 @@ User: {instruction}
 Assistant:"""
 
 
-model = AutoModelForCausalLM.from_pretrained("RWKV/rwkv-6-world-7b", trust_remote_code=True, torch_dtype=torch.float16).to(0)
-tokenizer = AutoTokenizer.from_pretrained("RWKV/rwkv-6-world-7b", trust_remote_code=True, padding_side='left', pad_token="<s>")
+model = AutoModelForCausalLM.from_pretrained("RWKV/v5-Eagle-7B-HF", trust_remote_code=True, torch_dtype=torch.float16).to(0)
+tokenizer = AutoTokenizer.from_pretrained("RWKV/v5-Eagle-7B-HF", trust_remote_code=True)
 
 text = "介绍一下大熊猫"
 prompt = generate_prompt(text)
@@ -130,8 +162,8 @@ User: {instruction}
 
 Assistant:"""
 
-model = AutoModelForCausalLM.from_pretrained("RWKV/rwkv-6-world-7b", trust_remote_code=True).to(torch.float32)
-tokenizer = AutoTokenizer.from_pretrained("RWKV/rwkv-6-world-7b", trust_remote_code=True, padding_side='left', pad_token="<s>")
+model = AutoModelForCausalLM.from_pretrained("RWKV/v5-Eagle-7B-HF", trust_remote_code=True).to(torch.float32)
+tokenizer = AutoTokenizer.from_pretrained("RWKV/v5-Eagle-7B-HF", trust_remote_code=True)
 
 texts = ["请介绍北京的旅游景点", "介绍一下大熊猫", "乌兰察布"]
 prompts = [generate_prompt(text) for text in texts]
@@ -172,3 +204,16 @@ User: 乌兰察布
 
 Assistant: 乌兰察布是中国新疆维吾尔自治区的一个县级市，位于新疆维吾尔自治区中部，是新疆的第二大城市。乌兰察布市是新疆的第一大城市，也是新疆的重要城市之一。乌兰察布市是新疆的经济中心，也是新疆的重要交通枢纽之一。乌兰察布市的人口约为2.5万人，其中汉族占绝大多数。乌
 ```
+
+## Links
+- [Our wiki](https://wiki.rwkv.com)
+- [Recursal.AI Cloud Platform](https://recursal.ai)
+- [Featherless Inference](https://featherless.ai/models/RWKV/Finch-14B)
+- [Blog article, detailing our model launch](https://blog.rwkv.com/p/rwkv-v6-finch-14b-is-here)
+
+## Acknowledgement
+We are grateful for the help and support from the following key groups:
+
+- [Recursal.ai](https://recursal.ai) team for financing the GPU resources, and managing the training of this foundation model - you can run the Finch line of RWKV models on their cloud / on-premise platform today.
+- EleutherAI for their support, especially in the v5/v6 Eagle/Finch paper
+- Linux Foundation AI & Data group for supporting and hosting the RWKV project