|
--- |
|
license: apache-2.0 |
|
language: |
|
- zh |
|
library_name: transformers |
|
tags: |
|
- llama |
|
- llama3 |
|
- dpo-zh |
|
- emoji |
|
--- |
|
|
|
Github：https://github.com/CrazyBoyM/llama3-Chinese-chat |
|
放出训练配方细节供网友参考分享： |
|
DPO(beta 0.5) + lora rank128, alpha256 + 打开"lm_head", "input_layernorm", "post_attention_layernorm", "norm"层训练。 |
|
偏好中文和emoji表情，且不损伤原instruct版模型能力。 |
|
|
|
 |
|
|
|
Git下载 |
|
``` |
|
#Git模型下载 |
|
git clone https://www.modelscope.cn/baicai003/Llama3-Chinese-instruct-DPO-beta0.5.git |
|
``` |
