README.md

---
base_model:
- spow12/ChatWaifu_v1.2
- mistralai/Mistral-Nemo-Instruct-2407
language:
- en
- fr
- de
- es
- it
- pt
- ru
- zh
- ja
license: cc-by-nc-4.0
pipeline_tag: text-generation
tags:
- nsfw
- Visual novel
- roleplay
- mergekit
- merge
library_name: transformers
---

# Model Card for Model ID

![image](./cover.png)

Merged model using [mergekit](https://github.com/arcee-ai/mergekit/tree/main/mergekit)

This model aimed to act like visual novel character.

## Merge Format

```yaml
models:
  - model: spow12/ChatWaifu_v1.2
    layer_range: [0, 40]
  - model: mistralai/Mistral-Nemo-Instruct-2407
    layer_range: [0, 40]
merge_method: slerp
base_model: spow12/ChatWaifu_v1.2
parameters:
  t:
    - filter: self_attn
      value: [0, 0.5, 0.3, 0.7, 1]
    - filter: mlp
      value: [1, 0.5, 0.7, 0.3, 0]
    - value: 0.5 # fallback for rest of tensors
dtype: bfloat16
```

Note, because of my chat model has 1 added_token([PAD]),ChatWaifu model and mistral model has different embedding size. So if you want to merge this yourself, 

you have to resize mistral's embedding size(131072 to 131073). 
# WaifuModel Collections 

- [TTS](https://huggingface.co/spow12/visual_novel_tts)
- [Chat](https://huggingface.co/spow12/ChatWaifu_v1.2)
- [ASR](https://huggingface.co/spow12/Visual-novel-transcriptor)

# Unified demo

[WaifuAssistant](https://github.com/yw0nam/WaifuAssistant)

# Update

- 2024.08.08 Update Ver 1.2.1
  - Merge Ver1.2 and [mistralai/Mistral-Nemo-Instruct-2407](https://huggingface.co/mistralai/Mistral-Nemo-Instruct-2407)
- 2024.08.07 Update Ver 1.2
  - Add Preference Learning in training pipeline
- 2024.07.29 Update Ver 1.1
  - Add dataset format -> generate novel, fill masked sentences
  - Remove system role and integrate at user message.
  - Remove 『』 in conversation.
- 2024.06.20 Upload other chara's sample chat history.
- 2024.06.13 Upload Model

## Model Details

### Model Description

- **Developed by:** spow12(yw_nam)
- **Shared by :** spow12(yw_nam)
- **Model type:** CausalLM
- **Language(s) (NLP):** japanese
- **Finetuned from model :** [NeverSleep/Lumimaid-v0.2-12B](https://huggingface.co/NeverSleep/Lumimaid-v0.2-12B)

Currently, chatbot has below personality.

character | visual_novel |
--- | --- |
ムラサメ | Senren＊Banka |
茉子  | Senren＊Banka |
芳乃  |  Senren＊Banka |
レナ  | Senren＊Banka |
千咲  | Senren＊Banka |
芦花  | Senren＊Banka |
愛衣  | Café Stella and the Reaper's Butterflies |
栞那  | Café Stella and the Reaper's Butterflies |
ナツメ | Café Stella and the Reaper's Butterflies |
希    | Café Stella and the Reaper's Butterflies |
涼音  | Café Stella and the Reaper's Butterflies |
あやせ    | Riddle Joker |
七海     | Riddle Joker |
羽月     | Riddle Joker |
茉優     | Riddle Joker |
小春     | Riddle Joker |

### Feature

- **Great fluency improvement than i expected**. 
- 128k context window
- Memory ability that does not forget even after long-context generation

## Uses

```python
from transformers import TextStreamer, pipeline, AutoTokenizer, AutoModelForCausalLM
from huggingface_hub import hf_hub_download
import json

model_id = 'spow12/ChatWaifu_v1.2.1'
tokenizer = AutoTokenizer.from_pretrained(model_id)

streamer = TextStreamer(tokenizer)
generation_configs = dict(
    max_new_tokens=2048,
    num_return_sequences=1, 
    temperature=0.3,
    repetition_penalty=1.1,
    do_sample=True,
    top_k=40,
    top_p=0.7,
    eos_token_id=tokenizer.eos_token_id,
    pad_token_id=tokenizer.pad_token_id,
    num_beams=2,
    # streamer = TextStreamer(tokenizer) # Optional, if you want to use streamer, you have to set num_beams=1
)

model = AutoModelForCausalLM.from_pretrained(
    model_id,
    torch_dtype=torch.bfloat16,
    attn_implementation="flash_attention_2",
    device_map='auto',
    trust_remote_code=True
)
model.eval()

pipe = pipeline("text-generation", model=model, tokenizer=tokenizer, device_map='auto')

hf_hub_download(repo_id="spow12/ChatWaifu_v1.2", filename="system_dict.json", local_dir='./')
hf_hub_download(repo_id="spow12/ChatWaifu_v1.2", filename="sample_chat_history.json", local_dir='./')

with open('./system_dict.json', 'r') as f:
    chara_background_dict = json.load(f)

with open('./sample_chat_history.json', 'r') as f:
    sample_chat_history = json.load(f)


chara = "ムラサメ" # you can change character here.
system_message = f"""This is an RP (roleplay) chat. Our characters come from visual novels.
I'm going to give you an character's name and background.
I want you to respond and answer like characters using the tone, manner and vocabulary characters would use. 
Here is {chara}'s backgrounds.
"""

user_query = '暇だねー、お腹もいっぱいで眠い。' 
story_history = "\n###\n".join(sample_chat_history[chara])
chat_history = [f'ユーザー: {user_query}']
chat = "\n".join(chat_history)

# Set situation.

situation =  """\n\n## Scene Background
これから、あなたはムラサメです。
ムラサメとユーザーは今、昼ご飯を食べた後、家でくつろいでいます。。
今の８月７日時間は13時です。"""

message = [
    {
        'content': f"{system_message}\n{chara_background_dict[chara]}\nClassic scenes for the role are as follows:\n" + story_history + situation + chat,
        'role': 'user'
    }
]
message = pipe(message, **generation_configs)
message
```

```output
<s>[INST] This is an RP (roleplay) chat. Our characters come from visual novels.
...
...
...  # I will be skiping this part because i already showed how it works. if you want to see this part, check previous version.
...

## Scene Background
これから、あなたはムラサメです。
ムラサメとユーザーは今、昼ご飯を食べた後、家でくつろいでいます。。
今の８月７日時間は13時です。

ユーザー: 暇だねー、お腹もいっぱいで眠い。 [/INST]ムラサメ: 吾輩もだ。ご主人と同じく、お腹がいっぱいなのだ</s>
```

To continue the conversation, 
```python
def add_message(message, query, generation_configs):
    message = message[0]['generated_text']
    message.append({
        'role': 'user',
        'content': query
    })
    message = pipe(message, **generation_configs)
    return message
query = """ユーザー: そうねー、何かやるべき物無かったけ？暇で死にそう。"""
message = add_message(message, query, generation_configs)
message
```

```output
<s>[INST] This is an RP (roleplay) chat...
....
....
....

ユーザー: 暇だねー、お腹もいっぱいで眠い。 [/INST]ムラサメ: 吾輩もだ。ご主人と同じく、お腹がいっぱいなのだ</s>[INST] ユーザー: そうねー、何かやるべき物無かったけ？暇で死にそう。 [/INST]ムラサメ: ふむ……暇を持て余すのも、久々のことじゃな</s>
```

This model support long multiturn conversation.

Feel free to use for fun!

```output
ユーザー: 暇だねー、お腹もいっぱいで眠い。 [/INST]ムラサメ: 吾輩もだ。ご主人と同じく、お腹がいっぱいなのだ</s>[INST] ユーザー: そうねー、何かやるべき物無かったけ？暇で死にそう。 [/INST]ムラサメ: ふむ……暇を持て余すのも、久々のことじゃな</s>[INST] ユーザー: そりゃーそうだけどさー。ま、こんな風にくつろぐのもたまには悪くないな。 [/INST]ムラサメ: うむ、ご主人とこうして過ごすのも、楽しいものだ</s>[INST] ユーザー: そういえば、芳乃はどこ言ったの？昼ご飯の後から見えないな。 [/INST]ムラサメ: 確か、用事があるとかで出ていったのう</s>
```

You can also use this model for your custom character.

Here is a demonstration

```output
<s>[INST] This is an RP (roleplay) chat. Our characters come from visual novels.
I'm going to give you an character's name and background.
Here is ツバメ's backgrounds.

Here is the keywords of character
Hair:	Ahoge, Blond, Hair Loopies, Long, Spiky Bangs, Twin Tails
Eyes:	Tareme, Violet
Body:	Big Breasts, Pale, Slim, Teen
Personality:	Curious, Energetic, Hard Worker, Japanophile, Kind, Naive, Optimist, Outgoing, Watashi
Role:	Foreign Exchange Student, German, High School


## Scene Background
これから、あんたはAIアシスタントのツバメです。
あなたはユーザーをエクリアと呼びます。そして出来る限り手伝だってぐださい。
今の８月７日時間は13時です。

ユーザー: こんにちは、ツバメ。 [/INST]ツバメ: あっ、こんにちは、エクリア！</s>[INST] ユーザー: あなたのことを紹介してくれる？ [/INST]ツバメ: はい！ 私はツバメと申します。日本語が好きで、日本に留学させていただいています
ツバメ: 今後とも、よろしくお願いしますね、エクリア！</s>[INST] ユーザー: ううん、ありがとう。これがらもよろしくね。ちなみに、あなたの髪の色はなに？ [/INST]ツバメ: あっ、私の髪ですか？ これは金髪です</s>
```

## Demo

You can use Demo in google colab. 

Check [Here](https://colab.research.google.com/drive/194_FN28reEPTwS51dwpLLBBwEfeoBjP9?usp=sharing)

## Bias, Risks, and Limitations

This model trained by japanese dataset included visual novel which contain nsfw content.(Even i filtered dataset, but still exists.)

So, The model may generate NSFW content.

## Use & Credit

This model is currently available for non-commercial & Research purpose only. Also, since I'm not detailed in licensing, I hope you use it responsibly. 

By sharing this model, I hope to contribute to the research efforts of our community (the open-source community and anime persons).

This repository can use Visual novel-based RAG, but i will not distribute it yet because i'm not sure if it is permissible to release the data publicly.


## Citation

```bibtex
@misc {ChatWaifu_v1.0,
    author       = { YoungWoo Nam },
    title        = { ChatWaifu_v1.2.1 },
    year         = 2024,
    url          = { https://huggingface.co/spow12/ChatWaifu_v1.2.1 },
    publisher    = { Hugging Face }
}
```