File size: 1,301 Bytes
1a3002d
 
3efc0d5
 
 
 
 
1a3002d
3efc0d5
1a3002d
3efc0d5
1a3002d
3efc0d5
1a3002d
3efc0d5
1a3002d
3efc0d5
1a3002d
3efc0d5
1a3002d
3efc0d5
 
 
 
 
 
 
 
1a3002d
3efc0d5
1a3002d
3efc0d5
 
1a3002d
3efc0d5
 
 
1a3002d
3efc0d5
1a3002d
3efc0d5
 
 
 
1a3002d
3efc0d5
1a3002d
3efc0d5
1a3002d
3efc0d5
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
---
library_name: transformers
license: apache-2.0
base_model:
- nbeerbower/Flammen-Mahou-mistral-7B
datasets:
- flammenai/MahouMix-v1
---
![image/png](https://huggingface.co/flammenai/Mahou-1.0-mistral-7B/resolve/main/mahou1.png)

# Mahou-1.3-mistral-7B

Mahou is our attempt to build a production-ready conversational/roleplay LLM.

Future versions will be released iteratively and finetuned from flammen.ai conversational data.

### Chat Format

This model has been trained to use ChatML format.

```
<|im_start|>system
{{system}}<|im_end|>
<|im_start|>{{char}}
{{message}}<|im_end|>
<|im_start|>{{user}}
{{message}}<|im_end|>
```

### Roleplay Format

- Speech without quotes.
- Actions in `*asterisks*`

```
*leans against wall cooly* so like, i just casted a super strong spell at magician academy today, not gonna lie, felt badass.
```

### ST Settings

1. Use ChatML for the Context Template.
2. Enable Instruct Mode.
3. Use the [Mahou preset](https://huggingface.co/datasets/flammenai/Mahou-ST-ChatML-Instruct/raw/main/Mahou.json).
4. Recommended: Add newline as a stopping string: `["\n"]`

### Method

Finetuned for 10 epochs using an A100 on Google Colab.

[Fine-tune Llama 3 with ORPO](https://huggingface.co/blog/mlabonne/orpo-llama-3) - [Maxime Labonne](https://huggingface.co/mlabonne)