File size: 528 Bytes
8ea5dd9 b52e609 8ea5dd9 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 |
---
license: cc-by-nc-4.0
---
# Noromaid-v0.4-Mixtral-Instruct-8x7b-Zloss 3.5bpw
Exllama quant of [NeverSleep/Noromaid-v0.4-Mixtral-Instruct-8x7b-Zloss](https://huggingface.co/NeverSleep/Noromaid-v0.4-Mixtral-Instruct-8x7b-Zloss)
You will need 24gb of vram to run this model at about half context (16k, you can probably go a bit higher too)
### Prompt format: Chatml
```
<|im_start|>system
{sysprompt}<|im_end|>
<|im_start|>user
{input}<|im_end|>
<|im_start|>assistant
{output}<|im_end|>
```
### Contact
Kooten on discord.
|