File size: 891 Bytes
7dbb6e9 59a7800 0691701 7dbb6e9 59a7800 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 |
---
tags:
- gguf
- mistral
- conversational
- iMat
license: cc-by-4.0
---
<img src="https://i.imgur.com/P68dXux.png" width="400"/>
# Open_Gpt4_8x7B_v0.2 iMat GGUF
Open_Gpt4_8x7B_v0.2 iMat GGUF quantized from fp16 with love.
* Quantizations made possible using mixtral-8x7b-instruct-v0.1.imatrix file from [this](https://huggingface.co/datasets/ikawrakow/imatrix-from-wiki-train) repo (special thanks to [ikawrakow](https://huggingface.co/ikawrakow) again)
<b>Legacy quants (i.e. Q8, Q5_K_M) in this repo have all been enhanced with importance matrix calculation. These quants show improved KL-Divergence over their static counterparts.</b>
All files have been tested for your safety and convenience. No need to clone the entire repo, just pick the quant that's right for you.
For more information on latest iMatrix quants see this PR - https://github.com/ggerganov/llama.cpp/pull/5747 |