Lewdiculous/Azure_Dusk-v0.2-GGUF-IQ-Imatrix

Model name:
Azure_Dusk-v0.2

Description:
"Following up on Crimson_Dawn-v0.2 we have Azure_Dusk-v0.2! Training on Mistral-Nemo-Base-2407 this time I've added significantly more data, as well as trained using RSLoRA as opposed to regular LoRA. Another key change is training on ChatML as opposed to Mistral Formatting."
– by Author.

As described, use the ChatML prompt format.

Presets:
You can use ChatML presets within SillyTavern and adjust from there.
Alternatively, check out Virt-io's ChatML v1.9 presets here, make sure you read the repository page for how to use them properly.

Original model page:
https://huggingface.co/Epiculous/Azure_Dusk-v0.2

Quantized using llama.cpp-b3733:

1. Base⇢ Convert-GGUF(FP16)⇢ Generate-Imatrix-Data(FP16)
2. Base⇢ Convert-GGUF(BF16)⇢ Use-Imatrix-Data(FP16)⇢ Quantize-GGUF(Imatrix-Quants)