GGUF versions?
#1
by
froggeric
- opened
Sound very interesting. I have been doing extensive tests on solar variants, and you made a great choice. Hopefully the frankenmerge also makes it smarter by increasing the size. I would love to test it, but I need a GGUF version
Hi, thanks for your interest in the model! I've started uploading GGUF versions here: https://huggingface.co/vicgalle/franken-SOLAR-18B-v1.0-GGUF
Since with transformers I'd just tested the 4bit variant, that's the first quantisation I've uploaded there. In my tests for creative writing and rp, this model was much better than each of its two components, so I hope nothing is lost when converting to GGUF.
I'll be uploading more quantisations during the next hours.
Please, if you try it, let me know how it works for you!