quantize gguf models
I appreciate the team's effort in producing this masterpiece. However, I'm having trouble finding the download page for various quantized models like Q_6_K or Q_8. I was also wondering when will it release the benchmarks since can't wait to see the model dominating the leaderboard.
Hi there, this model is a result of model merging without using any "potentially contaminated" models (berkeley-nest/Starling-LM-7B-alpha, Q-bert/MetaMath-Cybertron-Starling, v1olet/v1olet_marcoroni-go-bruins-merge-7B), and thus does not score as well as the ones that do.
For a high-scoring model, you can checkout our EmbeddedLLM/Mistral-7B-Merge-14-v0.1 and EmbeddedLLM/Mistral-7B-Merge-14-v0.2 (average score of 72.77).
As this is meant as an experiment of model merging, we do not have plans to quantize the models.