Limour/CausalLM-14B-GGUF

Quantize

• https://www.kaggle.com/code/reginliu/causallm2gguf-imatrix
• https://colab.research.google.com/drive/1JT3XFjD7CTRB97pu3QpeGuzWA1yYEAM7?usp=sharing

Perplexity

• causallm_14b.IQ4_XS.gguf: PPL = 13.4127 +/- 0.13762
• causallm_14b.IQ3_XS.gguf: PPL = 13.3798 +/- 0.13641
• causallm_14b.IQ2_XXS.gguf: PPL = 15.0160 +/- 0.15004

https://www.kaggle.com/code/reginliu/perplexity