<p><strong>Information</strong></p> | |
Alpaca 30B 4-bit. It's working with the newest GPTQ. Quantized using --true-sequential and --groupsize 128 optimizations. | |
This was made using Chansung's 30B Alpaca Lora: https://huggingface.co/chansung/alpaca-lora-30b | |
<p><strong>Benchmarks</strong></p> | |
Wikitext2: 4.37 | |
Ptb: 7.58 | |
C4: 6.19 | |
Note: This version uses grouping, therefore evaluations are better. However, this version takes up more VRAM. |