README.md

Information

Alpaca 30B 4-bit. It's working with the newest GPTQ. Quantized using --true-sequential and --groupsize 128 optimizations. This was made using Chansung's 30B Alpaca Lora: https://huggingface.co/chansung/alpaca-lora-30b

Benchmarks

Wikitext2: 4.37

Ptb: 7.58

C4: 6.19

Note: This version uses grouping, therefore evaluations are better. However, this version takes up more VRAM.