MetaIX's picture
Create README.md
aad1998
|
raw
history blame
437 Bytes
<p><strong>Information</strong></p>
Alpaca 30B 4-bit. It's working with the newest GPTQ. Quantized using --true-sequential and --groupsize 128 optimizations.
This was made using Chansung's 30B Alpaca Lora: https://huggingface.co/chansung/alpaca-lora-30b
<p><strong>Benchmarks</strong></p>
Wikitext2: 4.37
Ptb: 7.58
C4: 6.19
Note: This version uses grouping, therefore evaluations are better. However, this version takes up more VRAM.