TheBloke
/

StableBeluga2-70B-GPTQ

Text Generation

text-generation-inference

4-bit precision

Model card Files Files and versions Community

Resources

View closed (8)

Why so few 8 bit capable models?

#13 opened over 1 year ago by

Can Run "gptq_model-4bit--1g" but not "gptq-4bit-32g-actorder_True"

#12 opened over 1 year ago by

comparison with bitsandbytes nf4, hope to increase GPTQ accuracy

#11 opened over 1 year ago by

Mininum VRAM?

#9 opened over 1 year ago by

GGML version possible/coming?

#8 opened over 1 year ago by

vram requirements

#5 opened over 1 year ago by