Failed to run this model on A6000 48GB VRAM Machine

#3
by Leegohi - opened

I followed the model card to run it with both webui and native python. It killed everytime when i attemp to load it. Since not enough VRAM? I don't know why.

What did the log say? If the entire process is terminated, it typically indicates a shortage of system RAM rather than VRAM.

@jlzhou thanks a lot. The reason is my limit RAM 12G not VRAM. 33B GPTQ model can run on my A10 24G machine.

Leegohi changed discussion status to closed

Sign up or log in to comment