FYI: I quantized this with AutoAWQ

#5
by tachyphylaxis - opened

In case you wanted to add it along with info on other quantizations:

tachyphylaxis/QuartetAnemoi-70B-t0.0001-AWQ

Haven't really used it yet, but I did load it and it doesn't output garbage, so ... ;-)

Lodrick_and_Lily org

Thanks, added to the card. We made an AWQ, but all I got was single token output in testing so I never made it public.

Lol, it's so funny, that happened to me, but then I just did it again and it worked (?)

Sign up or log in to comment