Forge support and updated convert script.
Hi. I had some people message me that these models don't work in forge.
I think the issue was that the model arch/key shape for img_in was triggering the SDXL key reshape logic. I've fixed that in this commit so I think if you re-convert the weights on the latest version should make it work with forge too.
(Edit: I think this might affect other quants made with the convert script as well? Not 100% sure, the original dev/schnell ones are fine apparently though those were made fairly early on...)
((I'd send a PR with updated weights but I'm stuck on like 10mbps upload lmao))
Hi,
I'm actually redoing the whole process and sending them to huggingface again, I did a quick test between the old Q8_0 and the new one, there's absolutely no difference in output so that's cool.
Yeah the actual quantization doesn't change much (also it's just one weight/key, see image below), it's mostly just about having the format match what forge/other backends expect so we don't end up with comfy-specific checkpoints lol - the whole orig shape/reshape logic wasn't meant to trigger for flux but I guess the checks weren't explicit enough originally.
I did have plants to improve the logic for the _M quants but yeah, been super busy lately. Still, the current logic does work mostly fine.
btw not sure how you're uploading it but I had better luck with the python API than the cli or actual git lfs. Also this though idk if it's even active for uploads.
from huggingface_hub import HfApi
fname = "flux-dev-de-distill-Q8_0.gguf"
api.upload_file(
path_or_fileobj=fname,
path_in_repo=fname,
repo_id=f"TheYuriLover/flux-dev-de-distill-GGUF",
repo_type="model",
)
All right they're all uploaded, should be working with Forge now, don't hesitate to tell me if something's wrong with one of those quants.