Hugging Face's logo

KBlueLeaf
/
Stable-Cascade-FP16-fixed

English
stable cascade
Model card Files Community
2
Stable-Cascade-FP16-fixed / README.md
KBlueLeaf's picture
KBlueLeaf
Update README.md
b21b792 verified
preview code
|
raw
history blame
3.03 kB
metadata
license: cc-by-nc-4.0
language:
  - en
tags:
  - stable cascade

Stable-Cascade FP16 fix

A modified version of Stable-Cascade which is compatibile with fp16 inference

Demo

FP16 BF16
image/png image/png

LPIPS difference: 0.088

FP16 BF16
image/png image/png

LPIPS difference: 0.012

How

After doing some check to the L1 norm of each hidden state. I found the last block group(8, 24, 24, 8 <- this one) make the hiddens states become bigger and bigger.

So I just apply some transformation on the TimestepBlock to directly modify the scale of hidden state. (Since it is not a residual block, so this is possible)

How the transformation be done is written in the modified "stable_cascade.py", you can put the file into kohya-ss/sd-scripts' stable-cascade branch and uncomment things to check weights or doing the conversion by yourselve.

FP8

Some people may know the FP8 quant for inference SDXL with lowvram cards. The technique can be applied to this model too.
But since the last block group is basically ruined, so it is recommend to ignore the last block group:

for name, module in generator_c.named_modules():
    if "up_blocks.1" in name: continue
    if isinstance(module, torch.nn.Linear):
        module.to(torch.float8_e5m2)
    elif isinstance(module, torch.nn.Conv2d):
        module.to(torch.float8_e5m2)
    elif isinstance(module, torch.nn.MultiheadAttention):
        module.to(torch.float8_e5m2)

This sample code should transform 70% of weight into fp8. (Use FP8 weight with scale is better solution, it is recommended to implement that)

I have tried different transform settings which is more friendly for FP8 but the differences between original model is more significant.

FP8 Demo (Same Seed): image/png

Notice

The modified version of model will not be compatibile with the lora/lycoris trained on original weight.
(actually it can, just do the same transformation, I'm considering to rewrite a version to use key name to determine what to do.)

Also the ControlNets will not be compatible too. Unless you also apply the needed transformation to them.

I don't want to do all of these by myself so hope some others will do that.

License

Stable-Cascade is published with a non-commercial lisence so I use CC-BY-NC 4.0 to publish this model. The source code to make this model is published with apache-2.0 license