Perturbed-Attention Guidance for SD 1.5 (i2i)
The original Perturbed-Attention Guidance for unconditional models and SD1.5 by Hyoungwon Cho is availiable at hyoungwoncho/sd_perturbed_attention_guidance
This repository is just a simple implementation of the Perturbed-Attention Guidance (PAG) on Stable Diffusion 1.5 (SD 1.5) for the 🧨 diffusers library to "image-to-image".
Quickstart
Loading Custom Pipeline:
from diffusers import StableDiffusionImg2ImgPipeline
pipe = StableDiffusionImg2ImgPipeline.from_pretrained(
"runwayml/stable-diffusion-v1-5",
custom_pipeline="jyoung105/sd15_perturbed_attention_guidance_i2i",
torch_dtype=torch.float16
)
device="cuda"
pipe = pipe.to(device)
Unconditional sampling with PAG:
output = pipe(
"",
image=init_image,
strength=0.6,
num_inference_steps=50,
guidance_scale=0.0,
pag_scale=5.0,
pag_applied_layers_index=['m0']
).images
output = pipe(
"A man with hoodie on is looking at sky, photo",
image=init_image,
strength=0.6,
num_inference_steps=50,
guidance_scale=0.0,
pag_scale=5.0,
pag_applied_layers_index=['m0']
).images
Parameters
guidance_scale
: guidance scale of CFG (ex: 7.5
)
pag_scale
: guidance scale of PAG (ex: 4.0
)
pag_applied_layers
: layer to apply perturbation (ex: ['mid'])
pag_applied_layers_index
: index of the layers to apply perturbation (ex: ['m0', 'm1'])