File size: 1,846 Bytes
b27d22d b8f0ab3 b27d22d bc0a907 9085f30 bc0a907 b27d22d 56e9f19 c27a688 f549903 b27d22d f549903 82d2167 b27d22d 70d427f b27d22d 5193ae7 b27d22d 70d427f 5193ae7 b27d22d f3e3fe8 82d2167 b27d22d 70d427f fafd3b5 b27d22d f3e3fe8 82d2167 f3e3fe8 fafd3b5 f3e3fe8 bc0a907 b27d22d 8ac8c9e 1bb7127 aabeddd 58d4985 866f866 80e27cc 3790c8f |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 |
---
language:
- en
pipeline_tag: unconditional-image-generation
tags:
- Diffusion Models
- Stable Diffusion
- Perturbed-Attention Guidance
- PAG
---
# Perturbed-Attention Guidance
[Project](https://ku-cvlab.github.io/Perturbed-Attention-Guidance/) / [arXiv](https://arxiv.org/abs/2403.17377) / [GitHub](https://github.com/KU-CVLAB/Perturbed-Attention-Guidance)
This repository is based on [Diffusers](https://huggingface.co/docs/diffusers/index). The pipeline is a modification of StableDiffusionPipeline to support Perturbed-Attention Guidance (PAG).
## Quickstart
Loading Custom Piepline:
```
from diffusers import StableDiffusionPipeline
pipe = StableDiffusionPipeline.from_pretrained(
"runwayml/stable-diffusion-v1-5",
custom_pipeline="hyoungwoncho/sd_perturbed_attention_guidance",
torch_dtype=torch.float16,
safety_checker=None
)
device="cuda"
pipe = pipe.to(device)
prompts = ["a corgi"]
```
Sampling with PAG:
```
output = pipe(
prompts,
width=512,
height=512,
num_inference_steps=50,
guidance_scale=0.0,
pag_scale=5.0,
pag_applied_layers_index=['m0']
).images
```
Sampling with PAG and CFG:
```
output = pipe(
prompts,
width=512,
height=512,
num_inference_steps=50,
guidance_scale=4.0,
pag_scale=3.0,
pag_applied_layers_index=['m0']
).images
```
## Parameters
guidance_scale : gudiance scale of CFG (ex: 7.5)
pag_scale : gudiance scale of PAG (ex: 5.0)
pag_applied_layers_index : index of the layer to apply perturbation (ex: ['m0'])
## Stable Diffusion Demo
To join a demo of PAG on Stable Diffusion, run [sd_pag_demo.ipynb](https://huggingface.co/hyoungwoncho/sd_perturbed_attention_guidance/blob/main/sd_pag_demo.ipynb).
|