Update README.md
Browse files
README.md
CHANGED
@@ -24,4 +24,161 @@ datasets:
|
|
24 |
- Linaqruf/sdxl-dataset
|
25 |
---
|
26 |
|
27 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
24 |
- Linaqruf/sdxl-dataset
|
25 |
---
|
26 |
|
27 |
+
<style>
|
28 |
+
.title-container {
|
29 |
+
display: flex;
|
30 |
+
flex-direction: column; /* Allow vertical stacking of title and subtitle */
|
31 |
+
justify-content: center;
|
32 |
+
align-items: center;
|
33 |
+
height: 100vh;
|
34 |
+
background-color: #f5f5f5;
|
35 |
+
}
|
36 |
+
|
37 |
+
.title {
|
38 |
+
font-size: 2.5em;
|
39 |
+
text-align: center;
|
40 |
+
color: #333;
|
41 |
+
font-family: 'Verdana', sans-serif;
|
42 |
+
text-transform: uppercase;
|
43 |
+
letter-spacing: 0.2em;
|
44 |
+
padding: 1em;
|
45 |
+
border: 2px solid #7ed56f;
|
46 |
+
box-shadow: 5px 5px 15px rgba(0,0,0,0.1);
|
47 |
+
}
|
48 |
+
|
49 |
+
.title span, .subtitle span {
|
50 |
+
background: -webkit-linear-gradient(45deg, #ff9a9e, #fad0c4, #f6d365);
|
51 |
+
-webkit-background-clip: text;
|
52 |
+
-webkit-text-fill-color: transparent;
|
53 |
+
}
|
54 |
+
|
55 |
+
.subtitle {
|
56 |
+
margin-top: 15px;
|
57 |
+
font-size: 1em;
|
58 |
+
font-family: 'Verdana', sans-serif;
|
59 |
+
color: #666;
|
60 |
+
text-align: center;
|
61 |
+
}
|
62 |
+
.custom-table {
|
63 |
+
table-layout: fixed;
|
64 |
+
width: 100%;
|
65 |
+
border-collapse: collapse;
|
66 |
+
margin-top: 2em;
|
67 |
+
}
|
68 |
+
.custom-table td {
|
69 |
+
width: 50%;
|
70 |
+
vertical-align: top;
|
71 |
+
padding: 10px;
|
72 |
+
box-shadow: 0px 0px 10px 0px rgba(0,0,0,0.15);
|
73 |
+
}
|
74 |
+
.custom-image {
|
75 |
+
width: 100%;
|
76 |
+
height: auto;
|
77 |
+
object-fit: cover;
|
78 |
+
border-radius: 10px;
|
79 |
+
transition: transform .2s;
|
80 |
+
margin-bottom: 1em;
|
81 |
+
}
|
82 |
+
.custom-image:hover {
|
83 |
+
transform: scale(1.05);
|
84 |
+
}
|
85 |
+
</style>
|
86 |
+
|
87 |
+
<h1 class="title"><span>Pastel Anime LoRA for SDXL</span></h1>
|
88 |
+
<h2 class="subtitle"><span>TRAINED WITH </span><a href="https://huggingface.co/Linaqruf/animagine-xl"><span>ANIMAGINE XL</span></a></h2>
|
89 |
+
|
90 |
+
<hr>
|
91 |
+
|
92 |
+
<table class="custom-table">
|
93 |
+
<tr>
|
94 |
+
<td>
|
95 |
+
<a href="https://huggingface.co/Linaqruf/pastel-anime-xl-lora/blob/main/samples/xl_output_upscaled_00001_.png">
|
96 |
+
<img class="custom-image" src="https://huggingface.co/Linaqruf/pastel-anime-xl-lora/resolve/main/samples/xl_output_upscaled_00001_.png" alt="sample1">
|
97 |
+
</a>
|
98 |
+
</td>
|
99 |
+
<td>
|
100 |
+
<a href="https://huggingface.co/Linaqruf/pastel-anime-xl-lora/blob/main/samples/xl_output_upscaled_00006_.png">
|
101 |
+
<img class="custom-image" src="https://huggingface.co/Linaqruf/pastel-anime-xl-lora/resolve/main/samples/xl_output_upscaled_00006_.png" alt="sample2">
|
102 |
+
</a>
|
103 |
+
</td>
|
104 |
+
</tr>
|
105 |
+
</table>
|
106 |
+
|
107 |
+
<hr>
|
108 |
+
|
109 |
+
## Overview
|
110 |
+
|
111 |
+
**Pastel Anime LoRA for SDXL** is a high-resolution, Low-Rank Adaptation model for Stable Diffusion XL. The model has been fine-tuned using a learning rate of 1e-5 over 1300 global steps with a batch size of 24 on a curated dataset of superior-quality anime-style images. This model is derived from Animagine XL.
|
112 |
+
|
113 |
+
Like other anime-style Stable Diffusion models, it also supports Danbooru tags to generate images.
|
114 |
+
|
115 |
+
e.g. _**face focus, cute, masterpiece, best quality, 1girl, green hair, sweater, looking at viewer, upper body, beanie, outdoors, night, turtleneck**_
|
116 |
+
|
117 |
+
|
118 |
+
<hr>
|
119 |
+
|
120 |
+
## Model Details
|
121 |
+
|
122 |
+
- **Developed by:** [Linaqruf](https://github.com/Linaqruf)
|
123 |
+
- **Model type:** Low-rank adaptation of diffusion-based text-to-image generative model
|
124 |
+
- **Model Description:** This is a small model that should be used with big model and can be used to generate and modify high quality anime-themed images based on text prompts.
|
125 |
+
- **License:** [CreativeML Open RAIL++-M License](https://huggingface.co/stabilityai/stable-diffusion-2/blob/main/LICENSE-MODEL)
|
126 |
+
- **Finetuned from model:** [Animagine XL](https://huggingface.co/Linaqruf/animagine-xl)
|
127 |
+
|
128 |
+
<hr>
|
129 |
+
|
130 |
+
## 🧨 Diffusers
|
131 |
+
|
132 |
+
Make sure to upgrade diffusers to >= 0.18.2:
|
133 |
+
```
|
134 |
+
pip install diffusers --upgrade
|
135 |
+
```
|
136 |
+
|
137 |
+
In addition make sure to install `transformers`, `safetensors`, `accelerate` as well as the invisible watermark:
|
138 |
+
```
|
139 |
+
pip install invisible_watermark transformers accelerate safetensors
|
140 |
+
```
|
141 |
+
|
142 |
+
Running the pipeline (if you don't swap the scheduler it will run with the default **EulerDiscreteScheduler** in this example we are swapping it to **EulerAncestralDiscreteScheduler**:
|
143 |
+
```py
|
144 |
+
import torch
|
145 |
+
from torch import autocast
|
146 |
+
from diffusers import StableDiffusionXLPipeline, EulerAncestralDiscreteScheduler
|
147 |
+
|
148 |
+
base_model = "Linaqruf/animagine-xl"
|
149 |
+
lora_model_id = "Linaqruf/pastel-anime-xl-lora"
|
150 |
+
lora_filename = "pastel-anime-xl.safetensors"
|
151 |
+
|
152 |
+
pipe = StableDiffusionXLPipeline.from_pretrained(
|
153 |
+
model,
|
154 |
+
torch_dtype=torch.float16,
|
155 |
+
use_safetensors=True,
|
156 |
+
variant="fp16"
|
157 |
+
)
|
158 |
+
|
159 |
+
pipe.scheduler = EulerAncestralDiscreteScheduler.from_config(pipe.scheduler.config)
|
160 |
+
pipe.to('cuda')
|
161 |
+
|
162 |
+
pipe.load_lora_weights(lora_model_id, weight_name=lora_filename)
|
163 |
+
|
164 |
+
prompt = "face focus, cute, masterpiece, best quality, 1girl, green hair, sweater, looking at viewer, upper body, beanie, outdoors, night, turtleneck"
|
165 |
+
negative_prompt = "lowres, bad anatomy, bad hands, text, error, missing fingers, extra digit, fewer digits, cropped, worst quality, low quality, normal quality, jpeg artifacts, signature, watermark, username, blurry"
|
166 |
+
|
167 |
+
image = pipe(
|
168 |
+
prompt,
|
169 |
+
negative_prompt=negative_prompt,
|
170 |
+
width=1024,
|
171 |
+
height=1024,
|
172 |
+
guidance_scale=12,
|
173 |
+
target_size=(1024,1024),
|
174 |
+
original_size=(4096,4096),
|
175 |
+
num_inference_steps=50
|
176 |
+
).images[0]
|
177 |
+
|
178 |
+
image.save("anime_girl.png")
|
179 |
+
```
|
180 |
+
<hr>
|
181 |
+
|
182 |
+
## Limitation
|
183 |
+
This model inherit Stable Diffusion XL 1.0 [limitation](https://huggingface.co/stabilityai/stable-diffusion-xl-base-1.0#limitations)
|
184 |
+
|