Linaqruf commited on
Commit
2b164fc
1 Parent(s): 56bf24a

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +158 -1
README.md CHANGED
@@ -24,4 +24,161 @@ datasets:
24
  - Linaqruf/sdxl-dataset
25
  ---
26
 
27
- # **Pastel Anime LoRA for SDXL**
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
24
  - Linaqruf/sdxl-dataset
25
  ---
26
 
27
+ <style>
28
+ .title-container {
29
+ display: flex;
30
+ flex-direction: column; /* Allow vertical stacking of title and subtitle */
31
+ justify-content: center;
32
+ align-items: center;
33
+ height: 100vh;
34
+ background-color: #f5f5f5;
35
+ }
36
+
37
+ .title {
38
+ font-size: 2.5em;
39
+ text-align: center;
40
+ color: #333;
41
+ font-family: 'Verdana', sans-serif;
42
+ text-transform: uppercase;
43
+ letter-spacing: 0.2em;
44
+ padding: 1em;
45
+ border: 2px solid #7ed56f;
46
+ box-shadow: 5px 5px 15px rgba(0,0,0,0.1);
47
+ }
48
+
49
+ .title span, .subtitle span {
50
+ background: -webkit-linear-gradient(45deg, #ff9a9e, #fad0c4, #f6d365);
51
+ -webkit-background-clip: text;
52
+ -webkit-text-fill-color: transparent;
53
+ }
54
+
55
+ .subtitle {
56
+ margin-top: 15px;
57
+ font-size: 1em;
58
+ font-family: 'Verdana', sans-serif;
59
+ color: #666;
60
+ text-align: center;
61
+ }
62
+ .custom-table {
63
+ table-layout: fixed;
64
+ width: 100%;
65
+ border-collapse: collapse;
66
+ margin-top: 2em;
67
+ }
68
+ .custom-table td {
69
+ width: 50%;
70
+ vertical-align: top;
71
+ padding: 10px;
72
+ box-shadow: 0px 0px 10px 0px rgba(0,0,0,0.15);
73
+ }
74
+ .custom-image {
75
+ width: 100%;
76
+ height: auto;
77
+ object-fit: cover;
78
+ border-radius: 10px;
79
+ transition: transform .2s;
80
+ margin-bottom: 1em;
81
+ }
82
+ .custom-image:hover {
83
+ transform: scale(1.05);
84
+ }
85
+ </style>
86
+
87
+ <h1 class="title"><span>Pastel Anime LoRA for SDXL</span></h1>
88
+ <h2 class="subtitle"><span>TRAINED WITH </span><a href="https://huggingface.co/Linaqruf/animagine-xl"><span>ANIMAGINE XL</span></a></h2>
89
+
90
+ <hr>
91
+
92
+ <table class="custom-table">
93
+ <tr>
94
+ <td>
95
+ <a href="https://huggingface.co/Linaqruf/pastel-anime-xl-lora/blob/main/samples/xl_output_upscaled_00001_.png">
96
+ <img class="custom-image" src="https://huggingface.co/Linaqruf/pastel-anime-xl-lora/resolve/main/samples/xl_output_upscaled_00001_.png" alt="sample1">
97
+ </a>
98
+ </td>
99
+ <td>
100
+ <a href="https://huggingface.co/Linaqruf/pastel-anime-xl-lora/blob/main/samples/xl_output_upscaled_00006_.png">
101
+ <img class="custom-image" src="https://huggingface.co/Linaqruf/pastel-anime-xl-lora/resolve/main/samples/xl_output_upscaled_00006_.png" alt="sample2">
102
+ </a>
103
+ </td>
104
+ </tr>
105
+ </table>
106
+
107
+ <hr>
108
+
109
+ ## Overview
110
+
111
+ **Pastel Anime LoRA for SDXL** is a high-resolution, Low-Rank Adaptation model for Stable Diffusion XL. The model has been fine-tuned using a learning rate of 1e-5 over 1300 global steps with a batch size of 24 on a curated dataset of superior-quality anime-style images. This model is derived from Animagine XL.
112
+
113
+ Like other anime-style Stable Diffusion models, it also supports Danbooru tags to generate images.
114
+
115
+ e.g. _**face focus, cute, masterpiece, best quality, 1girl, green hair, sweater, looking at viewer, upper body, beanie, outdoors, night, turtleneck**_
116
+
117
+
118
+ <hr>
119
+
120
+ ## Model Details
121
+
122
+ - **Developed by:** [Linaqruf](https://github.com/Linaqruf)
123
+ - **Model type:** Low-rank adaptation of diffusion-based text-to-image generative model
124
+ - **Model Description:** This is a small model that should be used with big model and can be used to generate and modify high quality anime-themed images based on text prompts.
125
+ - **License:** [CreativeML Open RAIL++-M License](https://huggingface.co/stabilityai/stable-diffusion-2/blob/main/LICENSE-MODEL)
126
+ - **Finetuned from model:** [Animagine XL](https://huggingface.co/Linaqruf/animagine-xl)
127
+
128
+ <hr>
129
+
130
+ ## 🧨 Diffusers
131
+
132
+ Make sure to upgrade diffusers to >= 0.18.2:
133
+ ```
134
+ pip install diffusers --upgrade
135
+ ```
136
+
137
+ In addition make sure to install `transformers`, `safetensors`, `accelerate` as well as the invisible watermark:
138
+ ```
139
+ pip install invisible_watermark transformers accelerate safetensors
140
+ ```
141
+
142
+ Running the pipeline (if you don't swap the scheduler it will run with the default **EulerDiscreteScheduler** in this example we are swapping it to **EulerAncestralDiscreteScheduler**:
143
+ ```py
144
+ import torch
145
+ from torch import autocast
146
+ from diffusers import StableDiffusionXLPipeline, EulerAncestralDiscreteScheduler
147
+
148
+ base_model = "Linaqruf/animagine-xl"
149
+ lora_model_id = "Linaqruf/pastel-anime-xl-lora"
150
+ lora_filename = "pastel-anime-xl.safetensors"
151
+
152
+ pipe = StableDiffusionXLPipeline.from_pretrained(
153
+ model,
154
+ torch_dtype=torch.float16,
155
+ use_safetensors=True,
156
+ variant="fp16"
157
+ )
158
+
159
+ pipe.scheduler = EulerAncestralDiscreteScheduler.from_config(pipe.scheduler.config)
160
+ pipe.to('cuda')
161
+
162
+ pipe.load_lora_weights(lora_model_id, weight_name=lora_filename)
163
+
164
+ prompt = "face focus, cute, masterpiece, best quality, 1girl, green hair, sweater, looking at viewer, upper body, beanie, outdoors, night, turtleneck"
165
+ negative_prompt = "lowres, bad anatomy, bad hands, text, error, missing fingers, extra digit, fewer digits, cropped, worst quality, low quality, normal quality, jpeg artifacts, signature, watermark, username, blurry"
166
+
167
+ image = pipe(
168
+ prompt,
169
+ negative_prompt=negative_prompt,
170
+ width=1024,
171
+ height=1024,
172
+ guidance_scale=12,
173
+ target_size=(1024,1024),
174
+ original_size=(4096,4096),
175
+ num_inference_steps=50
176
+ ).images[0]
177
+
178
+ image.save("anime_girl.png")
179
+ ```
180
+ <hr>
181
+
182
+ ## Limitation
183
+ This model inherit Stable Diffusion XL 1.0 [limitation](https://huggingface.co/stabilityai/stable-diffusion-xl-base-1.0#limitations)
184
+