Arcane-Diffusion / README.md
nitrosocke's picture
Add Arcane Diffusion for Diffusers (#11)
ee738ff
|
raw
history blame
2.61 kB
metadata
license: creativeml-openrail-m

Arcane Diffusion

This is the fine-tuned Stable Diffusion model trained on images from the TV Show Arcane. Use the tokens arcane style in your prompts for the effect.

If you enjoy this model, please check out my other models on Huggingface

🧨 Diffusers

This model can be used just like any other Stable Diffusion model. For more information, please have a look at the Stable Diffusion.

You can also export the model to ONNX, MPS and/or FLAX/JAX.

from diffusers import StableDiffusionPipeline
import torch

model_id = "nitrosocke/Arcane-Diffusion"
pipe = StableDiffusionPipeline.from_pretrained(model_id, torch_dtype=torch.float16)
pipe = pipe.to("cuda")

prompt = "arcane style, a magical princess with golden hair"
image = pipe(prompt).images[0]

image.save("./magical_princess.png")

img

Sample images from v3:

output Samples v3 output Samples v3

Sample images from the model:

output Samples

Sample images used for training:

Training Samples

Version 3 (arcane-diffusion-v3): This version uses the new train-text-encoder setting and improves the quality and edibility of the model immensely. Trained on 95 images from the show in 8000 steps.

Version 2 (arcane-diffusion-v2): This uses the diffusers based dreambooth training and prior-preservation loss is way more effective. The diffusers where then converted with a script to a ckpt file in order to work with automatics repo. Training was done with 5k steps for a direct comparison to v1 and results show that it needs more steps for a more prominent result. Version 3 will be tested with 11k steps.

Version 1 (arcane-diffusion-5k): This model was trained using Unfrozen Model Textual Inversion utilizing the Training with prior-preservation loss methods. There is still a slight shift towards the style, while not using the arcane token.