sd35-training-m-init

This is a LyCORIS adapter derived from stabilityai/stable-diffusion-3.5-medium.

The main validation prompt used during training was:

loona from helluva boss.

Validation settings

• CFG: 5.5
• CFG Rescale: 0.0
• Steps: 20
• Sampler: None
• Seed: 42
• Resolution: 1024x1024

Note: The validation settings are not necessarily the same as the training settings.

You can find some example images in the following gallery:

Prompt

unconditional (blank prompt)

Negative Prompt

'

Prompt

Loona is a striking anthropomorphic hellhound from the animated web series "Helluva Boss," serving as the receptionist for the demonic assassination company I.M.P. She has dark gray or black fur accented with white markings, giving her a distinctive and edgy look. Her eyes feature red irises set against yellow sclera, adding to her fierce appearance. Embracing a gothic or punk fashion style, Loona typically wears a black crop top, ripped jeans, and combat boots. She accessorizes with spiked bracelets and a choker, which enhance her rebellious image and complement her aloof and sarcastic personality.

Negative Prompt

'

Prompt

A blonde-haired anime girl with green eyes, wearing a black dress adorned with green and gold patterns, is depicted in three different poses. She has a braid, wears a choker, and is seen with a smile, a belt pouch, and a green cape. The background showcases a serene landscape with mountains and a wooden fence.

Negative Prompt

'

Prompt

A captivating portrait of a young boy rendered in oil paint on canvas, with short, light-brown hair framing his face, turned slightly toward the viewer. He wears a blue and white striped shirt with a brown undershirt visible at the neck. A single strand of barbed wire extends from the bottom right corner, adding a poignant touch. In the background, a gray tone is punctuated by darker, abstract shapes evoking desolation, creating a somber and contemplative mood. The artist's use of light and shadow creates depth, drawing the viewer's eye to the boy's face and the barbed wire. The overall effect is one of quiet intensity, inviting reflection on the emotions and experiences behind the boy's expression. The painting's title, "liberated," suggests the boy has been freed from oppression, inviting the viewer to ponder the complexities of freedom and its consequences.

Negative Prompt

'

Prompt

A young boy in a blue and white striped shirt with a brown undershirt, surrounded by desolate abstract shapes, is depicted with a poignant barbed wire in the background.

Negative Prompt

'

Prompt

A serene and picturesque scene of a house situated near a body of water, with a garden in the foreground, features a reddish-brown exterior house with brown wooden shutters, surrounded by a colorful assortment of flowers and lush greenery bounded by a wrought iron fence. The garden path, made of stone or concrete, is flanked by four large orange pots containing greenery and flowers, and a small potted plant sits on the path. The water's edge is lined with a rocky shoreline, and a hill or mountain rises up to meet the sky. The overall atmosphere is one of tranquility and beauty, with a painting, signed "J. Ruval", visible in the bottom-right corner.

Negative Prompt

'

Prompt

A house near a body of water features a reddish-brown exterior, colorful garden, and a rocky shoreline, with a painting signed "J. Ruval" in the corner.

Negative Prompt

'

Prompt

loona from helluva boss.

Negative Prompt

'

The text encoder was not trained. You may reuse the base model text encoder for inference.

Training settings

• Training epochs: 11
• Training steps: 20000
• Learning rate: 1e-06
• Max grad norm: 0.01
• Effective batch size: 1
  • Micro-batch size: 1
  • Gradient accumulation steps: 1
  • Number of GPUs: 1
• Prediction type: flow-matching
• Rescaled betas zero SNR: False
• Optimizer: adamw_bf16
• Precision: Pure BF16
• Quantised: Yes: int8-quanto
• Xformers: Not used
• LyCORIS Config:

{
    "bypass_mode": true,
    "algo": "lokr",
    "multiplier": 1.0,
    "full_matrix": true,
    "linear_dim": 10000,
    "linear_alpha": 1,
    "factor": 12,
    "apply_preset": {
        "target_module": [
            "JointTransformerBlock"
        ],
        "module_algo_map": {
            "FeedForward": {
                "factor": 1
            },
            "JointTransformerBlock": {
                "factor": 2
            }
        }
    }
}

Datasets

default_dataset_art

• Repeats: 0
• Total number of images: 1805
• Total number of aspect buckets: 374
• Resolution: 2048 px
• Cropped: True
• Crop style: random
• Crop aspect: closest
• Used for regularisation data: No

Inference

import torch
from diffusers import DiffusionPipeline
from lycoris import create_lycoris_from_weights

model_id = 'stabilityai/stable-diffusion-3.5-medium'
adapter_id = 'pytorch_lora_weights.safetensors' # you will have to download this manually
lora_scale = 1.0
wrapper, _ = create_lycoris_from_weights(lora_scale, adapter_id, pipeline.transformer)
wrapper.merge_to()

prompt = "loona from helluva boss."
negative_prompt = ''
pipeline.to('cuda' if torch.cuda.is_available() else 'mps' if torch.backends.mps.is_available() else 'cpu')
image = pipeline(
    prompt=prompt,
    negative_prompt=negative_prompt,
    num_inference_steps=20,
    generator=torch.Generator(device='cuda' if torch.cuda.is_available() else 'mps' if torch.backends.mps.is_available() else 'cpu').manual_seed(1641421826),
    width=1024,
    height=1024,
    guidance_scale=5.5,
).images[0]
image.save("output.png", format="PNG")