diffusers

/

controlnet-canny-sdxl-1.0

like 485

Follow

🧨Diffusers 304

sample code is posted, right?

@ViperYX I think he wants something even more readymade :p

@ViperYX , @Neilblaze I tried the sample code in colab with not modification except the canny image I got this
with this

prompt = "relaxed Cat at the table and holding a newspaper, anime style"
negative_prompt = ''

image = load_image("https://i.imgflip.com/4/tau4.jpg")

image = np.array(image)
image = cv2.Canny(image, 100, 200)
image = image[:, :, None]
image = np.concatenate([image, image, image], axis=2)
image = Image.fromarray(image)
image

@obi77
so sry for my late reply, the result is fllowing

@obi77
your code is okay, but your condition image quality is really bad.
your conditinal image size is (250, 183), the resolution is extremely low for SDXL, so i just directly using resize function on the canny edge image and padding it to the square, then downsample it to the proper resolution. And fed the downsampled image to the controlnet input. my code is fllowing:

prompt = "relaxed Cat at the table and holding a newspaper, anime style"
negative_prompt = 'low quality, bad quality, sketches'

image = load_image("https://i.imgflip.com/4/tau4.jpg")

image = np.array(image)
image = cv2.Canny(image, 100, 200)
image = image[:, :, None]
image = np.concatenate([image, image, image], axis=2)
image = Image.fromarray(image)
h, w = image.size

image = image.resize((6*h, 6*w))

new_image = Image.new(mode="RGB", size=(6*h, 6*h), color=(0,0,0))
new_image.paste(image, ((new_image.size[0]-image.size[0])//2, (new_image.size[1]-image.size[1])//2))

new_image = new_image.resize((1000, 1000))

images = pipe(
    prompt, negative_prompt=negative_prompt, image=new_image, controlnet_conditioning_scale=0.5,
    ).images

BTW, I did't test this code on the Google Colab, sicne free edition do not provide enough RAM, but the code work on my desktop.:P