Spaces:
Running
on
Zero
Double heads
I have found a weird effect of having a main character in the picture, posing as in the Reference Pose (RP) image, whose face has no relation to the "Face Photo", and then a second "floating head" next to the first, where the second is the one that actually incorporates the FP's resemblance.
This second head is sometimes hovering next to the first, obscured from the neck down by another element in the picture, or even within a circular coin-sized artifact overimposed on the image like it was a sticker. Some other times, the neck is extremely long, stretched like it was a snake's.
Changing the seed sometimes fixes it but it is not guaranteed, as I have gotten the same effect three times in a row with different seeds, which indicates a deeper-running issue.
Including tags in the negative prompt such as "two heads, two faces, double head" etc. did not seem to fix this effect for good either.
I would suggest adjusting the algorithm for these cases, and curating the training dataset if it includes pictures that can be classified as showing two heads in those settings (e.g.: circular face-containing stickers in magazine covers).
use a bigger face pose image
- Add some negative prompts to disable multi-heads.
- As we only have 5 facial landmarks as pose now, the degree of freedom may be too large. You can add other controlnet to constrain the spatial layouts. Anyway, we will also update for supporting other controlnets soon.
maybe the face is too small