Kolors-IP-Adapter-FaceID-Plus weights and inference code

     

📖 Introduction

We provide Kolors-IP-Adapter-FaceID-Plus module weights and inference code based on Kolors-Basemodel. Examples of Kolors-IP-Adapter-FaceID-Plus results are as follows:

• Our Kolors-IP-Adapter-FaceID-Plus module is trained on a large-scale and high-quality face dataset. We use the face ID embeddings generated by insightface and the CLIP features of face area to keep the face ID and structure information.

📊 Evaluation

For evaluation, we constructed a test set consisting of over 200 reference images and text prompts. We invited several image experts to provide fair ratings for the generated results of different models. The experts assessed the generated images based on five criteria: visual appeal, text faithfulness, face similarity, facial aesthetics and overall satisfaction. Visual appeal and text faithfulness are used to measure the text-to-image generation capability, adhering to the evaluation standards of BaseModel. Meanwhile, face similarity and facial aesthetics are used to evaluate the performance of the proposed Kolors-IP-Adapter-FaceID-Plus. The results are summarized in the table below, where Kolors-IP-Adapter-FaceID-Plus outperforms SDXL-IP-Adapter-FaceID-Plus across all metrics.

Model	Average Text Faithfulness	Average Visual Appeal	Average Face Similarity	Average Facial Aesthetics	Average Overall Satisfaction
SDXL-IP-Adapter-FaceID-Plus	4.014	3.455	3.05	2.584	2.448
Kolors-IP-Adapter-FaceID-Plus	4.235	4.374	4.415	3.887	3.561

---

Kolors-IP-Adapter-FaceID-Plus employs chinese prompts, while SDXL-IP-Adapter-FaceID-Plus use english prompts.

🛠️ Usage

Requirements

The dependencies and installation are basically the same as the Kolors-BaseModel.

1. Repository Cloning and Dependency Installation

apt-get install git-lfs
git clone https://github.com/Kwai-Kolors/Kolors
cd Kolors
conda create --name kolors python=3.8
conda activate kolors
pip install -r requirements.txt
pip install insightface onnxruntime-gpu
python3 setup.py install

2. Weights download link：

huggingface-cli download --resume-download Kwai-Kolors/Kolors-IP-Adapter-FaceID-Plus --local-dir weights/Kolors-IP-Adapter-FaceID-Plus

or

git lfs clone https://huggingface.co/Kwai-Kolors/Kolors-IP-Adapter-FaceID-Plus weights/Kolors-IP-Adapter-FaceID-Plus

3. Inference：

python ipadapter_FaceID/sample_ipadapter_faceid_plus.py ./ipadapter_FaceID/assets/image1.png "穿着晚礼服，在星光下的晚宴场景中，烛光闪闪，整个场景洋溢着浪漫而奢华的氛围"

python ipadapter_FaceID/sample_ipadapter_faceid_plus.py ./ipadapter_FaceID/assets/image2.png "西部牛仔，牛仔帽，荒野大镖客，背景是西部小镇，仙人掌，,日落余晖, 暖色调, 使用XT4胶片拍摄, 噪点, 晕影, 柯达胶卷，复古"

Acknowledgments

• Thanks to insightface for the face representations.
• Thanks to IP-Adapter for the codebase.