| ### Example | |
| - region prompt(txt2img / no controlnet) | |
| - region 0 ... 1girl, upper body etc | |
| - region 1 ... ((car)), street, road,no human etc | |
| - background ... town, outdoors etc | |
| - ip adapter input for background / region 0 / region 1 | |
| <img src="https://github.com/s9roll7/animatediff-cli-prompt-travel/assets/118420657/ca355f4b-f4c0-4405-88f4-1c80632e32a6" width="512"> | |
| - animatediff generate -c config/prompts/region_txt2img.json -W 512 -H 768 -L 32 -C 16 | |
| - region 0 mask / region 1 mask / txt2img | |
| <div><video controls src="https://github.com/s9roll7/animatediff-cli-prompt-travel/assets/118420657/88eb1572-2772-4d76-89c1-c6bf8142283d" muted="false"></video></div> | |
| <div><video controls src="https://github.com/s9roll7/animatediff-cli-prompt-travel/assets/118420657/55042dff-4e23-43b9-b4d6-6f2228b943d2" muted="false"></video></div> | |
| <br> | |
| - apply different lora for each region. | |
| - [abdiel](https://civitai.com/models/159943/abdiel-shin-megami-tensei-v-v) for region 0 | |
| - [amanozoko](https://civitai.com/models/159933/amanozoko-shin-megami-tensei-v-v) for region 1 | |
| - no lora for background | |
| <div><video controls src="https://github.com/s9roll7/animatediff-cli-prompt-travel/assets/118420657/a5260cf0-1f96-4c65-8408-805428d2528e" muted="false"></video></div> | |
| ```json | |
| # new lora_map format | |
| "lora_map": { | |
| # Specify lora as a path relative to /animatediff-cli/data | |
| "share/Lora/zs_Abdiel.safetensors": { # setting for abdiel lora | |
| "region" : ["0"], # target region. Multiple designations possible | |
| "scale" : { | |
| # "frame_no" : scale format | |
| "0": 0.75 # lora scale. same as prompt_map format. For example, it is possible to set the lora to be used from the 30th frame. | |
| } | |
| }, | |
| "share/Lora/zs_Amanazoko.safetensors": { # setting for amanozako lora | |
| "region" : ["1"], # target region | |
| "scale" : { | |
| "0": 0.75 | |
| } | |
| } | |
| }, | |
| ``` | |
| - more example [here](https://github.com/s9roll7/animatediff-cli-prompt-travel/issues/147) | |
| <br> | |
| - img2img | |
| - This can be improved using controlnet, but this sample does not use it. | |
| - source / denoising_strength 0.7 / denoising_strength 0.85 | |
| <div><video controls src="https://github.com/s9roll7/animatediff-cli-prompt-travel/assets/118420657/4a9ecf3e-d1c7-468a-a85f-abce7b4c4aab" muted="false"></video></div> | |
| <br> | |
| <br> | |
| - [A command to stylization with region has been added](https://github.com/s9roll7/animatediff-cli-prompt-travel#video-stylization-with-region). | |
| - (You can also create json manually without using the stylize command.) | |
| - region prompt | |
| - Region division into person shapes | |
| - source / img2img / txt2img | |
| <div><video controls src="https://github.com/s9roll7/animatediff-cli-prompt-travel/assets/118420657/10101ab8-39cc-4dbc-85c8-af558f80c6fe" muted="false"></video></div> | |
| <br> | |
| - source / Region division into person shapes / inpaint | |
| <div><video controls src="https://github.com/s9roll7/animatediff-cli-prompt-travel/assets/118420657/d9231c8e-94b6-4608-97ba-6ab3fc85bcfa" muted="false"></video></div> | |
| <br> | |
| <br> | |
| - [A command to stylization with mask has been added](https://github.com/s9roll7/animatediff-cli-prompt-travel#video-stylization-with-mask). | |
| - more example [here](https://github.com/s9roll7/animatediff-cli-prompt-travel/issues/111) | |
| <div><video controls src="https://github.com/s9roll7/animatediff-cli-prompt-travel/assets/118420657/461cd68a-511a-4ad8-a2ee-7078faed7354" muted="false"></video></div> | |
| <br> | |
| - [A command to automate video stylization has been added](https://github.com/s9roll7/animatediff-cli-prompt-travel#video-stylization). | |
| - Original / First generation result / Second generation(for upscaling) result | |
| - It took 4 minutes to generate the first one and about 5 minutes to generate the second one (on rtx 4090). | |
| - more example [here](https://github.com/s9roll7/animatediff-cli-prompt-travel/issues/29) | |
| <div><video controls src="https://github.com/s9roll7/animatediff-cli-prompt-travel/assets/118420657/2f1965f2-9a50-485e-ac95-e888a3189ba2" muted="false"></video></div> | |
| <br> | |
| - controlnet_openpose + controlnet_softedge | |
| - input frames for controlnet(0,16,32 frames) | |
| <img src="https://github.com/s9roll7/animatediff-cli-prompt-travel/assets/118420657/4adac698-75a4-4c6d-bf64-a5723d0e3e77" width="512"> | |
| - result | |
| <div><video controls src="https://github.com/s9roll7/animatediff-cli-prompt-travel/assets/118420657/50aa9d0d-15b6-4c84-a497-8d020d3bdb7c" muted="false"></video></div> | |
| <br> | |
| - In the latest version, generation can now be controlled more precisely through prompts. | |
| - sample 1 | |
| ```json | |
| "prompt_fixed_ratio": 0.8, | |
| "head_prompt": "1girl, wizard, circlet, earrings, jewelry, purple hair,", | |
| "prompt_map": { | |
| "0": "(standing,full_body),blue_sky, town", | |
| "8": "(sitting,full_body),rain, town", | |
| "16": "(standing,full_body),blue_sky, woods", | |
| "24": "(upper_body), beach", | |
| "32": "(upper_body, smile)", | |
| "40": "(upper_body, angry)", | |
| "48": "(upper_body, smile, from_above)", | |
| "56": "(upper_body, angry, from_side)", | |
| "64": "(upper_body, smile, from_below)", | |
| "72": "(upper_body, angry, from_behind, looking at viewer)", | |
| "80": "face,looking at viewer", | |
| "88": "face,looking at viewer, closed_eyes", | |
| "96": "face,looking at viewer, open eyes, open_mouth", | |
| "104": "face,looking at viewer, closed_eyes, closed_mouth", | |
| "112": "face,looking at viewer, open eyes,eyes, open_mouth, tongue, smile, laughing", | |
| "120": "face,looking at viewer, eating, bowl,chopsticks,holding,food" | |
| }, | |
| ``` | |
| <div><video controls src="https://github.com/s9roll7/animatediff-cli-prompt-travel/assets/118420657/c4de4b87-f302-4d61-98c7-9607dece386f" muted="false"></video></div> | |
| <br> | |
| - sample 2 | |
| ```json | |
| "prompt_fixed_ratio": 1.0, | |
| "head_prompt": "1girl, wizard, circlet, earrings, jewelry, purple hair,", | |
| "prompt_map": { | |
| "0": "", | |
| "8": "((fire magic spell, fire background))", | |
| "16": "((ice magic spell, ice background))", | |
| "24": "((thunder magic spell, thunder background))", | |
| "32": "((skull magic spell, skull background))", | |
| "40": "((wind magic spell, wind background))", | |
| "48": "((stone magic spell, stone background))", | |
| "56": "((holy magic spell, holy background))", | |
| "64": "((star magic spell, star background))", | |
| "72": "((plant magic spell, plant background))", | |
| "80": "((meteor magic spell, meteor background))" | |
| }, | |
| ``` | |
| <div><video controls src="https://github.com/s9roll7/animatediff-cli-prompt-travel/assets/118420657/31a5827d-e551-4937-8b67-51747a92d14c" muted="false"></video></div> | |
| <br> | |