cerspense patrickvonplaten commited on
Commit
592a8b0
1 Parent(s): c7d2cd7

Add code snippet for Diffusers (#2)

Browse files

- Add code snippet for Diffusers (de54b5141193bc2135850e3746863a9e4b52372c)


Co-authored-by: Patrick von Platen <[email protected]>

Files changed (1) hide show
  1. README.md +36 -0
README.md CHANGED
@@ -19,6 +19,42 @@ zeroscope_v2_576w uses 7.9gb of vram when rendering 30 frames at 576x320
19
 
20
  For upscaling, it's recommended to use [zeroscope_v2_XL](https://huggingface.co/cerspense/zeroscope_v2_XL) via vid2vid in the 1111 extension. It works best at 1024x576 with a denoise strength between 0.66 and 0.85. Remember to use the same prompt that was used to generate the original clip. <br />
21
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
22
 
23
  ### Known issues
24
 
 
19
 
20
  For upscaling, it's recommended to use [zeroscope_v2_XL](https://huggingface.co/cerspense/zeroscope_v2_XL) via vid2vid in the 1111 extension. It works best at 1024x576 with a denoise strength between 0.66 and 0.85. Remember to use the same prompt that was used to generate the original clip. <br />
21
 
22
+ ### Usage in 🧨 Diffusers
23
+
24
+ Let's first install the libraries required:
25
+
26
+ ```bash
27
+ $ pip install diffusers transformers accelerate torch
28
+ ```
29
+
30
+ Now, generate a video:
31
+
32
+ ```py
33
+ import torch
34
+ from diffusers import DiffusionPipeline, DPMSolverMultistepScheduler
35
+ from diffusers.utils import export_to_video
36
+
37
+ pipe = DiffusionPipeline.from_pretrained("cerspense/zeroscope_v2_576w", torch_dtype=torch.float16)
38
+ pipe.scheduler = DPMSolverMultistepScheduler.from_config(pipe.scheduler.config)
39
+ pipe.enable_model_cpu_offload()
40
+
41
+ prompt = "Darth Vader is surfing on waves"
42
+ video_frames = pipe(prompt, num_inference_steps=40, height=320, width=576, num_frames=24).frames
43
+ video_path = export_to_video(video_frames)
44
+ ```
45
+
46
+ Here are some results:
47
+
48
+ <table>
49
+ <tr>
50
+ Darth vader is surfing on waves.
51
+ <br>
52
+ <img src="https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/diffusers/darthvader_cerpense.gif"
53
+ alt="Darth vader surfing in waves."
54
+ style="width: 576;" />
55
+ </center></td>
56
+ </tr>
57
+ </table>
58
 
59
  ### Known issues
60