DANCECILIA commited on
Commit
8356bff
1 Parent(s): 6963642

Rename README.md to TESTE.md

Browse files

$ pip install diffusers transformers accelerate torch

Files changed (2) hide show
  1. README.md +0 -64
  2. TESTE.md +1 -0
README.md DELETED
@@ -1,64 +0,0 @@
1
- ---
2
- pipeline_tag: text-to-video
3
- license: cc-by-nc-4.0
4
- ---
5
-
6
- ![model example](https://i.imgur.com/1mrNnh8.png)
7
-
8
- # zeroscope_v2 576w
9
- A watermark-free Modelscope-based video model optimized for producing high-quality 16:9 compositions and a smooth video output. This model was trained from the [original weights](https://huggingface.co/damo-vilab/modelscope-damo-text-to-video-synthesis) using 9,923 clips and 29,769 tagged frames at 24 frames, 576x320 resolution.<br />
10
- zeroscope_v2_567w is specifically designed for upscaling with [zeroscope_v2_XL](https://huggingface.co/cerspense/zeroscope_v2_XL) using vid2vid in the [1111 text2video](https://github.com/kabachuha/sd-webui-text2video) extension by [kabachuha](https://github.com/kabachuha). Leveraging this model as a preliminary step allows for superior overall compositions at higher resolutions in zeroscope_v2_XL, permitting faster exploration in 576x320 before transitioning to a high-resolution render. See some [example outputs](https://www.youtube.com/watch?v=HO3APT_0UA4) that have been upscaled to 1024x576 using zeroscope_v2_XL. (courtesy of [dotsimulate](https://www.instagram.com/dotsimulate/))<br />
11
-
12
- zeroscope_v2_576w uses 7.9gb of vram when rendering 30 frames at 576x320
13
-
14
- ### Using it with the 1111 text2video extension
15
-
16
- 1. Download files in the zs2_576w folder.
17
- 2. Replace the respective files in the 'stable-diffusion-webui\models\ModelScope\t2v' directory.
18
-
19
- ### Upscaling recommendations
20
-
21
- For upscaling, it's recommended to use [zeroscope_v2_XL](https://huggingface.co/cerspense/zeroscope_v2_XL) via vid2vid in the 1111 extension. It works best at 1024x576 with a denoise strength between 0.66 and 0.85. Remember to use the same prompt that was used to generate the original clip. <br />
22
-
23
- ### Usage in 🧨 Diffusers
24
-
25
- Let's first install the libraries required:
26
-
27
- ```bash
28
- $ pip install diffusers transformers accelerate torch
29
- ```
30
-
31
- Now, generate a video:
32
-
33
- ```py
34
- import torch
35
- from diffusers import DiffusionPipeline, DPMSolverMultistepScheduler
36
- from diffusers.utils import export_to_video
37
-
38
- pipe = DiffusionPipeline.from_pretrained("cerspense/zeroscope_v2_576w", torch_dtype=torch.float16)
39
- pipe.scheduler = DPMSolverMultistepScheduler.from_config(pipe.scheduler.config)
40
- pipe.enable_model_cpu_offload()
41
-
42
- prompt = "Darth Vader is surfing on waves"
43
- video_frames = pipe(prompt, num_inference_steps=40, height=320, width=576, num_frames=24).frames
44
- video_path = export_to_video(video_frames)
45
- ```
46
-
47
- Here are some results:
48
-
49
- <table>
50
- <tr>
51
- Darth vader is surfing on waves.
52
- <br>
53
- <img src="https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/diffusers/darthvader_cerpense.gif"
54
- alt="Darth vader surfing in waves."
55
- style="width: 576;" />
56
- </center></td>
57
- </tr>
58
- </table>
59
-
60
- ### Known issues
61
-
62
- Lower resolutions or fewer frames could lead to suboptimal output. <br />
63
-
64
- Thanks to [camenduru](https://github.com/camenduru), [kabachuha](https://github.com/kabachuha), [ExponentialML](https://github.com/ExponentialML), [dotsimulate](https://www.instagram.com/dotsimulate/), [VANYA](https://twitter.com/veryVANYA), [polyware](https://twitter.com/polyware_ai), [tin2tin](https://github.com/tin2tin)<br />
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
TESTE.md ADDED
@@ -0,0 +1 @@
 
 
1
+ A ROBO invades a university with books in his hands and builds a house in the middle of a lawn.