Watermarks in videos

#7
by TheBigBlockPC - opened

The model has a very good generalisation and petforms similar to Sora but the model sometimes adds the text "POND5" in the middle of the video. I notice this artifact if the videos are in the style of stock videos.

Knowledge Engineering Group (KEG) & Data Mining at Tsinghua University org

Could you provide us with some prompts that might reproduce this issue? It would be very helpful, and I’ll pass this along to the relevant colleagues. Thank you!

Just to make sure. Do i need a commercial license for the model if i want to upload the generated videos on tiktok when i earn bo money with the videos.

The watermark issue isn't 100% consistent but here is a prompt where i encountered it.
Prompt:
A movie trailer featuring the adventures of the 30 year old space man wearing a red wool knitted motorcycle helmet, blue sky, salt desert, cinematic style, shot on 35mm film, vivid colors.

Knowledge Engineering Group (KEG) & Data Mining at Tsinghua University org

Of course, you don’t need a commercial license since this is for personal entertainment. You can use the model directly and just give proper credit.

As for the watermark issue with the prompt, we haven’t been able to reproduce it yet. We’ll try a few more times. Thanks for your suggestion!

hopefully this watermark issue gets fixed. it's great finally having a AI model that i can run natively that can generate high quality videos

Knowledge Engineering Group (KEG) & Data Mining at Tsinghua University org

We have already tested the new model, and in the upcoming open-source release, this issue will no longer occur.​

I use the model with diffusers. Will the model automatically update to the newest version or do i need to do something that will update the model

Knowledge Engineering Group (KEG) & Data Mining at Tsinghua University org

Oh, that will be a new model. You’ll need to download the model weights again. This model will have more parameters and better quality. We expect to release it before September.

Knowledge Engineering Group (KEG) & Data Mining at Tsinghua University org

Once it’s completed, I’ll remind you here. Generally speaking, it should also be able to run on a 4090 GPU

Do the hardware requirements change for the new version . Ifbthe hardware requirements stay the same then it should rin on a 4070 GPU and 32 GB of RAM (woth model offload turned on)

Knowledge Engineering Group (KEG) & Data Mining at Tsinghua University org
edited 28 days ago

Oh, the requirements will definitely change. These are two different model sizes. The 2B model was a small experiment, and we plan to open-source a 5B model. Running this full model is expected to require 21GB of GPU memory, while the 2B model only needs 12GB.

what part of the model will scale. the VAE or the Transformer and is there a planned realease data for this new version

Knowledge Engineering Group (KEG) & Data Mining at Tsinghua University org

the transformers part is a larger size(5B),public now. the model info is on the readme page

Sign up or log in to comment