Spaces:
Running
on
Zero
Apply for community grant: Academic project (gpu)
Hello, our project is on active-speaker-detection and video-synchronization using gestures. Imagine you have a video of multiple people speaking and you want to find the active speaker throughout the video. Now if the faces of the people are visible, then it becomes a classical problem of active-speaker-detection using face, lip-movements and speech. The issue arises if the faces are not covered by the camera, or if the faces are unclear, occluded, lips are not visible, or people are wearing masks. In all such cases, our model can be used to find the active speaker, which is soleley based on gesture movements (sample attached). In addition, our model can also correct off-sync videos - just give an out-of-sync video as input and our model outputs the sync-corrected video. Again, in the task as well, we do not need the faces to be visible. Since the project involves videos, the availability of GPU is crucial. The current gradio space is built for both of these applications, but they are extremely slow and unusable due to the unavailibility of GPUs. Kindly approve our request and grant us a GPU for this space!
Hi @sindhuhegde , we've assigned ZeroGPU to this Space. Please check the compatibility and usage sections of this page so your Space can run on ZeroGPU.
Thanks a lot!
This is a wonderful paper. Can't wait to try it out!
Hi
@hysts
,
I am getting the following error when I try to obtain embeddings from my model (https://huggingface.co/spaces/sindhuhegde/gestsync/blob/main/app.py#L1078):
"Error: CUDA must not be initialized in the main process on Spaces with Stateless GPU environment.
You can look at this Stacktrace to find out which part of your code triggered a CUDA init"
It would be very helpful if you could take a look and point me in the right direction to resolve this issue.
@sindhuhegde
I don't see the error in the Space log. Do you have the full stack trace? It's hard to investigate the error without it.
Anyway, on ZeroGPU Spaces, CUDA is only available in functions decorated with @spaces.GPU, and the error is usually raised when you try to use CUDA outside of such functions.
(.to() of torch.nn is monkey-patched when importing spaces so that ZeroGPU backend can remember which models are supposed to move to CUDA, so it's fine to call model.to("cuda"), though.)
Not sure if this is related, but looking at the code, it seems that you are moving your model to CUDA here. Can you try instantiating your model in the global scope? On ZeroGPU Spaces, the ZeroGPU backend automatically moves model weights between CPU RAM and VRAM when needed.
OK, looks like the error is raised because returned tensors are on GPU. https://huggingface.co/spaces/sindhuhegde/gestsync/blob/5bbddf299668bc4f6f3d61283b33598577ed946f/app.py#L1114-L1121
You can avoid the error by calling .cpu().
Thanks
@hysts
for the suggestion! I have changed it, but now it seems like there is some issue in returning from the function encoded with @spaces.GPU. The code runs till this line: https://huggingface.co/spaces/sindhuhegde/gestsync/blob/main/app.py#L1116 and then never comes out of the function. The line after returning from the function does not print (https://huggingface.co/spaces/sindhuhegde/gestsync/blob/main/app.py#L1336) and then the code errors out saying GPU task aborted
@sindhuhegde
The GPU task aborted error is raised when the function takes longer than the specified duration, so adjusting it would fix the issue. The duration parameter now supports generators like this, it might be useful.
The issue was resolved when I converted the returned tensor from the model to numpy array, thanks for your assistance @hysts ! The demo is up and running now:)