Spaces:
Running
Running
NaN tensors during inference with Zero GPU after the September update
#128
by
anton-bushuiev
- opened
Hi 👋!
After the September update of Zero GPU, predictions on my Zero GPU Hugging Face Space fail due to NaN tensors that appear during inference on GPU. Does anyone please encounter the same issue or have any tips how to resolve this? To reproduce the bug, please run any example from my Zero GPU Hugging Face Space.
- My space used to work fine on Zero GPU before the update
- It also works fine if I run the same code locally with A100 GPU (and using same versions of packages, including torch, gradio and spaces)
- It also works fine on the Spaces if I switch my space back to CPU
- I made sure inputs do not contain NaNs, they somehow appear during inference using Zero GPU.
Thank you!
Anton
anton-bushuiev
changed discussion title from
NaN tensors after inference with Zero GPU after the September update
to NaN tensors during inference with Zero GPU after the September update