Spaces:
Running
on
A10G
Add `hf_transfer` for faster download
Hey there 👋
Apparently, this space takes a lot of time (~1h?) to initialize as it downloads heavy files at startup. To speed up initialization, you can try to use the new hf_transfer lib that should speed up downloads (up to 2x). It is a Rust-based library developed by
@Narsil
that will be integrated in next version of huggingface_hub. You can already get it from the main branch on Github. It is still experimental so feedback is very welcome here.
You must also set HF_HUB_ENABLE_HF_TRANSFER=1 as environment variable in the Space settings.
Related to internal slack discussion.
Worth mentioning: the caveat of using hf_transfer is that you don't have stuff like progress bars when downloading files. Not a real issue though, just don't be surprise if it seems "blocked" in the logs :)
Another note for future readers. hf_transfer only helps on the highest grades of machine where network allows real high bandwidth. On most common machines, the regular code already saturates the bandwidth.
@Wauplin could the update on requirements.txt caused this ?
runtime error
launch timed out, space was not healthy after 30 min
Container logs:
unable to retrieve container logs