On premise deployment
#1
by
lukasz050
- opened
I am new in this world. Can this model be used locally on my own GPU and resources - Is there any simple method to run this locally?
Hey
@lukasz050
Yes you can use it locally with your GPU if you have any. I would advise you to run it in 8bit precision for a memory efficient inference
please refer to blip2-flan-t5-xxl model card: https://huggingface.co/Salesforce/blip2-flan-t5-xxl#in-8-bit-precision-int8 for more details