Single T4

#1
by Taylor658 - opened

Just wanted to shout out to the C4AI team on how efficiently this space runs and returns text, audio and image results on a single T4! :-)

Cohere For AI org

Thank you, Taylor! :)

Hello, I would like to ask how to deploy this aya 32B model on a single T4

Hello, I would like to ask how to deploy this aya 32B model on a single T4

hello. I'm curious too. i have rtx 4070 gpu and 32 gb ram. i think i have more computing power than here. but i'm running it pretty slow :) is there a documentation etc about how you do it?

Cohere For AI org

Hello, we don't deploy the model, it is API calls :)

Hello, we don't deploy the model, it is API calls :)

ah bad news :)

shivi changed discussion status to closed

Sign up or log in to comment