Will it run on a 4090?
#1
by
brifl
- opened
I ran this successfully on a home PC with a GeForce 4090 RTX. It ran on oobabooga. I'm sharing my success or failure of running these models larger than 13b locally to save some people time, disk, and bandwidth.
This model hallucinates pretty bad. I may try changing some settings like temperature so I can leverage the huge context window, but for now I will move on to others.