a 2b model in gguf format is 10Gb?
That's right! The weights are store in fp32 precision (see here) meaning 2.6B params * 4 bytes per param = ~10GB.
2.6B params * 4 bytes per param = ~10GB
· Sign up or log in to comment
