Hugging Face's logo

knifeayumu
/
Llama-3.1-Herrsimian-8B-GGUF

Transformers
GGUF
English
Inference Endpoints
Model card Files Community
knifeayumu commited on
Commit
3fa7052
•
1 Parent(s): d21561c

Upload 12 files

Browse files
Files changed (13) hide show
  1. .gitattributes +11 -0
  2. Llama-3.1-Herrsimian-8B-F16.gguf +3 -0
  3. Llama-3.1-Herrsimian-8B-Q2_K.gguf +3 -0
  4. Llama-3.1-Herrsimian-8B-Q3_K_L.gguf +3 -0
  5. Llama-3.1-Herrsimian-8B-Q3_K_M.gguf +3 -0
  6. Llama-3.1-Herrsimian-8B-Q3_K_S.gguf +3 -0
  7. Llama-3.1-Herrsimian-8B-Q4_K_M.gguf +3 -0
  8. Llama-3.1-Herrsimian-8B-Q4_K_S.gguf +3 -0
  9. Llama-3.1-Herrsimian-8B-Q5_K_M.gguf +3 -0
  10. Llama-3.1-Herrsimian-8B-Q5_K_S.gguf +3 -0
  11. Llama-3.1-Herrsimian-8B-Q6_K.gguf +3 -0
  12. Llama-3.1-Herrsimian-8B-Q8_0.gguf +3 -0
  13. README.md +22 -0
.gitattributes CHANGED
@@ -33,3 +33,14 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
33
  *.zip filter=lfs diff=lfs merge=lfs -text
34
  *.zst filter=lfs diff=lfs merge=lfs -text
35
  *tfevents* filter=lfs diff=lfs merge=lfs -text
 
 
 
 
 
 
 
 
 
 
 
 
33
  *.zip filter=lfs diff=lfs merge=lfs -text
34
  *.zst filter=lfs diff=lfs merge=lfs -text
35
  *tfevents* filter=lfs diff=lfs merge=lfs -text
36
+ Llama-3.1-Herrsimian-8B-F16.gguf filter=lfs diff=lfs merge=lfs -text
37
+ Llama-3.1-Herrsimian-8B-Q2_K.gguf filter=lfs diff=lfs merge=lfs -text
38
+ Llama-3.1-Herrsimian-8B-Q3_K_L.gguf filter=lfs diff=lfs merge=lfs -text
39
+ Llama-3.1-Herrsimian-8B-Q3_K_M.gguf filter=lfs diff=lfs merge=lfs -text
40
+ Llama-3.1-Herrsimian-8B-Q3_K_S.gguf filter=lfs diff=lfs merge=lfs -text
41
+ Llama-3.1-Herrsimian-8B-Q4_K_M.gguf filter=lfs diff=lfs merge=lfs -text
42
+ Llama-3.1-Herrsimian-8B-Q4_K_S.gguf filter=lfs diff=lfs merge=lfs -text
43
+ Llama-3.1-Herrsimian-8B-Q5_K_M.gguf filter=lfs diff=lfs merge=lfs -text
44
+ Llama-3.1-Herrsimian-8B-Q5_K_S.gguf filter=lfs diff=lfs merge=lfs -text
45
+ Llama-3.1-Herrsimian-8B-Q6_K.gguf filter=lfs diff=lfs merge=lfs -text
46
+ Llama-3.1-Herrsimian-8B-Q8_0.gguf filter=lfs diff=lfs merge=lfs -text
Llama-3.1-Herrsimian-8B-F16.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:9bbb87e3171c6e6ceed48baa7e46e300dcc018a2f444fb6bfc479fc0872c4916
3
+ size 16068891424
Llama-3.1-Herrsimian-8B-Q2_K.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:a8935075a3c2e57cc6613d29422aea4549df5311c42bd74030331db1a78ddd24
3
+ size 3179131680
Llama-3.1-Herrsimian-8B-Q3_K_L.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:0712bc7f7bafd73415f073fe60b2d0a1e8275d9b3a1661e66a417e230622e527
3
+ size 4321956640
Llama-3.1-Herrsimian-8B-Q3_K_M.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:d36d9592bf0461fda6c17ed09cff29c701af551a921786923e5d9870f252b635
3
+ size 4018918176
Llama-3.1-Herrsimian-8B-Q3_K_S.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:7df5d1a3c7ab53ed62f2d2d1dc75ac80e794c48ae9d3779dcfe5e832daa729a3
3
+ size 3664499488
Llama-3.1-Herrsimian-8B-Q4_K_M.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:7233249a501ce85950f78ff54520bd3b81a93deb6db6e2029f81c260187852fa
3
+ size 4920734496
Llama-3.1-Herrsimian-8B-Q4_K_S.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:9233db40a7bd60c3974b55b0b03e14f4fe7a961ac9f3c742dee87412f01c6074
3
+ size 4692669216
Llama-3.1-Herrsimian-8B-Q5_K_M.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:57d05faafb877e46792171db8a4756c9efd2cc1f26ea3445d6e34d729be5cb07
3
+ size 5732987680
Llama-3.1-Herrsimian-8B-Q5_K_S.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:dc22c7e25c7e2d09c6ae731b3aec8d39112d7a33ca39e1babdeb8d1e517a522f
3
+ size 5599294240
Llama-3.1-Herrsimian-8B-Q6_K.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:46564dfffde9151ead602aa2cbb04c206c05aa6bbcd5a468b726a2cabfcec13f
3
+ size 6596006688
Llama-3.1-Herrsimian-8B-Q8_0.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:7dff0c6b54652aaedb2f7be53bdab147a689b4087d966277f83f3d9487127b46
3
+ size 8540771104
README.md ADDED
@@ -0,0 +1,22 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ## Llamacpp Quantizations of Llama-3.1-Herrsimian-8B
2
+
3
+ Using <a href="https://github.com/ggerganov/llama.cpp/">llama.cpp</a> release <a href="https://github.com/ggerganov/llama.cpp/releases/tag/b3703">b3703</a> for quantization.
4
+
5
+ Original model: https://huggingface.co/lemonilia/Llama-3.1-Herrsimian-8B
6
+
7
+
8
+ ## Quant Types:
9
+
10
+ | Filename | Quant type | File Size | Required VRAM at 32k ctx |
11
+ | -------- | ---------- | --------- | ------------------------ |
12
+ | [Llama-3.1-Herrsimian-8B-F16](https://huggingface.co/knifeayumu/Llama-3.1-Herrsimian-8B-GGUF/blob/main/Llama-3.1-Herrsimian-8B-F16.gguf) | F16 | 14.9GB | 18.6GB |
13
+ | [Llama-3.1-Herrsimian-8B-Q8_0.gguf](https://huggingface.co/knifeayumu/Llama-3.1-Herrsimian-8B-GGUF/blob/main/Llama-3.1-Herrsimian-8B-Q8_0.gguf) | Q8_0 | 7.95GB | 14.0GB |
14
+ | [Llama-3.1-Herrsimian-8B-Q6_K.gguf](https://huggingface.co/knifeayumu/Llama-3.1-Herrsimian-8B-GGUF/blob/main/Llama-3.1-Herrsimian-8B-Q6_K.gguf) | Q6_K | 6.14GB | 12.2GB |
15
+ | [Llama-3.1-Herrsimian-8B-Q5_K_M.gguf](https://huggingface.co/knifeayumu/Llama-3.1-Herrsimian-8B-GGUF/blob/main/Llama-3.1-Herrsimian-8B-Q5_K_M.gguf) | Q5_K_M | 5.33GB | 11.4GB |
16
+ | [Llama-3.1-Herrsimian-8B-Q5_K_S.gguf](https://huggingface.co/knifeayumu/Llama-3.1-Herrsimian-8B-GGUF/blob/main/Llama-3.1-Herrsimian-8B-Q5_K_S.gguf) | Q5_K_S | 5.21GB | 11.3GB |
17
+ | [Llama-3.1-Herrsimian-8B-Q4_K_M.gguf](https://huggingface.co/knifeayumu/Llama-3.1-Herrsimian-8B-GGUF/blob/main/Llama-3.1-Herrsimian-8B-Q4_K_M.gguf) | Q4_K_M | 4.58GB | 10.6GB |
18
+ | [Llama-3.1-Herrsimian-8B-Q4_K_S.gguf](https://huggingface.co/knifeayumu/Llama-3.1-Herrsimian-8B-GGUF/blob/main/Llama-3.1-Herrsimian-8B-Q4_K_S.gguf) | Q4_K_S | 4.37GB | 10.4GB |
19
+ | [Llama-3.1-Herrsimian-8B-Q3_K_L.gguf](https://huggingface.co/knifeayumu/Llama-3.1-Herrsimian-8B-GGUF/blob/main/Llama-3.1-Herrsimian-8B-Q3_K_L.gguf) | Q3_K_L | 4.02GB | 10.1GB |
20
+ | [Llama-3.1-Herrsimian-8B-Q3_K_M.gguf](https://huggingface.co/knifeayumu/Llama-3.1-Herrsimian-8B-GGUF/blob/main/Llama-3.1-Herrsimian-8B-Q3_K_M.gguf) | Q3_K_M | 3.74GB | 9.7GB |
21
+ | [Llama-3.1-Herrsimian-8B-Q3_K_S.gguf](https://huggingface.co/knifeayumu/Llama-3.1-Herrsimian-8B-GGUF/blob/main/Llama-3.1-Herrsimian-8B-Q3_K_S.gguf) | Q3_K_S | 3.41GB | 9.4GB |
22
+ | [Llama-3.1-Herrsimian-8B-Q2_K.gguf](https://huggingface.co/knifeayumu/Llama-3.1-Herrsimian-8B-GGUF/blob/main/Llama-3.1-Herrsimian-8B-Q2_K.gguf) | Q2_K | 2.95GB | 9.2GB |