Hugging Face's logo

knifeayumu
/
Llama-3.1-Herrsimian-8B-GGUF

Transformers
GGUF
English
Inference Endpoints
Model card Files Community
knifeayumu commited on
Commit
63c2130
•
1 Parent(s): 3fa7052

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +11 -1
README.md CHANGED
@@ -1,3 +1,13 @@
 
 
 
 
 
 
 
 
 
 
1
  ## Llamacpp Quantizations of Llama-3.1-Herrsimian-8B
2
 
3
  Using <a href="https://github.com/ggerganov/llama.cpp/">llama.cpp</a> release <a href="https://github.com/ggerganov/llama.cpp/releases/tag/b3703">b3703</a> for quantization.
@@ -9,7 +19,7 @@ Original model: https://huggingface.co/lemonilia/Llama-3.1-Herrsimian-8B
9
 
10
  | Filename | Quant type | File Size | Required VRAM at 32k ctx |
11
  | -------- | ---------- | --------- | ------------------------ |
12
- | [Llama-3.1-Herrsimian-8B-F16](https://huggingface.co/knifeayumu/Llama-3.1-Herrsimian-8B-GGUF/blob/main/Llama-3.1-Herrsimian-8B-F16.gguf) | F16 | 14.9GB | 18.6GB |
13
  | [Llama-3.1-Herrsimian-8B-Q8_0.gguf](https://huggingface.co/knifeayumu/Llama-3.1-Herrsimian-8B-GGUF/blob/main/Llama-3.1-Herrsimian-8B-Q8_0.gguf) | Q8_0 | 7.95GB | 14.0GB |
14
  | [Llama-3.1-Herrsimian-8B-Q6_K.gguf](https://huggingface.co/knifeayumu/Llama-3.1-Herrsimian-8B-GGUF/blob/main/Llama-3.1-Herrsimian-8B-Q6_K.gguf) | Q6_K | 6.14GB | 12.2GB |
15
  | [Llama-3.1-Herrsimian-8B-Q5_K_M.gguf](https://huggingface.co/knifeayumu/Llama-3.1-Herrsimian-8B-GGUF/blob/main/Llama-3.1-Herrsimian-8B-Q5_K_M.gguf) | Q5_K_M | 5.33GB | 11.4GB |
 
1
+ ---
2
+ base_model:
3
+ - lemonilia/Llama-3.1-Herrsimian-8B
4
+ language:
5
+ - en
6
+ library_name: transformers
7
+ license: llama3.1
8
+ quantized_by: knifeayumu
9
+ ---
10
+
11
  ## Llamacpp Quantizations of Llama-3.1-Herrsimian-8B
12
 
13
  Using <a href="https://github.com/ggerganov/llama.cpp/">llama.cpp</a> release <a href="https://github.com/ggerganov/llama.cpp/releases/tag/b3703">b3703</a> for quantization.
 
19
 
20
  | Filename | Quant type | File Size | Required VRAM at 32k ctx |
21
  | -------- | ---------- | --------- | ------------------------ |
22
+ | [Llama-3.1-Herrsimian-8B-F16.gguf](https://huggingface.co/knifeayumu/Llama-3.1-Herrsimian-8B-GGUF/blob/main/Llama-3.1-Herrsimian-8B-F16.gguf) | F16 | 14.9GB | 18.6GB |
23
  | [Llama-3.1-Herrsimian-8B-Q8_0.gguf](https://huggingface.co/knifeayumu/Llama-3.1-Herrsimian-8B-GGUF/blob/main/Llama-3.1-Herrsimian-8B-Q8_0.gguf) | Q8_0 | 7.95GB | 14.0GB |
24
  | [Llama-3.1-Herrsimian-8B-Q6_K.gguf](https://huggingface.co/knifeayumu/Llama-3.1-Herrsimian-8B-GGUF/blob/main/Llama-3.1-Herrsimian-8B-Q6_K.gguf) | Q6_K | 6.14GB | 12.2GB |
25
  | [Llama-3.1-Herrsimian-8B-Q5_K_M.gguf](https://huggingface.co/knifeayumu/Llama-3.1-Herrsimian-8B-GGUF/blob/main/Llama-3.1-Herrsimian-8B-Q5_K_M.gguf) | Q5_K_M | 5.33GB | 11.4GB |