Hugging Face's logo

leafspark
/
Reflection-Llama-3.1-70B-GGUF

Text Generation
GGUF
ggml
llama
Inference Endpoints
conversational
Model card Files Community
3
leafspark commited on
Commit
c8908ad
•
1 Parent(s): 52249b8

Add metadata

Browse files
Files changed (1) hide show
  1. README.md +28 -6
README.md CHANGED
@@ -2,6 +2,15 @@
2
  license: llama3.1
3
  language:
4
  - en
 
 
 
 
 
 
 
 
 
5
  base_model: mattshumer/ref_70_e3
6
  pipeline_tag: text-generation
7
  library_name: ggml
@@ -22,22 +31,35 @@ GGUF quantized models of [mattshumer/ref_70_e3](https://huggingface.co/mattshume
22
  | Quantization | Size | Split | iMatrix |
23
  | ------------ | ------ | ----- | ------- |
24
  | FP16 | 141GB | true | false |
25
- | Q8_0_L | 73GB | true | false |
26
- | Q6_K_L | 56.2GB | true | false |
 
27
  | Q6_K | ??.?GB | true | false |
28
  | Q5_K_L | 52.6GB | true | false |
29
  | Q5_K_M | ??.?GB | true | false |
30
- | Q5_K_S | ??.?GB | false | false |
31
- | Q4_K_L | 42.1GB | false | false |
32
  | Q4_K_M | ??.?GB | false | false |
33
  | Q4_K_S | ??.?GB | false | false |
34
- | Q3_K_XL | ??.?GB | false | false |
35
- | Q3_K_L | 40GB | false | false |
 
 
36
  | Q3_K_M | ??.?GB | false | false |
 
37
  | Q3_K_S | ??.?GB | false | false |
 
38
  | Q2_K_L | 29.4GB | false | false |
 
 
39
  | Q2_K | ??.?GB | false | false |
40
  | Q2_K_S | ??.?GB | false | true |
 
 
 
 
 
 
41
 
42
  The `_L` or `_XL` suffix means that the token embeddings and output weight are at fp16 precision.
43
 
 
2
  license: llama3.1
3
  language:
4
  - en
5
+ - de
6
+ - fr
7
+ - it
8
+ - pt
9
+ - hi
10
+ - es
11
+ - th
12
+ tags:
13
+ - llama
14
  base_model: mattshumer/ref_70_e3
15
  pipeline_tag: text-generation
16
  library_name: ggml
 
31
  | Quantization | Size | Split | iMatrix |
32
  | ------------ | ------ | ----- | ------- |
33
  | FP16 | 141GB | true | false |
34
+ | Q8_0_L | ??.?GB | true | false |
35
+ | Q8_0 | ??.?GB | true | false |
36
+ | Q6_K_L | ??.?GB | true | false |
37
  | Q6_K | ??.?GB | true | false |
38
  | Q5_K_L | 52.6GB | true | false |
39
  | Q5_K_M | ??.?GB | true | false |
40
+ | Q5_K_S | 48.7GB | false | false |
41
+ | Q4_K_L | 45.3GB | false | false |
42
  | Q4_K_M | ??.?GB | false | false |
43
  | Q4_K_S | ??.?GB | false | false |
44
+ | IQ4_NL | ??.?GB | false | true |
45
+ | IQ4_XS | ??.?GB | false | true |
46
+ | Q3_K_XL | 37.2GB | false | false |
47
+ | Q3_K_L | ??.?GB | false | false |
48
  | Q3_K_M | ??.?GB | false | false |
49
+ | IQ3_M | ??.?GB | false | true |
50
  | Q3_K_S | ??.?GB | false | false |
51
+ | IQ3_S | ??.?GB | false | true |
52
  | Q2_K_L | 29.4GB | false | false |
53
+ | IQ3_XS | ??.?GB | false | true |
54
+ | IQ3_XXS | ??.?GB | false | true |
55
  | Q2_K | ??.?GB | false | false |
56
  | Q2_K_S | ??.?GB | false | true |
57
+ | IQ2_M | ??.?GB | false | true |
58
+ | IQ2_S | ??.?GB | false | true |
59
+ | IQ2_XS | ??.?GB | false | true |
60
+ | IQ2_XXS | ??.?GB | false | true |
61
+ | IQ1_M | ??.?GB | false | true |
62
+ | IQ1_S | ??.?GB | false | true |
63
 
64
  The `_L` or `_XL` suffix means that the token embeddings and output weight are at fp16 precision.
65