Update chat template to behave identically to Mistral's InstructTokenizerV3
Browse files- Mistral-7B-Instruct-v0.3.IQ1_M.gguf +2 -2
- Mistral-7B-Instruct-v0.3.IQ1_S.gguf +2 -2
- Mistral-7B-Instruct-v0.3.IQ2_M.gguf +2 -2
- Mistral-7B-Instruct-v0.3.IQ2_S.gguf +2 -2
- Mistral-7B-Instruct-v0.3.IQ2_XS.gguf +2 -2
- Mistral-7B-Instruct-v0.3.IQ2_XXS.gguf +2 -2
- Mistral-7B-Instruct-v0.3.IQ3_M.gguf +2 -2
- Mistral-7B-Instruct-v0.3.IQ3_S.gguf +2 -2
- Mistral-7B-Instruct-v0.3.IQ3_XS.gguf +2 -2
- Mistral-7B-Instruct-v0.3.IQ3_XXS.gguf +2 -2
- Mistral-7B-Instruct-v0.3.IQ4_XS.gguf +2 -2
- Mistral-7B-Instruct-v0.3.fp16.gguf +2 -2
- README.md +2 -2
Mistral-7B-Instruct-v0.3.IQ1_M.gguf
CHANGED
@@ -1,3 +1,3 @@
|
|
1 |
version https://git-lfs.github.com/spec/v1
|
2 |
-
oid sha256:
|
3 |
-
size
|
|
|
1 |
version https://git-lfs.github.com/spec/v1
|
2 |
+
oid sha256:b2d65cb3c1d02d1fae2b2a2eadf6a41cc5c98533dc15dea66a541eaa69393c59
|
3 |
+
size 1757664096
|
Mistral-7B-Instruct-v0.3.IQ1_S.gguf
CHANGED
@@ -1,3 +1,3 @@
|
|
1 |
version https://git-lfs.github.com/spec/v1
|
2 |
-
oid sha256:
|
3 |
-
size
|
|
|
1 |
version https://git-lfs.github.com/spec/v1
|
2 |
+
oid sha256:dc9492b72a4b738eae124e4f3e54e68be4141d9bd0cd939524ab9acaa7433828
|
3 |
+
size 1615319904
|
Mistral-7B-Instruct-v0.3.IQ2_M.gguf
CHANGED
@@ -1,3 +1,3 @@
|
|
1 |
version https://git-lfs.github.com/spec/v1
|
2 |
-
oid sha256:
|
3 |
-
size
|
|
|
1 |
version https://git-lfs.github.com/spec/v1
|
2 |
+
oid sha256:10c6cd4d8c5ea48a4ecdcbb6fd3e25f3947a7c541f90203564bfd2bf3c6427ba
|
3 |
+
size 2504250208
|
Mistral-7B-Instruct-v0.3.IQ2_S.gguf
CHANGED
@@ -1,3 +1,3 @@
|
|
1 |
version https://git-lfs.github.com/spec/v1
|
2 |
-
oid sha256:
|
3 |
-
size
|
|
|
1 |
version https://git-lfs.github.com/spec/v1
|
2 |
+
oid sha256:6e3c35069f8e0d60bf8bb86ce7bcf295b1f8e4ed25f921362ee5116ccd98117c
|
3 |
+
size 2314457952
|
Mistral-7B-Instruct-v0.3.IQ2_XS.gguf
CHANGED
@@ -1,3 +1,3 @@
|
|
1 |
version https://git-lfs.github.com/spec/v1
|
2 |
-
oid sha256:
|
3 |
-
size
|
|
|
1 |
version https://git-lfs.github.com/spec/v1
|
2 |
+
oid sha256:38c3feb885912c6d433dea57ddc9c9502576df460968552edfc22934196f0561
|
3 |
+
size 2201473888
|
Mistral-7B-Instruct-v0.3.IQ2_XXS.gguf
CHANGED
@@ -1,3 +1,3 @@
|
|
1 |
version https://git-lfs.github.com/spec/v1
|
2 |
-
oid sha256:
|
3 |
-
size
|
|
|
1 |
version https://git-lfs.github.com/spec/v1
|
2 |
+
oid sha256:c5b782d4e5e65ed0bcda50eb1566428952badf1c1460e4e0a58a05295bb6cba1
|
3 |
+
size 1994904416
|
Mistral-7B-Instruct-v0.3.IQ3_M.gguf
CHANGED
@@ -1,3 +1,3 @@
|
|
1 |
version https://git-lfs.github.com/spec/v1
|
2 |
-
oid sha256:
|
3 |
-
size
|
|
|
1 |
version https://git-lfs.github.com/spec/v1
|
2 |
+
oid sha256:472e1279d1f929546d0ac17e1d94092a74bfa61893a79232b71f0d0b46290459
|
3 |
+
size 3288847200
|
Mistral-7B-Instruct-v0.3.IQ3_S.gguf
CHANGED
@@ -1,3 +1,3 @@
|
|
1 |
version https://git-lfs.github.com/spec/v1
|
2 |
-
oid sha256:
|
3 |
-
size
|
|
|
1 |
version https://git-lfs.github.com/spec/v1
|
2 |
+
oid sha256:7f0ec6ea42f4f2b1c36c794e5b82b6e44214e93dab966f99a3b9ad1e7e5fa822
|
3 |
+
size 3186348896
|
Mistral-7B-Instruct-v0.3.IQ3_XS.gguf
CHANGED
@@ -1,3 +1,3 @@
|
|
1 |
version https://git-lfs.github.com/spec/v1
|
2 |
-
oid sha256:
|
3 |
-
size
|
|
|
1 |
version https://git-lfs.github.com/spec/v1
|
2 |
+
oid sha256:228654d681e2339ac7b419056d75de460a0191815e72844f54fe893aa26073e3
|
3 |
+
size 3022771040
|
Mistral-7B-Instruct-v0.3.IQ3_XXS.gguf
CHANGED
@@ -1,3 +1,3 @@
|
|
1 |
version https://git-lfs.github.com/spec/v1
|
2 |
-
oid sha256:
|
3 |
-
size
|
|
|
1 |
version https://git-lfs.github.com/spec/v1
|
2 |
+
oid sha256:8a3c6fc31d093c4c068da8a8939a59e13e67c67136028dddd4ab44b1d396eb62
|
3 |
+
size 2830881632
|
Mistral-7B-Instruct-v0.3.IQ4_XS.gguf
CHANGED
@@ -1,3 +1,3 @@
|
|
1 |
version https://git-lfs.github.com/spec/v1
|
2 |
-
oid sha256:
|
3 |
-
size
|
|
|
1 |
version https://git-lfs.github.com/spec/v1
|
2 |
+
oid sha256:aa5ad02ed8f6f7531c75f81fd5eea88c7c5ddd749ec3b5a5d72900c08a2e28ea
|
3 |
+
size 3911963488
|
Mistral-7B-Instruct-v0.3.fp16.gguf
CHANGED
@@ -1,3 +1,3 @@
|
|
1 |
version https://git-lfs.github.com/spec/v1
|
2 |
-
oid sha256:
|
3 |
-
size
|
|
|
1 |
version https://git-lfs.github.com/spec/v1
|
2 |
+
oid sha256:117c7ce7495e69a13ae556c1814814d35f6d5025a0b2b9099931597eaa424b84
|
3 |
+
size 14497337984
|
README.md
CHANGED
@@ -30,7 +30,7 @@ The embedded chat template has been extended to support function calling via Ope
|
|
30 |
## Prompt template: Mistral v3
|
31 |
|
32 |
```
|
33 |
-
[AVAILABLE_TOOLS][{"name": "function_name", "description": "Description", "parameters": {...}}, ...][/AVAILABLE_TOOLS][INST] {prompt}
|
34 |
```
|
35 |
|
36 |
<!-- prompt-template end -->
|
@@ -96,7 +96,7 @@ Generated importance matrix file: [Mistral-7B-Instruct-v0.3.imatrix.dat](https:/
|
|
96 |
Make sure you are using `llama.cpp` from commit [0becb22](https://github.com/ggerganov/llama.cpp/commit/0becb22ac05b6542bd9d5f2235691aa1d3d4d307) or later.
|
97 |
|
98 |
```shell
|
99 |
-
./main -ngl 33 -m Mistral-7B-Instruct-v0.3.IQ4_XS.gguf --color -c 32768 --temp 0 --repeat-penalty 1.1 -p "[AVAILABLE_TOOLS]{tools}[/AVAILABLE_TOOLS][INST] {prompt}
|
100 |
```
|
101 |
|
102 |
Change `-ngl 33` to the number of layers to offload to GPU. Remove it if you don't have GPU acceleration.
|
|
|
30 |
## Prompt template: Mistral v3
|
31 |
|
32 |
```
|
33 |
+
[AVAILABLE_TOOLS] [{"name": "function_name", "description": "Description", "parameters": {...}}, ...][/AVAILABLE_TOOLS][INST] {prompt}[/INST]
|
34 |
```
|
35 |
|
36 |
<!-- prompt-template end -->
|
|
|
96 |
Make sure you are using `llama.cpp` from commit [0becb22](https://github.com/ggerganov/llama.cpp/commit/0becb22ac05b6542bd9d5f2235691aa1d3d4d307) or later.
|
97 |
|
98 |
```shell
|
99 |
+
./main -ngl 33 -m Mistral-7B-Instruct-v0.3.IQ4_XS.gguf --color -c 32768 --temp 0 --repeat-penalty 1.1 -p "[AVAILABLE_TOOLS] {tools}[/AVAILABLE_TOOLS][INST] {prompt}[/INST]"
|
100 |
```
|
101 |
|
102 |
Change `-ngl 33` to the number of layers to offload to GPU. Remove it if you don't have GPU acceleration.
|