CISCai commited on
Commit
ca40449
1 Parent(s): 7487169

Update chat template to behave identically to Mistral's InstructTokenizerV3

Browse files
Mistral-7B-Instruct-v0.3.IQ1_M.gguf CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:5a4c2e132197ce0f48c2f4f305ed7c73b576484f1c4274cd5dcc7cb3d1464157
3
- size 1757663808
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:b2d65cb3c1d02d1fae2b2a2eadf6a41cc5c98533dc15dea66a541eaa69393c59
3
+ size 1757664096
Mistral-7B-Instruct-v0.3.IQ1_S.gguf CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:010412320006c0cfbec504bf49b521cd082fa3e7baa703c2521664b92d76bff4
3
- size 1615319616
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:dc9492b72a4b738eae124e4f3e54e68be4141d9bd0cd939524ab9acaa7433828
3
+ size 1615319904
Mistral-7B-Instruct-v0.3.IQ2_M.gguf CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:d772726da0d89582af1879e3bce1f206eecee4dabbf43373d20c3e3c463e033a
3
- size 2504249920
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:10c6cd4d8c5ea48a4ecdcbb6fd3e25f3947a7c541f90203564bfd2bf3c6427ba
3
+ size 2504250208
Mistral-7B-Instruct-v0.3.IQ2_S.gguf CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:5c74ffe534be1845b3de096c4bee6258eabee8f4590662a04e7cf11325dcf4df
3
- size 2314457664
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:6e3c35069f8e0d60bf8bb86ce7bcf295b1f8e4ed25f921362ee5116ccd98117c
3
+ size 2314457952
Mistral-7B-Instruct-v0.3.IQ2_XS.gguf CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:714090e00e6d3774a09e5fcb97f208d8c9caa82182001e510b9ba6748f85f9ad
3
- size 2201473600
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:38c3feb885912c6d433dea57ddc9c9502576df460968552edfc22934196f0561
3
+ size 2201473888
Mistral-7B-Instruct-v0.3.IQ2_XXS.gguf CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:7e261e54d7e79d3a2829d946f012a07463435b0ea2674a3ad9ef11065eddac34
3
- size 1994904128
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:c5b782d4e5e65ed0bcda50eb1566428952badf1c1460e4e0a58a05295bb6cba1
3
+ size 1994904416
Mistral-7B-Instruct-v0.3.IQ3_M.gguf CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:7dcc6670467d7a5bfb40428843a6c874b9e919bfa156061eeee5372815a177ab
3
- size 3288846912
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:472e1279d1f929546d0ac17e1d94092a74bfa61893a79232b71f0d0b46290459
3
+ size 3288847200
Mistral-7B-Instruct-v0.3.IQ3_S.gguf CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:33bd7373ab791bed06cc5ee9f6e4993e0a4edc308a78a6440732060056dd705b
3
- size 3186348608
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:7f0ec6ea42f4f2b1c36c794e5b82b6e44214e93dab966f99a3b9ad1e7e5fa822
3
+ size 3186348896
Mistral-7B-Instruct-v0.3.IQ3_XS.gguf CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:c38302b457361cb0e4daa4a5629d36ba7c232ad57928cc7b867c31ca9647c79b
3
- size 3022770752
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:228654d681e2339ac7b419056d75de460a0191815e72844f54fe893aa26073e3
3
+ size 3022771040
Mistral-7B-Instruct-v0.3.IQ3_XXS.gguf CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:61212d266155c5525b68791db73fcbbab7ac4f43e8357cf3b99e62f832c56c5f
3
- size 2830881344
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:8a3c6fc31d093c4c068da8a8939a59e13e67c67136028dddd4ab44b1d396eb62
3
+ size 2830881632
Mistral-7B-Instruct-v0.3.IQ4_XS.gguf CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:b217dadd62f9799bc8dcb2a06d24b8bc6c77e38027d93185dd57f392cfc2fb4c
3
- size 3911963200
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:aa5ad02ed8f6f7531c75f81fd5eea88c7c5ddd749ec3b5a5d72900c08a2e28ea
3
+ size 3911963488
Mistral-7B-Instruct-v0.3.fp16.gguf CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:7128d0c16ca917c0e5ffcb23c257d7f51bc44412900cbdeb136981bc9fb237f1
3
- size 14497337696
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:117c7ce7495e69a13ae556c1814814d35f6d5025a0b2b9099931597eaa424b84
3
+ size 14497337984
README.md CHANGED
@@ -30,7 +30,7 @@ The embedded chat template has been extended to support function calling via Ope
30
  ## Prompt template: Mistral v3
31
 
32
  ```
33
- [AVAILABLE_TOOLS][{"name": "function_name", "description": "Description", "parameters": {...}}, ...][/AVAILABLE_TOOLS][INST] {prompt} [/INST]
34
  ```
35
 
36
  <!-- prompt-template end -->
@@ -96,7 +96,7 @@ Generated importance matrix file: [Mistral-7B-Instruct-v0.3.imatrix.dat](https:/
96
  Make sure you are using `llama.cpp` from commit [0becb22](https://github.com/ggerganov/llama.cpp/commit/0becb22ac05b6542bd9d5f2235691aa1d3d4d307) or later.
97
 
98
  ```shell
99
- ./main -ngl 33 -m Mistral-7B-Instruct-v0.3.IQ4_XS.gguf --color -c 32768 --temp 0 --repeat-penalty 1.1 -p "[AVAILABLE_TOOLS]{tools}[/AVAILABLE_TOOLS][INST] {prompt} [/INST]"
100
  ```
101
 
102
  Change `-ngl 33` to the number of layers to offload to GPU. Remove it if you don't have GPU acceleration.
 
30
  ## Prompt template: Mistral v3
31
 
32
  ```
33
+ [AVAILABLE_TOOLS] [{"name": "function_name", "description": "Description", "parameters": {...}}, ...][/AVAILABLE_TOOLS][INST] {prompt}[/INST]
34
  ```
35
 
36
  <!-- prompt-template end -->
 
96
  Make sure you are using `llama.cpp` from commit [0becb22](https://github.com/ggerganov/llama.cpp/commit/0becb22ac05b6542bd9d5f2235691aa1d3d4d307) or later.
97
 
98
  ```shell
99
+ ./main -ngl 33 -m Mistral-7B-Instruct-v0.3.IQ4_XS.gguf --color -c 32768 --temp 0 --repeat-penalty 1.1 -p "[AVAILABLE_TOOLS] {tools}[/AVAILABLE_TOOLS][INST] {prompt}[/INST]"
100
  ```
101
 
102
  Change `-ngl 33` to the number of layers to offload to GPU. Remove it if you don't have GPU acceleration.