Update README.md
Browse files
README.md
CHANGED
@@ -24,6 +24,20 @@ This model works best with the following prompt template:
|
|
24 |
### Assistant:
|
25 |
```
|
26 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
27 |
## GIBBERISH OUTPUT IN `text-generation-webui`?
|
28 |
|
29 |
Please read the Provided Files section below. You should use `stable-vicuna-13B-GPTQ-4bit.compat.no-act-order.safetensors` unless you are able to use the latest GPTQ-for-LLaMa code.
|
@@ -59,20 +73,6 @@ Unless you are able to use the latest GPTQ-for-LLaMa code, please use `stable-vi
|
|
59 |
CUDA_VISIBLE_DEVICES=0 python3 llama.py stable-vicuna-13B-HF c4 --wbits 4 --true-sequential --act-order --groupsize 128 --save_safetensors stable-vicuna-13B-GPTQ-4bit.act-order.safetensors
|
60 |
```
|
61 |
|
62 |
-
## How to easily download and use a model in text-generation-webui
|
63 |
-
|
64 |
-
Load text-generation-webui as you normally do.
|
65 |
-
|
66 |
-
1. Click the **Model tab**.
|
67 |
-
2. Under **Download custom model or LoRA**, enter the repo name to download: `TheBloke/stable-vicuna-13B-GPTQ`.
|
68 |
-
3. Click **Download**.
|
69 |
-
4. Wait until it says it's finished downloading.
|
70 |
-
5. As this is a GPTQ model, fill in the `GPTQ parameters` on the right: `Bits = 4`, `Groupsize = 128`, `model_type = Llama`
|
71 |
-
6. Now click the **Refresh** icon next to **Model** in the top left.
|
72 |
-
7. In the **Model drop-down**: choose the model you just downloaded, eg `stable-vicuna-13B-GPTQ`.
|
73 |
-
8. Click **Reload the Model** in the top right.
|
74 |
-
9. Once it says it's loaded, click the **Text Generation tab** and enter a prompt!
|
75 |
-
|
76 |
## Manual instructions for `text-generation-webui`
|
77 |
|
78 |
File `stable-vicuna-13B-GPTQ-4bit.compat.no-act-order.safetensors` can be loaded the same as any other GPTQ file, without requiring any updates to [oobaboogas text-generation-webui](https://github.com/oobabooga/text-generation-webui).
|
|
|
24 |
### Assistant:
|
25 |
```
|
26 |
|
27 |
+
## How to easily download and use this model in text-generation-webui
|
28 |
+
|
29 |
+
Load text-generation-webui as you normally do.
|
30 |
+
|
31 |
+
1. Click the **Model tab**.
|
32 |
+
2. Under **Download custom model or LoRA**, enter the repo name to download: `TheBloke/stable-vicuna-13B-GPTQ`.
|
33 |
+
3. Click **Download**.
|
34 |
+
4. Wait until it says it's finished downloading.
|
35 |
+
5. As this is a GPTQ model, fill in the `GPTQ parameters` on the right: `Bits = 4`, `Groupsize = 128`, `model_type = Llama`
|
36 |
+
6. Now click the **Refresh** icon next to **Model** in the top left.
|
37 |
+
7. In the **Model drop-down**: choose the model you just downloaded, eg `stable-vicuna-13B-GPTQ`.
|
38 |
+
8. Click **Reload the Model** in the top right.
|
39 |
+
9. Once it says it's loaded, click the **Text Generation tab** and enter a prompt!
|
40 |
+
|
41 |
## GIBBERISH OUTPUT IN `text-generation-webui`?
|
42 |
|
43 |
Please read the Provided Files section below. You should use `stable-vicuna-13B-GPTQ-4bit.compat.no-act-order.safetensors` unless you are able to use the latest GPTQ-for-LLaMa code.
|
|
|
73 |
CUDA_VISIBLE_DEVICES=0 python3 llama.py stable-vicuna-13B-HF c4 --wbits 4 --true-sequential --act-order --groupsize 128 --save_safetensors stable-vicuna-13B-GPTQ-4bit.act-order.safetensors
|
74 |
```
|
75 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
76 |
## Manual instructions for `text-generation-webui`
|
77 |
|
78 |
File `stable-vicuna-13B-GPTQ-4bit.compat.no-act-order.safetensors` can be loaded the same as any other GPTQ file, without requiring any updates to [oobaboogas text-generation-webui](https://github.com/oobabooga/text-generation-webui).
|