Text Generation
Transformers
Safetensors
English
llama
causal-lm
text-generation-inference
4-bit precision
gptq
TheBloke commited on
Commit
7e7c4d7
1 Parent(s): e202309

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +14 -14
README.md CHANGED
@@ -24,6 +24,20 @@ This model works best with the following prompt template:
24
  ### Assistant:
25
  ```
26
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
27
  ## GIBBERISH OUTPUT IN `text-generation-webui`?
28
 
29
  Please read the Provided Files section below. You should use `stable-vicuna-13B-GPTQ-4bit.compat.no-act-order.safetensors` unless you are able to use the latest GPTQ-for-LLaMa code.
@@ -59,20 +73,6 @@ Unless you are able to use the latest GPTQ-for-LLaMa code, please use `stable-vi
59
  CUDA_VISIBLE_DEVICES=0 python3 llama.py stable-vicuna-13B-HF c4 --wbits 4 --true-sequential --act-order --groupsize 128 --save_safetensors stable-vicuna-13B-GPTQ-4bit.act-order.safetensors
60
  ```
61
 
62
- ## How to easily download and use a model in text-generation-webui
63
-
64
- Load text-generation-webui as you normally do.
65
-
66
- 1. Click the **Model tab**.
67
- 2. Under **Download custom model or LoRA**, enter the repo name to download: `TheBloke/stable-vicuna-13B-GPTQ`.
68
- 3. Click **Download**.
69
- 4. Wait until it says it's finished downloading.
70
- 5. As this is a GPTQ model, fill in the `GPTQ parameters` on the right: `Bits = 4`, `Groupsize = 128`, `model_type = Llama`
71
- 6. Now click the **Refresh** icon next to **Model** in the top left.
72
- 7. In the **Model drop-down**: choose the model you just downloaded, eg `stable-vicuna-13B-GPTQ`.
73
- 8. Click **Reload the Model** in the top right.
74
- 9. Once it says it's loaded, click the **Text Generation tab** and enter a prompt!
75
-
76
  ## Manual instructions for `text-generation-webui`
77
 
78
  File `stable-vicuna-13B-GPTQ-4bit.compat.no-act-order.safetensors` can be loaded the same as any other GPTQ file, without requiring any updates to [oobaboogas text-generation-webui](https://github.com/oobabooga/text-generation-webui).
 
24
  ### Assistant:
25
  ```
26
 
27
+ ## How to easily download and use this model in text-generation-webui
28
+
29
+ Load text-generation-webui as you normally do.
30
+
31
+ 1. Click the **Model tab**.
32
+ 2. Under **Download custom model or LoRA**, enter the repo name to download: `TheBloke/stable-vicuna-13B-GPTQ`.
33
+ 3. Click **Download**.
34
+ 4. Wait until it says it's finished downloading.
35
+ 5. As this is a GPTQ model, fill in the `GPTQ parameters` on the right: `Bits = 4`, `Groupsize = 128`, `model_type = Llama`
36
+ 6. Now click the **Refresh** icon next to **Model** in the top left.
37
+ 7. In the **Model drop-down**: choose the model you just downloaded, eg `stable-vicuna-13B-GPTQ`.
38
+ 8. Click **Reload the Model** in the top right.
39
+ 9. Once it says it's loaded, click the **Text Generation tab** and enter a prompt!
40
+
41
  ## GIBBERISH OUTPUT IN `text-generation-webui`?
42
 
43
  Please read the Provided Files section below. You should use `stable-vicuna-13B-GPTQ-4bit.compat.no-act-order.safetensors` unless you are able to use the latest GPTQ-for-LLaMa code.
 
73
  CUDA_VISIBLE_DEVICES=0 python3 llama.py stable-vicuna-13B-HF c4 --wbits 4 --true-sequential --act-order --groupsize 128 --save_safetensors stable-vicuna-13B-GPTQ-4bit.act-order.safetensors
74
  ```
75
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
76
  ## Manual instructions for `text-generation-webui`
77
 
78
  File `stable-vicuna-13B-GPTQ-4bit.compat.no-act-order.safetensors` can be loaded the same as any other GPTQ file, without requiring any updates to [oobaboogas text-generation-webui](https://github.com/oobabooga/text-generation-webui).