Updating model files
Browse files
README.md
CHANGED
@@ -4,6 +4,17 @@ datasets:
|
|
4 |
- c-s-ale/alpaca-gpt4-data
|
5 |
pipeline_tag: text2text-generation
|
6 |
---
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
7 |
|
8 |
## GPT4-Alpaca-LoRA_MLP-65B GGML
|
9 |
|
@@ -37,15 +48,26 @@ Note: no q8_0 will be provided as HF won't allow uploading of files larger than
|
|
37 |
|
38 |
I am investigating other methods, eg a split ZIP file, and will try to upload this soon.
|
39 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
40 |
# Original model card
|
41 |
|
42 |
-
This repo provides the training checkpoint of LLaMA on the alpaca_data_gpt4 dataset via LoRA [MLP] on 8xA100(80G).
|
43 |
|
44 |
He et al. 2022 gave an insight that FFN can better utilize modification at larger capacities.
|
45 |
|
46 |
The codes is provided by [tloen/alpaca-lora: Instruct-tune LLaMA on consumer hardware (github.com)](https://github.com/tloen/alpaca-lora).
|
47 |
|
48 |
-
We modify the running scripts to
|
49 |
```bash
|
50 |
torchrun --nproc_per_node=8 finetune.py \
|
51 |
--base_model '/cache1/chtan/large_models/llama-hf/llama-65b' \
|
@@ -69,7 +91,7 @@ torchrun --nproc_per_node=8 finetune.py \
|
|
69 |
|
70 |
**Instruction**: Tell me about alpacas.
|
71 |
|
72 |
-
**gpt4-alpaca-lora_mlp-65b**:
|
73 |
|
74 |
Alpacas are small, domesticated mammals that are closely related to llamas. They are native to the Andes Mountains of South America, primarily in Peru, Bolivia, and Chile. These animals have been domesticated for thousands of years and were used by the Incas for their fleece, meat, and as pack animals.
|
75 |
|
|
|
4 |
- c-s-ale/alpaca-gpt4-data
|
5 |
pipeline_tag: text2text-generation
|
6 |
---
|
7 |
+
<div style="width: 100%;">
|
8 |
+
<img src="https://i.imgur.com/EBdldam.jpg" alt="TheBlokeAI" style="width: 100%; min-width: 400px; display: block; margin: auto;">
|
9 |
+
</div>
|
10 |
+
<div style="display: flex; justify-content: space-between; width: 100%;">
|
11 |
+
<div style="display: flex; flex-direction: column; align-items: flex-start;">
|
12 |
+
<p><a href="https://discord.gg/UBgz4VXf">Chat & support: my new Discord server</a></p>
|
13 |
+
</div>
|
14 |
+
<div style="display: flex; flex-direction: column; align-items: flex-end;">
|
15 |
+
<p><a href="https://www.patreon.com/TheBlokeAI">Want to contribute? Patreon coming soon!</a></p>
|
16 |
+
</div>
|
17 |
+
</div>
|
18 |
|
19 |
## GPT4-Alpaca-LoRA_MLP-65B GGML
|
20 |
|
|
|
48 |
|
49 |
I am investigating other methods, eg a split ZIP file, and will try to upload this soon.
|
50 |
|
51 |
+
## Want to support my work?
|
52 |
+
|
53 |
+
I've had a lot of people ask if they can contribute. I love providing models and helping people, but it is starting to rack up pretty big cloud computing bills.
|
54 |
+
|
55 |
+
So if you're able and willing to contribute, it'd be most gratefully received and will help me to keep providing models, and work on various AI projects.
|
56 |
+
|
57 |
+
Donaters will get priority support on any and all AI/LLM/model questions, and I'll gladly quantise any model you'd like to try.
|
58 |
+
|
59 |
+
* Patreon: coming soon! (just awaiting approval)
|
60 |
+
* Ko-Fi: https://ko-fi.com/TheBlokeAI
|
61 |
+
* Discord: https://discord.gg/UBgz4VXf
|
62 |
# Original model card
|
63 |
|
64 |
+
This repo provides the training checkpoint of LLaMA on the alpaca_data_gpt4 dataset via LoRA [MLP] on 8xA100(80G).
|
65 |
|
66 |
He et al. 2022 gave an insight that FFN can better utilize modification at larger capacities.
|
67 |
|
68 |
The codes is provided by [tloen/alpaca-lora: Instruct-tune LLaMA on consumer hardware (github.com)](https://github.com/tloen/alpaca-lora).
|
69 |
|
70 |
+
We modify the running scripts to
|
71 |
```bash
|
72 |
torchrun --nproc_per_node=8 finetune.py \
|
73 |
--base_model '/cache1/chtan/large_models/llama-hf/llama-65b' \
|
|
|
91 |
|
92 |
**Instruction**: Tell me about alpacas.
|
93 |
|
94 |
+
**gpt4-alpaca-lora_mlp-65b**:
|
95 |
|
96 |
Alpacas are small, domesticated mammals that are closely related to llamas. They are native to the Andes Mountains of South America, primarily in Peru, Bolivia, and Chile. These animals have been domesticated for thousands of years and were used by the Incas for their fleece, meat, and as pack animals.
|
97 |
|