Commit
•
0a6a2da
1
Parent(s):
3a6b7dc
Update README.md
Browse files
README.md
CHANGED
|
@@ -13,6 +13,7 @@ license: apache-2.0
|
|
| 13 |
### Finetuning Overview:
|
| 14 |
|
| 15 |
**Model Used:** gpt2
|
|
|
|
| 16 |
**Dataset:** HuggingFaceH4/no_robots
|
| 17 |
|
| 18 |
#### Dataset Insights:
|
|
@@ -34,22 +35,17 @@ With the utilization of [MonsterAPI](https://monsterapi.ai)'s [LLM finetuner](ht
|
|
| 34 |
- **Total Finetuning Cost:** $0.101
|
| 35 |
- **Model Path:** gpt2
|
| 36 |
- **Learning Rate:** 0.0002
|
| 37 |
-
- **Data Split:**
|
| 38 |
- **Gradient Accumulation Steps:** 4
|
| 39 |
- **lora r:** 32
|
| 40 |
- **lora alpha:** 64
|
| 41 |
|
| 42 |
-
|
| 43 |
-
Prompt Structure
|
| 44 |
```
|
| 45 |
-
|
| 46 |
-
[instruction]
|
| 47 |
-
|
| 48 |
-
### RESPONSE:
|
| 49 |
-
[output]
|
| 50 |
```
|
| 51 |
-
Training loss :
|
| 52 |
-
|
| 53 |
|
|
|
|
| 54 |
|
| 55 |
license: apache-2.0
|
|
|
|
| 13 |
### Finetuning Overview:
|
| 14 |
|
| 15 |
**Model Used:** gpt2
|
| 16 |
+
|
| 17 |
**Dataset:** HuggingFaceH4/no_robots
|
| 18 |
|
| 19 |
#### Dataset Insights:
|
|
|
|
| 35 |
- **Total Finetuning Cost:** $0.101
|
| 36 |
- **Model Path:** gpt2
|
| 37 |
- **Learning Rate:** 0.0002
|
| 38 |
+
- **Data Split:** 100% train
|
| 39 |
- **Gradient Accumulation Steps:** 4
|
| 40 |
- **lora r:** 32
|
| 41 |
- **lora alpha:** 64
|
| 42 |
|
| 43 |
+
#### Prompt Structure
|
|
|
|
| 44 |
```
|
| 45 |
+
<|system|> <|endoftext|> <|user|> [USER PROMPT]<|endoftext|> <|assistant|> [ASSISTANT ANSWER] <|endoftext|>
|
|
|
|
|
|
|
|
|
|
|
|
|
| 46 |
```
|
| 47 |
+
#### Training loss :
|
|
|
|
| 48 |
|
| 49 |
+
|
| 50 |
|
| 51 |
license: apache-2.0
|