Commit
•
bd1a130
1
Parent(s):
d604504
Update README.md
Browse files
README.md
CHANGED
|
@@ -13,7 +13,7 @@ should probably proofread and complete it, then remove this comment. -->
|
|
| 13 |
|
| 14 |
[<img src="https://raw.githubusercontent.com/OpenAccess-AI-Collective/axolotl/main/image/axolotl-badge-web.png" alt="Built with Axolotl" width="200" height="32"/>](https://github.com/OpenAccess-AI-Collective/axolotl)
|
| 15 |
|
| 16 |
-
# Fine-tune of
|
| 17 |
Three epochs of fine tuning with @jondurbin's SpicyBoros-3.1 dataset. 4.65bpw should fit on a single 3090/4090, 5.0bpw, 6.0bpw, and 8.0bpw will require more than one GPU 24 GB VRAM GPU.
|
| 18 |
|
| 19 |
**Please note:** you may have to turn down repetition penalty to 1.0. The model seems to get into "thesaurus" mode sometimes without this change.
|
|
|
|
| 13 |
|
| 14 |
[<img src="https://raw.githubusercontent.com/OpenAccess-AI-Collective/axolotl/main/image/axolotl-badge-web.png" alt="Built with Axolotl" width="200" height="32"/>](https://github.com/OpenAccess-AI-Collective/axolotl)
|
| 15 |
|
| 16 |
+
# Fine-tune of Yi-34B with Spicyboros-3.1
|
| 17 |
Three epochs of fine tuning with @jondurbin's SpicyBoros-3.1 dataset. 4.65bpw should fit on a single 3090/4090, 5.0bpw, 6.0bpw, and 8.0bpw will require more than one GPU 24 GB VRAM GPU.
|
| 18 |
|
| 19 |
**Please note:** you may have to turn down repetition penalty to 1.0. The model seems to get into "thesaurus" mode sometimes without this change.
|