Update README.md
Browse files
README.md
CHANGED
@@ -1,18 +1,20 @@
|
|
1 |
---
|
2 |
license: cc-by-sa-4.0
|
3 |
-
base_model:
|
4 |
tags:
|
5 |
- axolotl
|
6 |
- generated_from_trainer
|
7 |
model-index:
|
8 |
- name: stablelm-4e1t-2b-v0.1
|
9 |
results: []
|
|
|
|
|
10 |
---
|
11 |
|
12 |
# stablelm-4e1t-2b-v0.1
|
13 |
|
14 |
|
15 |
-
This is a layer pruning experiment based off of [stablelm-3b-4e1t](https://
|
16 |
|
17 |
- 10 layers pruned with [PruneMe](https://github.com/pszemraj/PruneMe/tree/upgrades)/MergeKit
|
18 |
- layers selected using [BEE-spoke-data/fineweb-100k_en-med](https://hf.co/datasets/BEE-spoke-data/fineweb-100k_en-med)
|
@@ -26,6 +28,9 @@ This is a layer pruning experiment based off of [stablelm-3b-4e1t](https://huggi
|
|
26 |
[<img src="https://raw.githubusercontent.com/OpenAccess-AI-Collective/axolotl/main/image/axolotl-badge-web.png" alt="Built with Axolotl" width="200" height="32"/>](https://github.com/OpenAccess-AI-Collective/axolotl)
|
27 |
<details><summary>See axolotl config</summary>
|
28 |
|
|
|
|
|
|
|
29 |
axolotl version: `0.4.0`
|
30 |
```yaml
|
31 |
base_model: pszemraj/stablelm-3b-4e1t-prune10
|
@@ -94,7 +99,6 @@ special_tokens:
|
|
94 |
|
95 |
</details><br>
|
96 |
|
97 |
-
---
|
98 |
|
99 |
### Training results
|
100 |
|
@@ -111,3 +115,4 @@ special_tokens:
|
|
111 |
| 2.2513 | 1.5905 | 2656 | 2.4761 |
|
112 |
| 2.1654 | 1.7909 | 2988 | 2.4769 |
|
113 |
|
|
|
|
1 |
---
|
2 |
license: cc-by-sa-4.0
|
3 |
+
base_model: stabilityai/stablelm-3b-4e1t
|
4 |
tags:
|
5 |
- axolotl
|
6 |
- generated_from_trainer
|
7 |
model-index:
|
8 |
- name: stablelm-4e1t-2b-v0.1
|
9 |
results: []
|
10 |
+
language:
|
11 |
+
- en
|
12 |
---
|
13 |
|
14 |
# stablelm-4e1t-2b-v0.1
|
15 |
|
16 |
|
17 |
+
This is a layer pruning experiment based off of [stablelm-3b-4e1t](https://hf.co/stabilityai/stablelm-3b-4e1t):
|
18 |
|
19 |
- 10 layers pruned with [PruneMe](https://github.com/pszemraj/PruneMe/tree/upgrades)/MergeKit
|
20 |
- layers selected using [BEE-spoke-data/fineweb-100k_en-med](https://hf.co/datasets/BEE-spoke-data/fineweb-100k_en-med)
|
|
|
28 |
[<img src="https://raw.githubusercontent.com/OpenAccess-AI-Collective/axolotl/main/image/axolotl-badge-web.png" alt="Built with Axolotl" width="200" height="32"/>](https://github.com/OpenAccess-AI-Collective/axolotl)
|
29 |
<details><summary>See axolotl config</summary>
|
30 |
|
31 |
+
### config
|
32 |
+
|
33 |
+
|
34 |
axolotl version: `0.4.0`
|
35 |
```yaml
|
36 |
base_model: pszemraj/stablelm-3b-4e1t-prune10
|
|
|
99 |
|
100 |
</details><br>
|
101 |
|
|
|
102 |
|
103 |
### Training results
|
104 |
|
|
|
115 |
| 2.2513 | 1.5905 | 2656 | 2.4761 |
|
116 |
| 2.1654 | 1.7909 | 2988 | 2.4769 |
|
117 |
|
118 |
+
---
|