Model save
Browse files- README.md +171 -0
- model.safetensors +1 -1
README.md
ADDED
@@ -0,0 +1,171 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
1 |
+
---
|
2 |
+
license: cc-by-nc-4.0
|
3 |
+
base_model: nguyenvulebinh/wav2vec2-base-vietnamese-250h
|
4 |
+
tags:
|
5 |
+
- generated_from_trainer
|
6 |
+
metrics:
|
7 |
+
- wer
|
8 |
+
model-index:
|
9 |
+
- name: w2v2_ablation_focal_ctc_a0.5_g1.0-best_on-ling_head-tp0.025_tl10_fp0.001_fl16
|
10 |
+
results: []
|
11 |
+
---
|
12 |
+
|
13 |
+
<!-- This model card has been generated automatically according to the information the Trainer had access to. You
|
14 |
+
should probably proofread and complete it, then remove this comment. -->
|
15 |
+
|
16 |
+
# w2v2_ablation_focal_ctc_a0.5_g1.0-best_on-ling_head-tp0.025_tl10_fp0.001_fl16
|
17 |
+
|
18 |
+
This model is a fine-tuned version of [nguyenvulebinh/wav2vec2-base-vietnamese-250h](https://huggingface.co/nguyenvulebinh/wav2vec2-base-vietnamese-250h) on an unknown dataset.
|
19 |
+
It achieves the following results on the evaluation set:
|
20 |
+
- Loss: 1.9952
|
21 |
+
- Wer: 0.0908
|
22 |
+
|
23 |
+
## Model description
|
24 |
+
|
25 |
+
More information needed
|
26 |
+
|
27 |
+
## Intended uses & limitations
|
28 |
+
|
29 |
+
More information needed
|
30 |
+
|
31 |
+
## Training and evaluation data
|
32 |
+
|
33 |
+
More information needed
|
34 |
+
|
35 |
+
## Training procedure
|
36 |
+
|
37 |
+
### Training hyperparameters
|
38 |
+
|
39 |
+
The following hyperparameters were used during training:
|
40 |
+
- learning_rate: 2e-05
|
41 |
+
- train_batch_size: 8
|
42 |
+
- eval_batch_size: 16
|
43 |
+
- seed: 42
|
44 |
+
- distributed_type: multi-GPU
|
45 |
+
- num_devices: 4
|
46 |
+
- total_train_batch_size: 32
|
47 |
+
- total_eval_batch_size: 64
|
48 |
+
- optimizer: Adam with betas=(0.9,0.98) and epsilon=1e-08
|
49 |
+
- lr_scheduler_type: cosine
|
50 |
+
- lr_scheduler_warmup_ratio: 0.1
|
51 |
+
- num_epochs: 100
|
52 |
+
- mixed_precision_training: Native AMP
|
53 |
+
|
54 |
+
### Training results
|
55 |
+
|
56 |
+
| Training Loss | Epoch | Step | Validation Loss | Wer |
|
57 |
+
|:-------------:|:-----:|:-----:|:---------------:|:-------:|
|
58 |
+
| 891.4405 | 0.94 | 100 | 581.3978 | 18.6410 |
|
59 |
+
| 615.8164 | 1.89 | 200 | 221.5820 | 17.0065 |
|
60 |
+
| 105.0527 | 2.83 | 300 | 43.9285 | 1.0 |
|
61 |
+
| 56.2539 | 3.77 | 400 | 40.2262 | 1.0 |
|
62 |
+
| 51.7117 | 4.72 | 500 | 38.2334 | 1.0 |
|
63 |
+
| 49.7296 | 5.66 | 600 | 37.4374 | 1.0 |
|
64 |
+
| 49.0593 | 6.6 | 700 | 36.8541 | 1.0 |
|
65 |
+
| 48.6631 | 7.55 | 800 | 36.4298 | 1.0 |
|
66 |
+
| 47.483 | 8.49 | 900 | 36.3610 | 1.0 |
|
67 |
+
| 46.5326 | 9.43 | 1000 | 34.7439 | 0.9656 |
|
68 |
+
| 39.0329 | 10.38 | 1100 | 19.4442 | 0.5706 |
|
69 |
+
| 22.0857 | 11.32 | 1200 | 8.4938 | 0.2356 |
|
70 |
+
| 14.0187 | 12.26 | 1300 | 5.6815 | 0.1756 |
|
71 |
+
| 10.601 | 13.21 | 1400 | 4.4978 | 0.1478 |
|
72 |
+
| 9.0735 | 14.15 | 1500 | 3.8777 | 0.1386 |
|
73 |
+
| 7.449 | 15.09 | 1600 | 3.3361 | 0.1255 |
|
74 |
+
| 6.8473 | 16.04 | 1700 | 3.1257 | 0.1285 |
|
75 |
+
| 6.3913 | 16.98 | 1800 | 2.9602 | 0.1233 |
|
76 |
+
| 5.8235 | 17.92 | 1900 | 2.6843 | 0.1152 |
|
77 |
+
| 5.8092 | 18.87 | 2000 | 2.5891 | 0.1091 |
|
78 |
+
| 5.5489 | 19.81 | 2100 | 2.6685 | 0.1283 |
|
79 |
+
| 5.4259 | 20.75 | 2200 | 2.6268 | 0.1195 |
|
80 |
+
| 4.9683 | 21.7 | 2300 | 2.4970 | 0.1146 |
|
81 |
+
| 4.8524 | 22.64 | 2400 | 2.4337 | 0.1124 |
|
82 |
+
| 4.8404 | 23.58 | 2500 | 2.3632 | 0.1018 |
|
83 |
+
| 4.3451 | 24.53 | 2600 | 2.3354 | 0.0964 |
|
84 |
+
| 4.3297 | 25.47 | 2700 | 2.2977 | 0.1017 |
|
85 |
+
| 4.0442 | 26.42 | 2800 | 2.3116 | 0.1115 |
|
86 |
+
| 3.7571 | 27.36 | 2900 | 2.2637 | 0.1078 |
|
87 |
+
| 3.7335 | 28.3 | 3000 | 2.2070 | 0.1031 |
|
88 |
+
| 3.736 | 29.25 | 3100 | 2.2637 | 0.0992 |
|
89 |
+
| 3.7796 | 30.19 | 3200 | 2.2364 | 0.1012 |
|
90 |
+
| 3.7623 | 31.13 | 3300 | 2.1827 | 0.0983 |
|
91 |
+
| 3.2842 | 32.08 | 3400 | 2.1322 | 0.1073 |
|
92 |
+
| 3.4898 | 33.02 | 3500 | 2.0692 | 0.0999 |
|
93 |
+
| 3.453 | 33.96 | 3600 | 2.0662 | 0.0958 |
|
94 |
+
| 3.1855 | 34.91 | 3700 | 2.1000 | 0.0908 |
|
95 |
+
| 3.1468 | 35.85 | 3800 | 2.0887 | 0.0948 |
|
96 |
+
| 2.9984 | 36.79 | 3900 | 2.0589 | 0.0961 |
|
97 |
+
| 3.215 | 37.74 | 4000 | 2.0436 | 0.0958 |
|
98 |
+
| 3.2076 | 38.68 | 4100 | 2.0969 | 0.0978 |
|
99 |
+
| 2.8793 | 39.62 | 4200 | 2.0420 | 0.0939 |
|
100 |
+
| 2.9688 | 40.57 | 4300 | 2.0713 | 0.0900 |
|
101 |
+
| 2.9882 | 41.51 | 4400 | 2.0373 | 0.0940 |
|
102 |
+
| 3.12 | 42.45 | 4500 | 2.0513 | 0.1008 |
|
103 |
+
| 2.7528 | 43.4 | 4600 | 2.0500 | 0.0960 |
|
104 |
+
| 2.441 | 44.34 | 4700 | 2.0692 | 0.0943 |
|
105 |
+
| 2.6396 | 45.28 | 4800 | 2.0387 | 0.0904 |
|
106 |
+
| 2.5982 | 46.23 | 4900 | 2.0974 | 0.0975 |
|
107 |
+
| 2.574 | 47.17 | 5000 | 2.0484 | 0.0933 |
|
108 |
+
| 2.3482 | 48.11 | 5100 | 2.0370 | 0.0981 |
|
109 |
+
| 2.4587 | 49.06 | 5200 | 2.0412 | 0.1032 |
|
110 |
+
| 2.3123 | 50.0 | 5300 | 2.0249 | 0.1020 |
|
111 |
+
| 2.27 | 50.94 | 5400 | 2.0079 | 0.0909 |
|
112 |
+
| 2.3862 | 51.89 | 5500 | 2.0595 | 0.0910 |
|
113 |
+
| 2.4499 | 52.83 | 5600 | 2.0382 | 0.0948 |
|
114 |
+
| 2.4291 | 53.77 | 5700 | 2.0174 | 0.0926 |
|
115 |
+
| 2.1468 | 54.72 | 5800 | 2.0347 | 0.0939 |
|
116 |
+
| 2.1434 | 55.66 | 5900 | 2.0004 | 0.0963 |
|
117 |
+
| 2.1786 | 56.6 | 6000 | 1.9845 | 0.0878 |
|
118 |
+
| 2.22 | 57.55 | 6100 | 1.9827 | 0.0880 |
|
119 |
+
| 2.0233 | 58.49 | 6200 | 1.9880 | 0.0923 |
|
120 |
+
| 2.1476 | 59.43 | 6300 | 1.9856 | 0.0852 |
|
121 |
+
| 1.9682 | 60.38 | 6400 | 2.0001 | 0.0838 |
|
122 |
+
| 2.2104 | 61.32 | 6500 | 2.0052 | 0.0885 |
|
123 |
+
| 2.1225 | 62.26 | 6600 | 1.9984 | 0.0856 |
|
124 |
+
| 2.1791 | 63.21 | 6700 | 1.9606 | 0.0838 |
|
125 |
+
| 2.1231 | 64.15 | 6800 | 1.9905 | 0.0917 |
|
126 |
+
| 2.0084 | 65.09 | 6900 | 1.9866 | 0.0921 |
|
127 |
+
| 2.0541 | 66.04 | 7000 | 1.9948 | 0.0933 |
|
128 |
+
| 1.9073 | 66.98 | 7100 | 1.9885 | 0.0903 |
|
129 |
+
| 1.9308 | 67.92 | 7200 | 2.0064 | 0.0919 |
|
130 |
+
| 2.1946 | 68.87 | 7300 | 1.9828 | 0.0916 |
|
131 |
+
| 1.9435 | 69.81 | 7400 | 1.9889 | 0.0928 |
|
132 |
+
| 1.8279 | 70.75 | 7500 | 1.9959 | 0.0911 |
|
133 |
+
| 1.7645 | 71.7 | 7600 | 2.0134 | 0.0929 |
|
134 |
+
| 1.6908 | 72.64 | 7700 | 2.0119 | 0.0913 |
|
135 |
+
| 1.7531 | 73.58 | 7800 | 1.9963 | 0.0879 |
|
136 |
+
| 1.6314 | 74.53 | 7900 | 1.9854 | 0.0915 |
|
137 |
+
| 1.7651 | 75.47 | 8000 | 1.9984 | 0.0920 |
|
138 |
+
| 1.8407 | 76.42 | 8100 | 1.9793 | 0.0903 |
|
139 |
+
| 1.8132 | 77.36 | 8200 | 2.0208 | 0.0912 |
|
140 |
+
| 1.6622 | 78.3 | 8300 | 2.0106 | 0.0906 |
|
141 |
+
| 2.1048 | 79.25 | 8400 | 1.9989 | 0.0915 |
|
142 |
+
| 1.7944 | 80.19 | 8500 | 1.9980 | 0.0913 |
|
143 |
+
| 1.8029 | 81.13 | 8600 | 1.9870 | 0.0897 |
|
144 |
+
| 1.8474 | 82.08 | 8700 | 1.9901 | 0.0890 |
|
145 |
+
| 1.5574 | 83.02 | 8800 | 1.9952 | 0.0905 |
|
146 |
+
| 1.5757 | 83.96 | 8900 | 1.9982 | 0.0907 |
|
147 |
+
| 1.6461 | 84.91 | 9000 | 1.9858 | 0.0900 |
|
148 |
+
| 1.7695 | 85.85 | 9100 | 1.9991 | 0.0905 |
|
149 |
+
| 1.6583 | 86.79 | 9200 | 2.0011 | 0.0902 |
|
150 |
+
| 1.7586 | 87.74 | 9300 | 1.9869 | 0.0911 |
|
151 |
+
| 1.7142 | 88.68 | 9400 | 1.9956 | 0.0888 |
|
152 |
+
| 1.7371 | 89.62 | 9500 | 1.9968 | 0.0888 |
|
153 |
+
| 1.6964 | 90.57 | 9600 | 1.9958 | 0.0892 |
|
154 |
+
| 1.7224 | 91.51 | 9700 | 1.9947 | 0.0891 |
|
155 |
+
| 1.8655 | 92.45 | 9800 | 1.9976 | 0.0908 |
|
156 |
+
| 1.6929 | 93.4 | 9900 | 1.9984 | 0.0909 |
|
157 |
+
| 1.6306 | 94.34 | 10000 | 2.0012 | 0.0911 |
|
158 |
+
| 1.7218 | 95.28 | 10100 | 2.0010 | 0.0913 |
|
159 |
+
| 1.7019 | 96.23 | 10200 | 1.9977 | 0.0908 |
|
160 |
+
| 1.902 | 97.17 | 10300 | 1.9989 | 0.0908 |
|
161 |
+
| 1.7555 | 98.11 | 10400 | 1.9964 | 0.0909 |
|
162 |
+
| 1.5272 | 99.06 | 10500 | 1.9957 | 0.0906 |
|
163 |
+
| 1.8033 | 100.0 | 10600 | 1.9952 | 0.0908 |
|
164 |
+
|
165 |
+
|
166 |
+
### Framework versions
|
167 |
+
|
168 |
+
- Transformers 4.35.2
|
169 |
+
- Pytorch 1.13.1+cu117
|
170 |
+
- Datasets 2.12.0
|
171 |
+
- Tokenizers 0.14.1
|
model.safetensors
CHANGED
@@ -1,3 +1,3 @@
|
|
1 |
version https://git-lfs.github.com/spec/v1
|
2 |
-
oid sha256:
|
3 |
size 197617854
|
|
|
1 |
version https://git-lfs.github.com/spec/v1
|
2 |
+
oid sha256:e7e3033f67fefe1ac4f5ad27ea990ff703624d99c7624cc10676d9d3b4612338
|
3 |
size 197617854
|