|
--- |
|
license: cc-by-nc-4.0 |
|
base_model: nguyenvulebinh/wav2vec2-base-vietnamese-250h |
|
tags: |
|
- generated_from_trainer |
|
metrics: |
|
- wer |
|
model-index: |
|
- name: w2v2_ablation_focal_ctc_a0.5_g1.0-best_on-ling_head-tp0.025_tl10_fp0.001_fl16 |
|
results: [] |
|
--- |
|
|
|
<!-- This model card has been generated automatically according to the information the Trainer had access to. You |
|
should probably proofread and complete it, then remove this comment. --> |
|
|
|
# w2v2_ablation_focal_ctc_a0.5_g1.0-best_on-ling_head-tp0.025_tl10_fp0.001_fl16 |
|
|
|
This model is a fine-tuned version of [nguyenvulebinh/wav2vec2-base-vietnamese-250h](https://huggingface.co/nguyenvulebinh/wav2vec2-base-vietnamese-250h) on an unknown dataset. |
|
It achieves the following results on the evaluation set: |
|
- Loss: 1.9952 |
|
- Wer: 0.0908 |
|
|
|
## Model description |
|
|
|
More information needed |
|
|
|
## Intended uses & limitations |
|
|
|
More information needed |
|
|
|
## Training and evaluation data |
|
|
|
More information needed |
|
|
|
## Training procedure |
|
|
|
### Training hyperparameters |
|
|
|
The following hyperparameters were used during training: |
|
- learning_rate: 2e-05 |
|
- train_batch_size: 8 |
|
- eval_batch_size: 16 |
|
- seed: 42 |
|
- distributed_type: multi-GPU |
|
- num_devices: 4 |
|
- total_train_batch_size: 32 |
|
- total_eval_batch_size: 64 |
|
- optimizer: Adam with betas=(0.9,0.98) and epsilon=1e-08 |
|
- lr_scheduler_type: cosine |
|
- lr_scheduler_warmup_ratio: 0.1 |
|
- num_epochs: 100 |
|
- mixed_precision_training: Native AMP |
|
|
|
### Training results |
|
|
|
| Training Loss | Epoch | Step | Validation Loss | Wer | |
|
|:-------------:|:-----:|:-----:|:---------------:|:-------:| |
|
| 891.4405 | 0.94 | 100 | 581.3978 | 18.6410 | |
|
| 615.8164 | 1.89 | 200 | 221.5820 | 17.0065 | |
|
| 105.0527 | 2.83 | 300 | 43.9285 | 1.0 | |
|
| 56.2539 | 3.77 | 400 | 40.2262 | 1.0 | |
|
| 51.7117 | 4.72 | 500 | 38.2334 | 1.0 | |
|
| 49.7296 | 5.66 | 600 | 37.4374 | 1.0 | |
|
| 49.0593 | 6.6 | 700 | 36.8541 | 1.0 | |
|
| 48.6631 | 7.55 | 800 | 36.4298 | 1.0 | |
|
| 47.483 | 8.49 | 900 | 36.3610 | 1.0 | |
|
| 46.5326 | 9.43 | 1000 | 34.7439 | 0.9656 | |
|
| 39.0329 | 10.38 | 1100 | 19.4442 | 0.5706 | |
|
| 22.0857 | 11.32 | 1200 | 8.4938 | 0.2356 | |
|
| 14.0187 | 12.26 | 1300 | 5.6815 | 0.1756 | |
|
| 10.601 | 13.21 | 1400 | 4.4978 | 0.1478 | |
|
| 9.0735 | 14.15 | 1500 | 3.8777 | 0.1386 | |
|
| 7.449 | 15.09 | 1600 | 3.3361 | 0.1255 | |
|
| 6.8473 | 16.04 | 1700 | 3.1257 | 0.1285 | |
|
| 6.3913 | 16.98 | 1800 | 2.9602 | 0.1233 | |
|
| 5.8235 | 17.92 | 1900 | 2.6843 | 0.1152 | |
|
| 5.8092 | 18.87 | 2000 | 2.5891 | 0.1091 | |
|
| 5.5489 | 19.81 | 2100 | 2.6685 | 0.1283 | |
|
| 5.4259 | 20.75 | 2200 | 2.6268 | 0.1195 | |
|
| 4.9683 | 21.7 | 2300 | 2.4970 | 0.1146 | |
|
| 4.8524 | 22.64 | 2400 | 2.4337 | 0.1124 | |
|
| 4.8404 | 23.58 | 2500 | 2.3632 | 0.1018 | |
|
| 4.3451 | 24.53 | 2600 | 2.3354 | 0.0964 | |
|
| 4.3297 | 25.47 | 2700 | 2.2977 | 0.1017 | |
|
| 4.0442 | 26.42 | 2800 | 2.3116 | 0.1115 | |
|
| 3.7571 | 27.36 | 2900 | 2.2637 | 0.1078 | |
|
| 3.7335 | 28.3 | 3000 | 2.2070 | 0.1031 | |
|
| 3.736 | 29.25 | 3100 | 2.2637 | 0.0992 | |
|
| 3.7796 | 30.19 | 3200 | 2.2364 | 0.1012 | |
|
| 3.7623 | 31.13 | 3300 | 2.1827 | 0.0983 | |
|
| 3.2842 | 32.08 | 3400 | 2.1322 | 0.1073 | |
|
| 3.4898 | 33.02 | 3500 | 2.0692 | 0.0999 | |
|
| 3.453 | 33.96 | 3600 | 2.0662 | 0.0958 | |
|
| 3.1855 | 34.91 | 3700 | 2.1000 | 0.0908 | |
|
| 3.1468 | 35.85 | 3800 | 2.0887 | 0.0948 | |
|
| 2.9984 | 36.79 | 3900 | 2.0589 | 0.0961 | |
|
| 3.215 | 37.74 | 4000 | 2.0436 | 0.0958 | |
|
| 3.2076 | 38.68 | 4100 | 2.0969 | 0.0978 | |
|
| 2.8793 | 39.62 | 4200 | 2.0420 | 0.0939 | |
|
| 2.9688 | 40.57 | 4300 | 2.0713 | 0.0900 | |
|
| 2.9882 | 41.51 | 4400 | 2.0373 | 0.0940 | |
|
| 3.12 | 42.45 | 4500 | 2.0513 | 0.1008 | |
|
| 2.7528 | 43.4 | 4600 | 2.0500 | 0.0960 | |
|
| 2.441 | 44.34 | 4700 | 2.0692 | 0.0943 | |
|
| 2.6396 | 45.28 | 4800 | 2.0387 | 0.0904 | |
|
| 2.5982 | 46.23 | 4900 | 2.0974 | 0.0975 | |
|
| 2.574 | 47.17 | 5000 | 2.0484 | 0.0933 | |
|
| 2.3482 | 48.11 | 5100 | 2.0370 | 0.0981 | |
|
| 2.4587 | 49.06 | 5200 | 2.0412 | 0.1032 | |
|
| 2.3123 | 50.0 | 5300 | 2.0249 | 0.1020 | |
|
| 2.27 | 50.94 | 5400 | 2.0079 | 0.0909 | |
|
| 2.3862 | 51.89 | 5500 | 2.0595 | 0.0910 | |
|
| 2.4499 | 52.83 | 5600 | 2.0382 | 0.0948 | |
|
| 2.4291 | 53.77 | 5700 | 2.0174 | 0.0926 | |
|
| 2.1468 | 54.72 | 5800 | 2.0347 | 0.0939 | |
|
| 2.1434 | 55.66 | 5900 | 2.0004 | 0.0963 | |
|
| 2.1786 | 56.6 | 6000 | 1.9845 | 0.0878 | |
|
| 2.22 | 57.55 | 6100 | 1.9827 | 0.0880 | |
|
| 2.0233 | 58.49 | 6200 | 1.9880 | 0.0923 | |
|
| 2.1476 | 59.43 | 6300 | 1.9856 | 0.0852 | |
|
| 1.9682 | 60.38 | 6400 | 2.0001 | 0.0838 | |
|
| 2.2104 | 61.32 | 6500 | 2.0052 | 0.0885 | |
|
| 2.1225 | 62.26 | 6600 | 1.9984 | 0.0856 | |
|
| 2.1791 | 63.21 | 6700 | 1.9606 | 0.0838 | |
|
| 2.1231 | 64.15 | 6800 | 1.9905 | 0.0917 | |
|
| 2.0084 | 65.09 | 6900 | 1.9866 | 0.0921 | |
|
| 2.0541 | 66.04 | 7000 | 1.9948 | 0.0933 | |
|
| 1.9073 | 66.98 | 7100 | 1.9885 | 0.0903 | |
|
| 1.9308 | 67.92 | 7200 | 2.0064 | 0.0919 | |
|
| 2.1946 | 68.87 | 7300 | 1.9828 | 0.0916 | |
|
| 1.9435 | 69.81 | 7400 | 1.9889 | 0.0928 | |
|
| 1.8279 | 70.75 | 7500 | 1.9959 | 0.0911 | |
|
| 1.7645 | 71.7 | 7600 | 2.0134 | 0.0929 | |
|
| 1.6908 | 72.64 | 7700 | 2.0119 | 0.0913 | |
|
| 1.7531 | 73.58 | 7800 | 1.9963 | 0.0879 | |
|
| 1.6314 | 74.53 | 7900 | 1.9854 | 0.0915 | |
|
| 1.7651 | 75.47 | 8000 | 1.9984 | 0.0920 | |
|
| 1.8407 | 76.42 | 8100 | 1.9793 | 0.0903 | |
|
| 1.8132 | 77.36 | 8200 | 2.0208 | 0.0912 | |
|
| 1.6622 | 78.3 | 8300 | 2.0106 | 0.0906 | |
|
| 2.1048 | 79.25 | 8400 | 1.9989 | 0.0915 | |
|
| 1.7944 | 80.19 | 8500 | 1.9980 | 0.0913 | |
|
| 1.8029 | 81.13 | 8600 | 1.9870 | 0.0897 | |
|
| 1.8474 | 82.08 | 8700 | 1.9901 | 0.0890 | |
|
| 1.5574 | 83.02 | 8800 | 1.9952 | 0.0905 | |
|
| 1.5757 | 83.96 | 8900 | 1.9982 | 0.0907 | |
|
| 1.6461 | 84.91 | 9000 | 1.9858 | 0.0900 | |
|
| 1.7695 | 85.85 | 9100 | 1.9991 | 0.0905 | |
|
| 1.6583 | 86.79 | 9200 | 2.0011 | 0.0902 | |
|
| 1.7586 | 87.74 | 9300 | 1.9869 | 0.0911 | |
|
| 1.7142 | 88.68 | 9400 | 1.9956 | 0.0888 | |
|
| 1.7371 | 89.62 | 9500 | 1.9968 | 0.0888 | |
|
| 1.6964 | 90.57 | 9600 | 1.9958 | 0.0892 | |
|
| 1.7224 | 91.51 | 9700 | 1.9947 | 0.0891 | |
|
| 1.8655 | 92.45 | 9800 | 1.9976 | 0.0908 | |
|
| 1.6929 | 93.4 | 9900 | 1.9984 | 0.0909 | |
|
| 1.6306 | 94.34 | 10000 | 2.0012 | 0.0911 | |
|
| 1.7218 | 95.28 | 10100 | 2.0010 | 0.0913 | |
|
| 1.7019 | 96.23 | 10200 | 1.9977 | 0.0908 | |
|
| 1.902 | 97.17 | 10300 | 1.9989 | 0.0908 | |
|
| 1.7555 | 98.11 | 10400 | 1.9964 | 0.0909 | |
|
| 1.5272 | 99.06 | 10500 | 1.9957 | 0.0906 | |
|
| 1.8033 | 100.0 | 10600 | 1.9952 | 0.0908 | |
|
|
|
|
|
### Framework versions |
|
|
|
- Transformers 4.35.2 |
|
- Pytorch 1.13.1+cu117 |
|
- Datasets 2.12.0 |
|
- Tokenizers 0.14.1 |
|
|
