--- license: cc-by-nc-4.0 base_model: nguyenvulebinh/wav2vec2-base-vietnamese-250h tags: - generated_from_trainer metrics: - wer model-index: - name: w2v2_ablation_focal_ctc_a0.5_g1.0-best_on-ling_head-tp0.025_tl10_fp0.001_fl16 results: [] --- # w2v2_ablation_focal_ctc_a0.5_g1.0-best_on-ling_head-tp0.025_tl10_fp0.001_fl16 This model is a fine-tuned version of [nguyenvulebinh/wav2vec2-base-vietnamese-250h](https://huggingface.co/nguyenvulebinh/wav2vec2-base-vietnamese-250h) on an unknown dataset. It achieves the following results on the evaluation set: - Loss: 1.9952 - Wer: 0.0908 ## Model description More information needed ## Intended uses & limitations More information needed ## Training and evaluation data More information needed ## Training procedure ### Training hyperparameters The following hyperparameters were used during training: - learning_rate: 2e-05 - train_batch_size: 8 - eval_batch_size: 16 - seed: 42 - distributed_type: multi-GPU - num_devices: 4 - total_train_batch_size: 32 - total_eval_batch_size: 64 - optimizer: Adam with betas=(0.9,0.98) and epsilon=1e-08 - lr_scheduler_type: cosine - lr_scheduler_warmup_ratio: 0.1 - num_epochs: 100 - mixed_precision_training: Native AMP ### Training results | Training Loss | Epoch | Step | Validation Loss | Wer | |:-------------:|:-----:|:-----:|:---------------:|:-------:| | 891.4405 | 0.94 | 100 | 581.3978 | 18.6410 | | 615.8164 | 1.89 | 200 | 221.5820 | 17.0065 | | 105.0527 | 2.83 | 300 | 43.9285 | 1.0 | | 56.2539 | 3.77 | 400 | 40.2262 | 1.0 | | 51.7117 | 4.72 | 500 | 38.2334 | 1.0 | | 49.7296 | 5.66 | 600 | 37.4374 | 1.0 | | 49.0593 | 6.6 | 700 | 36.8541 | 1.0 | | 48.6631 | 7.55 | 800 | 36.4298 | 1.0 | | 47.483 | 8.49 | 900 | 36.3610 | 1.0 | | 46.5326 | 9.43 | 1000 | 34.7439 | 0.9656 | | 39.0329 | 10.38 | 1100 | 19.4442 | 0.5706 | | 22.0857 | 11.32 | 1200 | 8.4938 | 0.2356 | | 14.0187 | 12.26 | 1300 | 5.6815 | 0.1756 | | 10.601 | 13.21 | 1400 | 4.4978 | 0.1478 | | 9.0735 | 14.15 | 1500 | 3.8777 | 0.1386 | | 7.449 | 15.09 | 1600 | 3.3361 | 0.1255 | | 6.8473 | 16.04 | 1700 | 3.1257 | 0.1285 | | 6.3913 | 16.98 | 1800 | 2.9602 | 0.1233 | | 5.8235 | 17.92 | 1900 | 2.6843 | 0.1152 | | 5.8092 | 18.87 | 2000 | 2.5891 | 0.1091 | | 5.5489 | 19.81 | 2100 | 2.6685 | 0.1283 | | 5.4259 | 20.75 | 2200 | 2.6268 | 0.1195 | | 4.9683 | 21.7 | 2300 | 2.4970 | 0.1146 | | 4.8524 | 22.64 | 2400 | 2.4337 | 0.1124 | | 4.8404 | 23.58 | 2500 | 2.3632 | 0.1018 | | 4.3451 | 24.53 | 2600 | 2.3354 | 0.0964 | | 4.3297 | 25.47 | 2700 | 2.2977 | 0.1017 | | 4.0442 | 26.42 | 2800 | 2.3116 | 0.1115 | | 3.7571 | 27.36 | 2900 | 2.2637 | 0.1078 | | 3.7335 | 28.3 | 3000 | 2.2070 | 0.1031 | | 3.736 | 29.25 | 3100 | 2.2637 | 0.0992 | | 3.7796 | 30.19 | 3200 | 2.2364 | 0.1012 | | 3.7623 | 31.13 | 3300 | 2.1827 | 0.0983 | | 3.2842 | 32.08 | 3400 | 2.1322 | 0.1073 | | 3.4898 | 33.02 | 3500 | 2.0692 | 0.0999 | | 3.453 | 33.96 | 3600 | 2.0662 | 0.0958 | | 3.1855 | 34.91 | 3700 | 2.1000 | 0.0908 | | 3.1468 | 35.85 | 3800 | 2.0887 | 0.0948 | | 2.9984 | 36.79 | 3900 | 2.0589 | 0.0961 | | 3.215 | 37.74 | 4000 | 2.0436 | 0.0958 | | 3.2076 | 38.68 | 4100 | 2.0969 | 0.0978 | | 2.8793 | 39.62 | 4200 | 2.0420 | 0.0939 | | 2.9688 | 40.57 | 4300 | 2.0713 | 0.0900 | | 2.9882 | 41.51 | 4400 | 2.0373 | 0.0940 | | 3.12 | 42.45 | 4500 | 2.0513 | 0.1008 | | 2.7528 | 43.4 | 4600 | 2.0500 | 0.0960 | | 2.441 | 44.34 | 4700 | 2.0692 | 0.0943 | | 2.6396 | 45.28 | 4800 | 2.0387 | 0.0904 | | 2.5982 | 46.23 | 4900 | 2.0974 | 0.0975 | | 2.574 | 47.17 | 5000 | 2.0484 | 0.0933 | | 2.3482 | 48.11 | 5100 | 2.0370 | 0.0981 | | 2.4587 | 49.06 | 5200 | 2.0412 | 0.1032 | | 2.3123 | 50.0 | 5300 | 2.0249 | 0.1020 | | 2.27 | 50.94 | 5400 | 2.0079 | 0.0909 | | 2.3862 | 51.89 | 5500 | 2.0595 | 0.0910 | | 2.4499 | 52.83 | 5600 | 2.0382 | 0.0948 | | 2.4291 | 53.77 | 5700 | 2.0174 | 0.0926 | | 2.1468 | 54.72 | 5800 | 2.0347 | 0.0939 | | 2.1434 | 55.66 | 5900 | 2.0004 | 0.0963 | | 2.1786 | 56.6 | 6000 | 1.9845 | 0.0878 | | 2.22 | 57.55 | 6100 | 1.9827 | 0.0880 | | 2.0233 | 58.49 | 6200 | 1.9880 | 0.0923 | | 2.1476 | 59.43 | 6300 | 1.9856 | 0.0852 | | 1.9682 | 60.38 | 6400 | 2.0001 | 0.0838 | | 2.2104 | 61.32 | 6500 | 2.0052 | 0.0885 | | 2.1225 | 62.26 | 6600 | 1.9984 | 0.0856 | | 2.1791 | 63.21 | 6700 | 1.9606 | 0.0838 | | 2.1231 | 64.15 | 6800 | 1.9905 | 0.0917 | | 2.0084 | 65.09 | 6900 | 1.9866 | 0.0921 | | 2.0541 | 66.04 | 7000 | 1.9948 | 0.0933 | | 1.9073 | 66.98 | 7100 | 1.9885 | 0.0903 | | 1.9308 | 67.92 | 7200 | 2.0064 | 0.0919 | | 2.1946 | 68.87 | 7300 | 1.9828 | 0.0916 | | 1.9435 | 69.81 | 7400 | 1.9889 | 0.0928 | | 1.8279 | 70.75 | 7500 | 1.9959 | 0.0911 | | 1.7645 | 71.7 | 7600 | 2.0134 | 0.0929 | | 1.6908 | 72.64 | 7700 | 2.0119 | 0.0913 | | 1.7531 | 73.58 | 7800 | 1.9963 | 0.0879 | | 1.6314 | 74.53 | 7900 | 1.9854 | 0.0915 | | 1.7651 | 75.47 | 8000 | 1.9984 | 0.0920 | | 1.8407 | 76.42 | 8100 | 1.9793 | 0.0903 | | 1.8132 | 77.36 | 8200 | 2.0208 | 0.0912 | | 1.6622 | 78.3 | 8300 | 2.0106 | 0.0906 | | 2.1048 | 79.25 | 8400 | 1.9989 | 0.0915 | | 1.7944 | 80.19 | 8500 | 1.9980 | 0.0913 | | 1.8029 | 81.13 | 8600 | 1.9870 | 0.0897 | | 1.8474 | 82.08 | 8700 | 1.9901 | 0.0890 | | 1.5574 | 83.02 | 8800 | 1.9952 | 0.0905 | | 1.5757 | 83.96 | 8900 | 1.9982 | 0.0907 | | 1.6461 | 84.91 | 9000 | 1.9858 | 0.0900 | | 1.7695 | 85.85 | 9100 | 1.9991 | 0.0905 | | 1.6583 | 86.79 | 9200 | 2.0011 | 0.0902 | | 1.7586 | 87.74 | 9300 | 1.9869 | 0.0911 | | 1.7142 | 88.68 | 9400 | 1.9956 | 0.0888 | | 1.7371 | 89.62 | 9500 | 1.9968 | 0.0888 | | 1.6964 | 90.57 | 9600 | 1.9958 | 0.0892 | | 1.7224 | 91.51 | 9700 | 1.9947 | 0.0891 | | 1.8655 | 92.45 | 9800 | 1.9976 | 0.0908 | | 1.6929 | 93.4 | 9900 | 1.9984 | 0.0909 | | 1.6306 | 94.34 | 10000 | 2.0012 | 0.0911 | | 1.7218 | 95.28 | 10100 | 2.0010 | 0.0913 | | 1.7019 | 96.23 | 10200 | 1.9977 | 0.0908 | | 1.902 | 97.17 | 10300 | 1.9989 | 0.0908 | | 1.7555 | 98.11 | 10400 | 1.9964 | 0.0909 | | 1.5272 | 99.06 | 10500 | 1.9957 | 0.0906 | | 1.8033 | 100.0 | 10600 | 1.9952 | 0.0908 | ### Framework versions - Transformers 4.35.2 - Pytorch 1.13.1+cu117 - Datasets 2.12.0 - Tokenizers 0.14.1