w2v2_ablation_focal_ctc_a0.5_g1.0-best_on-ling_head-tp0.025_tl10_fp0.001_fl16

This model is a fine-tuned version of nguyenvulebinh/wav2vec2-base-vietnamese-250h on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 1.9952
• Wer: 0.0908

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 8
• eval_batch_size: 16
• seed: 42
• distributed_type: multi-GPU
• num_devices: 4
• total_train_batch_size: 32
• total_eval_batch_size: 64
• optimizer: Adam with betas=(0.9,0.98) and epsilon=1e-08
• lr_scheduler_type: cosine
• lr_scheduler_warmup_ratio: 0.1
• num_epochs: 100
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Wer
891.4405	0.94	100	581.3978	18.6410
615.8164	1.89	200	221.5820	17.0065
105.0527	2.83	300	43.9285	1.0
56.2539	3.77	400	40.2262	1.0
51.7117	4.72	500	38.2334	1.0
49.7296	5.66	600	37.4374	1.0
49.0593	6.6	700	36.8541	1.0
48.6631	7.55	800	36.4298	1.0
47.483	8.49	900	36.3610	1.0
46.5326	9.43	1000	34.7439	0.9656
39.0329	10.38	1100	19.4442	0.5706
22.0857	11.32	1200	8.4938	0.2356
14.0187	12.26	1300	5.6815	0.1756
10.601	13.21	1400	4.4978	0.1478
9.0735	14.15	1500	3.8777	0.1386
7.449	15.09	1600	3.3361	0.1255
6.8473	16.04	1700	3.1257	0.1285
6.3913	16.98	1800	2.9602	0.1233
5.8235	17.92	1900	2.6843	0.1152
5.8092	18.87	2000	2.5891	0.1091
5.5489	19.81	2100	2.6685	0.1283
5.4259	20.75	2200	2.6268	0.1195
4.9683	21.7	2300	2.4970	0.1146
4.8524	22.64	2400	2.4337	0.1124
4.8404	23.58	2500	2.3632	0.1018
4.3451	24.53	2600	2.3354	0.0964
4.3297	25.47	2700	2.2977	0.1017
4.0442	26.42	2800	2.3116	0.1115
3.7571	27.36	2900	2.2637	0.1078
3.7335	28.3	3000	2.2070	0.1031
3.736	29.25	3100	2.2637	0.0992
3.7796	30.19	3200	2.2364	0.1012
3.7623	31.13	3300	2.1827	0.0983
3.2842	32.08	3400	2.1322	0.1073
3.4898	33.02	3500	2.0692	0.0999
3.453	33.96	3600	2.0662	0.0958
3.1855	34.91	3700	2.1000	0.0908
3.1468	35.85	3800	2.0887	0.0948
2.9984	36.79	3900	2.0589	0.0961
3.215	37.74	4000	2.0436	0.0958
3.2076	38.68	4100	2.0969	0.0978
2.8793	39.62	4200	2.0420	0.0939
2.9688	40.57	4300	2.0713	0.0900
2.9882	41.51	4400	2.0373	0.0940
3.12	42.45	4500	2.0513	0.1008
2.7528	43.4	4600	2.0500	0.0960
2.441	44.34	4700	2.0692	0.0943
2.6396	45.28	4800	2.0387	0.0904
2.5982	46.23	4900	2.0974	0.0975
2.574	47.17	5000	2.0484	0.0933
2.3482	48.11	5100	2.0370	0.0981
2.4587	49.06	5200	2.0412	0.1032
2.3123	50.0	5300	2.0249	0.1020
2.27	50.94	5400	2.0079	0.0909
2.3862	51.89	5500	2.0595	0.0910
2.4499	52.83	5600	2.0382	0.0948
2.4291	53.77	5700	2.0174	0.0926
2.1468	54.72	5800	2.0347	0.0939
2.1434	55.66	5900	2.0004	0.0963
2.1786	56.6	6000	1.9845	0.0878
2.22	57.55	6100	1.9827	0.0880
2.0233	58.49	6200	1.9880	0.0923
2.1476	59.43	6300	1.9856	0.0852
1.9682	60.38	6400	2.0001	0.0838
2.2104	61.32	6500	2.0052	0.0885
2.1225	62.26	6600	1.9984	0.0856
2.1791	63.21	6700	1.9606	0.0838
2.1231	64.15	6800	1.9905	0.0917
2.0084	65.09	6900	1.9866	0.0921
2.0541	66.04	7000	1.9948	0.0933
1.9073	66.98	7100	1.9885	0.0903
1.9308	67.92	7200	2.0064	0.0919
2.1946	68.87	7300	1.9828	0.0916
1.9435	69.81	7400	1.9889	0.0928
1.8279	70.75	7500	1.9959	0.0911
1.7645	71.7	7600	2.0134	0.0929
1.6908	72.64	7700	2.0119	0.0913
1.7531	73.58	7800	1.9963	0.0879
1.6314	74.53	7900	1.9854	0.0915
1.7651	75.47	8000	1.9984	0.0920
1.8407	76.42	8100	1.9793	0.0903
1.8132	77.36	8200	2.0208	0.0912
1.6622	78.3	8300	2.0106	0.0906
2.1048	79.25	8400	1.9989	0.0915
1.7944	80.19	8500	1.9980	0.0913
1.8029	81.13	8600	1.9870	0.0897
1.8474	82.08	8700	1.9901	0.0890
1.5574	83.02	8800	1.9952	0.0905
1.5757	83.96	8900	1.9982	0.0907
1.6461	84.91	9000	1.9858	0.0900
1.7695	85.85	9100	1.9991	0.0905
1.6583	86.79	9200	2.0011	0.0902
1.7586	87.74	9300	1.9869	0.0911
1.7142	88.68	9400	1.9956	0.0888
1.7371	89.62	9500	1.9968	0.0888
1.6964	90.57	9600	1.9958	0.0892
1.7224	91.51	9700	1.9947	0.0891
1.8655	92.45	9800	1.9976	0.0908
1.6929	93.4	9900	1.9984	0.0909
1.6306	94.34	10000	2.0012	0.0911
1.7218	95.28	10100	2.0010	0.0913
1.7019	96.23	10200	1.9977	0.0908
1.902	97.17	10300	1.9989	0.0908
1.7555	98.11	10400	1.9964	0.0909
1.5272	99.06	10500	1.9957	0.0906
1.8033	100.0	10600	1.9952	0.0908

Framework versions

• Transformers 4.35.2
• Pytorch 1.13.1+cu117
• Datasets 2.12.0
• Tokenizers 0.14.1