w2v2_ablation_focal_ctc_a0.5_g2.0-best_on-ling_head-tp0.025_tl10_fp0.001_fl16

This model is a fine-tuned version of nguyenvulebinh/wav2vec2-base-vietnamese-250h on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 1.9216
• Wer: 0.0907

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 8
• eval_batch_size: 16
• seed: 42
• distributed_type: multi-GPU
• num_devices: 4
• total_train_batch_size: 32
• total_eval_batch_size: 64
• optimizer: Adam with betas=(0.9,0.98) and epsilon=1e-08
• lr_scheduler_type: cosine
• lr_scheduler_warmup_ratio: 0.1
• num_epochs: 100
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Wer
891.441	0.94	100	581.4059	18.6409
615.9091	1.89	200	221.7206	17.0156
105.1541	2.83	300	43.9230	1.0
56.2587	3.77	400	40.2258	1.0
51.7195	4.72	500	38.2289	1.0
49.7313	5.66	600	37.4335	1.0
49.0642	6.6	700	36.8477	1.0
48.6566	7.55	800	36.4193	1.0
47.4859	8.49	900	36.3542	1.0
46.5463	9.43	1000	34.6432	0.9649
38.6934	10.38	1100	19.0466	0.5581
21.715	11.32	1200	8.4372	0.2382
13.9775	12.26	1300	5.7309	0.1828
10.6865	13.21	1400	4.6203	0.1534
9.1609	14.15	1500	3.8524	0.1423
7.4574	15.09	1600	3.3582	0.1280
6.8365	16.04	1700	3.1737	0.1274
6.3641	16.98	1800	2.9234	0.1260
5.7227	17.92	1900	2.6197	0.1136
5.7697	18.87	2000	2.5785	0.1054
5.1611	19.81	2100	2.5719	0.1167
5.1462	20.75	2200	2.5031	0.1114
4.8516	21.7	2300	2.3934	0.1029
4.6597	22.64	2400	2.3006	0.0993
4.6672	23.58	2500	2.3057	0.1013
4.2021	24.53	2600	2.2338	0.0902
4.2235	25.47	2700	2.2535	0.0959
3.8921	26.42	2800	2.1978	0.0996
3.6617	27.36	2900	2.1876	0.0980
3.6312	28.3	3000	2.1848	0.1006
3.6697	29.25	3100	2.1414	0.0934
3.6627	30.19	3200	2.1478	0.0965
3.7219	31.13	3300	2.1029	0.0922
3.1056	32.08	3400	2.1064	0.0998
3.3701	33.02	3500	2.0845	0.1005
3.2571	33.96	3600	2.0171	0.0936
3.0645	34.91	3700	2.0374	0.0891
3.0509	35.85	3800	2.0592	0.0927
2.9386	36.79	3900	2.0023	0.0953
3.133	37.74	4000	1.9790	0.0950
3.0533	38.68	4100	2.0205	0.1021
2.7386	39.62	4200	1.9805	0.0933
2.8599	40.57	4300	1.9517	0.0881
2.9106	41.51	4400	1.9846	0.0888
3.0177	42.45	4500	1.9896	0.1016
2.6221	43.4	4600	1.9678	0.0928
2.3645	44.34	4700	1.9904	0.0928
2.4846	45.28	4800	1.9529	0.0877
2.5123	46.23	4900	1.9605	0.0948
2.4518	47.17	5000	1.9608	0.0898
2.2337	48.11	5100	1.9674	0.0965
2.3486	49.06	5200	1.9782	0.0970
2.1713	50.0	5300	1.9257	0.0938
2.1746	50.94	5400	1.8913	0.0893
2.2803	51.89	5500	1.9196	0.0978
2.4001	52.83	5600	1.9311	0.1044
2.3443	53.77	5700	1.9317	0.0998
2.0512	54.72	5800	1.9758	0.0979
2.0084	55.66	5900	1.9526	0.0997
2.0652	56.6	6000	1.9397	0.0932
2.0525	57.55	6100	1.9185	0.0863
1.8526	58.49	6200	1.9570	0.0968
2.0296	59.43	6300	1.9325	0.0893
1.873	60.38	6400	1.9295	0.0905
2.0946	61.32	6500	1.9463	0.0949
1.9841	62.26	6600	1.9384	0.0882
2.0244	63.21	6700	1.9380	0.0916
1.9862	64.15	6800	1.9358	0.0912
1.9465	65.09	6900	1.9318	0.0912
1.9783	66.04	7000	1.9456	0.0941
1.8512	66.98	7100	1.9342	0.0909
1.8227	67.92	7200	1.9391	0.0938
2.0592	68.87	7300	1.9228	0.0914
1.9143	69.81	7400	1.9259	0.0928
1.7339	70.75	7500	1.9337	0.0919
1.7157	71.7	7600	1.9342	0.0934
1.6371	72.64	7700	1.9220	0.0914
1.6752	73.58	7800	1.9063	0.0889
1.5704	74.53	7900	1.9124	0.0878
1.7046	75.47	8000	1.9245	0.0946
1.7429	76.42	8100	1.9256	0.0898
1.7411	77.36	8200	1.9373	0.0933
1.5708	78.3	8300	1.9260	0.0904
2.031	79.25	8400	1.9297	0.0940
1.753	80.19	8500	1.9167	0.0927
1.7548	81.13	8600	1.9128	0.0899
1.7614	82.08	8700	1.9115	0.0898
1.492	83.02	8800	1.9240	0.0920
1.5172	83.96	8900	1.9183	0.0914
1.6205	84.91	9000	1.9248	0.0935
1.7142	85.85	9100	1.9338	0.0916
1.5832	86.79	9200	1.9305	0.0916
1.6942	87.74	9300	1.9162	0.0924
1.6287	88.68	9400	1.9219	0.0903
1.6635	89.62	9500	1.9185	0.0916
1.6361	90.57	9600	1.9162	0.0897
1.6741	91.51	9700	1.9210	0.0915
1.7979	92.45	9800	1.9252	0.0910
1.6517	93.4	9900	1.9182	0.0903
1.5479	94.34	10000	1.9202	0.0907
1.6716	95.28	10100	1.9225	0.0907
1.6293	96.23	10200	1.9247	0.0912
1.8255	97.17	10300	1.9229	0.0906
1.6997	98.11	10400	1.9233	0.0908
1.4612	99.06	10500	1.9214	0.0908
1.7217	100.0	10600	1.9216	0.0907

Framework versions

• Transformers 4.35.2
• Pytorch 1.13.1+cu117
• Datasets 2.12.0
• Tokenizers 0.14.1