w2v2_ablation_focal_ctc_a0.99_g1.0-best_on-ling_head-tp0.025_tl10_fp0.001_fl16

This model is a fine-tuned version of nguyenvulebinh/wav2vec2-base-vietnamese-250h on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 3.8596
• Wer: 0.0837

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 8
• eval_batch_size: 16
• seed: 42
• distributed_type: multi-GPU
• num_devices: 4
• total_train_batch_size: 32
• total_eval_batch_size: 64
• optimizer: Adam with betas=(0.9,0.98) and epsilon=1e-08
• lr_scheduler_type: cosine
• lr_scheduler_warmup_ratio: 0.1
• num_epochs: 100
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Wer
1766.7372	0.94	100	1156.0582	18.6404
1229.4073	1.89	200	450.8329	17.3512
214.3073	2.83	300	87.0284	1.0
111.4782	3.77	400	79.7095	1.0
102.4638	4.72	500	75.7158	1.0
98.4896	5.66	600	74.1315	1.0
97.1564	6.6	700	72.9691	1.0
96.3522	7.55	800	72.1302	1.0
94.0422	8.49	900	72.0005	1.0
92.1967	9.43	1000	68.7518	0.9656
76.9513	10.38	1100	37.9701	0.5587
43.2365	11.32	1200	16.6431	0.2343
27.6251	12.26	1300	11.1301	0.1692
20.8447	13.21	1400	8.9874	0.1524
17.8373	14.15	1500	7.4692	0.1361
14.7487	15.09	1600	6.5928	0.1239
13.6336	16.04	1700	6.2194	0.1299
12.555	16.98	1800	5.7715	0.1214
11.5061	17.92	1900	5.3908	0.1139
11.6699	18.87	2000	5.2032	0.1068
10.6667	19.81	2100	5.1093	0.1150
10.4679	20.75	2200	4.8240	0.1127
9.7079	21.7	2300	4.6822	0.1014
9.5015	22.64	2400	4.5904	0.1005
9.4079	23.58	2500	4.6035	0.0956
8.5396	24.53	2600	4.5186	0.0925
8.515	25.47	2700	4.3337	0.0952
7.8243	26.42	2800	4.4874	0.1039
7.3847	27.36	2900	4.5759	0.1023
7.21	28.3	3000	4.3791	0.0982
7.2427	29.25	3100	4.3136	0.0951
7.4534	30.19	3200	4.3483	0.0966
7.3518	31.13	3300	4.2078	0.0873
6.3859	32.08	3400	4.0946	0.0921
6.9129	33.02	3500	4.1588	0.0987
6.5691	33.96	3600	4.1710	0.0970
6.2206	34.91	3700	4.0954	0.0855
6.0789	35.85	3800	4.0725	0.0908
5.9323	36.79	3900	4.1219	0.0958
6.3516	37.74	4000	4.0783	0.0918
6.1443	38.68	4100	4.1006	0.0984
5.5555	39.62	4200	4.0554	0.0920
5.6912	40.57	4300	4.0286	0.0854
5.7894	41.51	4400	4.0064	0.0866
6.0628	42.45	4500	4.0710	0.0958
5.3048	43.4	4600	4.1454	0.0915
4.8898	44.34	4700	4.0502	0.0849
5.0275	45.28	4800	4.1216	0.0834
5.1021	46.23	4900	4.1039	0.0874
4.9446	47.17	5000	3.9512	0.0830
4.4623	48.11	5100	4.0285	0.0920
4.6488	49.06	5200	3.9784	0.0877
4.2519	50.0	5300	3.9272	0.0923
4.2767	50.94	5400	3.8387	0.0813
4.534	51.89	5500	3.8577	0.0850
4.7048	52.83	5600	3.8982	0.0893
4.6566	53.77	5700	3.8712	0.0863
4.0576	54.72	5800	3.8719	0.0873
3.9801	55.66	5900	3.8680	0.0909
4.0698	56.6	6000	3.8687	0.0858
4.2237	57.55	6100	3.8807	0.0895
3.8054	58.49	6200	3.9619	0.0895
4.1874	59.43	6300	3.9111	0.0779
3.814	60.38	6400	3.8386	0.0826
4.247	61.32	6500	3.8373	0.0898
4.0079	62.26	6600	3.8169	0.0800
4.0862	63.21	6700	3.8315	0.0815
4.0043	64.15	6800	3.8240	0.0846
3.9001	65.09	6900	3.8030	0.0830
3.9259	66.04	7000	3.7837	0.0831
3.6774	66.98	7100	3.7848	0.0831
3.6523	67.92	7200	3.8252	0.0827
4.134	68.87	7300	3.8342	0.0832
3.7915	69.81	7400	3.9004	0.0854
3.4758	70.75	7500	3.8368	0.0870
3.4349	71.7	7600	3.9218	0.0917
3.2445	72.64	7700	3.8714	0.0878
3.4006	73.58	7800	3.8748	0.0859
3.1283	74.53	7900	3.8612	0.0829
3.4212	75.47	8000	3.8820	0.0858
3.5104	76.42	8100	3.8582	0.0802
3.5082	77.36	8200	3.9104	0.0868
3.159	78.3	8300	3.8733	0.0860
4.0749	79.25	8400	3.8720	0.0863
3.4915	80.19	8500	3.8578	0.0847
3.4707	81.13	8600	3.8410	0.0850
3.5001	82.08	8700	3.8324	0.0826
3.0112	83.02	8800	3.8402	0.0848
3.0206	83.96	8900	3.8454	0.0832
3.1934	84.91	9000	3.8446	0.0865
3.4633	85.85	9100	3.8565	0.0868
3.1621	86.79	9200	3.8597	0.0838
3.396	87.74	9300	3.8587	0.0830
3.2816	88.68	9400	3.8582	0.0835
3.3555	89.62	9500	3.8680	0.0826
3.2959	90.57	9600	3.8578	0.0826
3.3447	91.51	9700	3.8584	0.0824
3.6274	92.45	9800	3.8596	0.0822
3.2739	93.4	9900	3.8569	0.0817
3.1363	94.34	10000	3.8513	0.0832
3.3418	95.28	10100	3.8542	0.0830
3.2307	96.23	10200	3.8588	0.0832
3.6204	97.17	10300	3.8566	0.0832
3.4027	98.11	10400	3.8590	0.0838
2.9325	99.06	10500	3.8594	0.0832
3.4195	100.0	10600	3.8596	0.0837

Framework versions

• Transformers 4.35.2
• Pytorch 1.13.1+cu117
• Datasets 2.12.0
• Tokenizers 0.14.1