w2v2_ablation_with_ling_head-drop0.05-not-load-best-wer-best_on_tp0.025_tl10_fp0.001_fl16

This model is a fine-tuned version of nguyenvulebinh/wav2vec2-base-vietnamese-250h on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 0.4048
• Wer: 0.0937

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 8
• eval_batch_size: 16
• seed: 42
• distributed_type: multi-GPU
• num_devices: 4
• total_train_batch_size: 32
• total_eval_batch_size: 64
• optimizer: Adam with betas=(0.9,0.98) and epsilon=1e-08
• lr_scheduler_type: cosine
• lr_scheduler_warmup_ratio: 0.1
• num_epochs: 100
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Wer
72.7625	1.89	200	10.9130	0.9986
5.0881	3.77	400	5.0233	1.0
4.4894	5.66	600	4.9570	1.0
4.3324	7.55	800	4.6909	1.0
3.9241	9.43	1000	3.4612	0.7320
1.4741	11.32	1200	1.0577	0.2072
0.8631	13.21	1400	0.6902	0.1496
0.6692	15.09	1600	0.5799	0.1261
0.5332	16.98	1800	0.5359	0.1109
0.4583	18.87	2000	0.4968	0.1098
0.3982	20.75	2200	0.4717	0.1119
0.4013	22.64	2400	0.4220	0.1064
0.3342	24.53	2600	0.4302	0.1077
0.3119	26.42	2800	0.4231	0.1043
0.2824	28.3	3000	0.4108	0.0984
0.2844	30.19	3200	0.4218	0.0930
0.2659	32.08	3400	0.4081	0.0915
0.2579	33.96	3600	0.4148	0.0924
0.2565	35.85	3800	0.4238	0.0950
0.2294	37.74	4000	0.3990	0.0897
0.2401	39.62	4200	0.4061	0.0946
0.2184	41.51	4400	0.4063	0.0928
0.2191	43.4	4600	0.3919	0.0894
0.2209	45.28	4800	0.4083	0.0959
0.1887	47.17	5000	0.4168	0.0952
0.1953	49.06	5200	0.4034	0.0980
0.1759	50.94	5400	0.3932	0.0903
0.1786	52.83	5600	0.4063	0.0918
0.1745	54.72	5800	0.4008	0.1070
0.1681	56.6	6000	0.4057	0.0935
0.1574	58.49	6200	0.4050	0.0998
0.1641	60.38	6400	0.4031	0.0878
0.1531	62.26	6600	0.4027	0.0892
0.1526	64.15	6800	0.4000	0.0952
0.1508	66.04	7000	0.3987	0.0981
0.145	67.92	7200	0.4027	0.0994
0.1521	69.81	7400	0.4039	0.0998
0.152	71.7	7600	0.4067	0.0972
0.1475	73.58	7800	0.4067	0.0948
0.1345	75.47	8000	0.4063	0.0926
0.1329	77.36	8200	0.4046	0.0880
0.1429	79.25	8400	0.4044	0.0958
0.1502	81.13	8600	0.4035	0.0926
0.1388	83.02	8800	0.4045	0.0920
0.1272	84.91	9000	0.4057	0.0933
0.1429	86.79	9200	0.4046	0.0933
0.1339	88.68	9400	0.4056	0.0921
0.1316	90.57	9600	0.4061	0.0927
0.1397	92.45	9800	0.4060	0.0932
0.1318	94.34	10000	0.4046	0.0938
0.1182	96.23	10200	0.4050	0.0941
0.1373	98.11	10400	0.4045	0.0933
0.1287	100.0	10600	0.4048	0.0937

Framework versions

• Transformers 4.35.2
• Pytorch 1.13.1+cu117
• Datasets 2.12.0
• Tokenizers 0.14.1