fine-w2v2base-bs16-ep100-lr2e-05-linguistic-rmsnorm-focal_ctc_a0.25_g2.0-0.05_10_0.004_40

This model is a fine-tuned version of nguyenvulebinh/wav2vec2-base-vietnamese-250h on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 1.0133
• Wer: 0.0959

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 16
• eval_batch_size: 8
• seed: 42
• distributed_type: multi-GPU
• num_devices: 4
• total_train_batch_size: 64
• total_eval_batch_size: 32
• optimizer: Adam with betas=(0.9,0.98) and epsilon=1e-08
• lr_scheduler_type: cosine
• lr_scheduler_warmup_ratio: 0.1
• num_epochs: 100

Training results

Training Loss	Epoch	Step	Validation Loss	Wer
516.9129	0.94	50	252.8849	14.0053
330.3668	1.89	100	87.2475	0.9919
61.4207	2.83	150	23.7193	1.0
28.4469	3.77	200	21.1814	1.0
27.2279	4.72	250	20.5718	1.0
26.2915	5.66	300	19.8798	1.0
25.4979	6.6	350	19.4406	1.0
24.5512	7.55	400	19.0059	1.0
24.1332	8.49	450	18.7581	1.0
24.4038	9.43	500	18.6778	1.0
24.0403	10.38	550	18.3450	1.0
22.7387	11.32	600	13.8828	0.7278
14.1877	12.26	650	6.3008	0.3182
8.1601	13.21	700	3.9807	0.2226
5.8662	14.15	750	2.9679	0.1789
4.6735	15.09	800	2.4585	0.1650
4.0491	16.04	850	2.1273	0.1467
3.6131	16.98	900	2.0185	0.1475
3.2409	17.92	950	1.7619	0.1315
3.0454	18.87	1000	1.6940	0.1296
2.8232	19.81	1050	1.5872	0.1246
2.6343	20.75	1100	1.5053	0.1180
2.5752	21.7	1150	1.4446	0.1149
2.4305	22.64	1200	1.4419	0.1215
2.2781	23.58	1250	1.4462	0.1245
2.2068	24.53	1300	1.3938	0.1117
2.2696	25.47	1350	1.3562	0.1115
2.0947	26.42	1400	1.3029	0.1115
2.0472	27.36	1450	1.2575	0.1109
1.9742	28.3	1500	1.2287	0.1076
1.9159	29.25	1550	1.2284	0.1095
1.8408	30.19	1600	1.2786	0.1130
1.819	31.13	1650	1.2388	0.1124
1.8066	32.08	1700	1.1771	0.0996
1.6811	33.02	1750	1.1634	0.1076
1.6524	33.96	1800	1.1327	0.1007
1.5504	34.91	1850	1.1447	0.1074
1.5791	35.85	1900	1.1347	0.1037
1.5679	36.79	1950	1.1095	0.0999
1.5048	37.74	2000	1.1328	0.1071
1.5465	38.68	2050	1.1442	0.1033
1.4368	39.62	2100	1.0938	0.1009
1.4346	40.57	2150	1.0875	0.1014
1.3809	41.51	2200	1.1307	0.1069
1.343	42.45	2250	1.0898	0.1019
1.2771	43.4	2300	1.0991	0.1039
1.263	44.34	2350	1.0925	0.0957
1.2803	45.28	2400	1.0552	0.0954
1.2236	46.23	2450	1.0765	0.1059
1.2075	47.17	2500	1.0713	0.1054
1.1767	48.11	2550	1.0560	0.1011
1.1757	49.06	2600	1.0584	0.1007
1.1324	50.0	2650	1.0491	0.1008
1.0932	50.94	2700	1.0302	0.0953
1.1574	51.89	2750	1.0367	0.0938
1.0113	52.83	2800	1.0461	0.0974
1.108	53.77	2850	1.0407	0.0955
1.081	54.72	2900	1.0483	0.0998
0.9996	55.66	2950	1.0381	0.0946
0.9785	56.6	3000	1.0296	0.0947
1.0465	57.55	3050	1.0366	0.0993
1.0241	58.49	3100	1.0341	0.1011
1.0015	59.43	3150	1.0302	0.0934
1.0161	60.38	3200	1.0456	0.1036
0.9228	61.32	3250	1.0287	0.0981
0.9959	62.26	3300	1.0318	0.0976
0.905	63.21	3350	1.0311	0.1031
0.9429	64.15	3400	1.0332	0.1004
0.9041	65.09	3450	1.0279	0.0965
0.907	66.04	3500	1.0192	0.0974
0.9223	66.98	3550	1.0288	0.0970
0.9433	67.92	3600	1.0205	0.0978
0.9044	68.87	3650	1.0229	0.0953
0.8956	69.81	3700	1.0178	0.0953
0.8719	70.75	3750	1.0178	0.0955
0.9081	71.7	3800	1.0198	0.0943
0.8458	72.64	3850	1.0253	0.0937
0.8462	73.58	3900	1.0195	0.0912
0.7924	74.53	3950	1.0253	0.0905
0.8997	75.47	4000	1.0275	0.0920
0.8403	76.42	4050	1.0175	0.0933
0.8519	77.36	4100	1.0261	0.0985
0.8286	78.3	4150	1.0216	0.0976
0.7825	79.25	4200	1.0164	0.0942
0.8315	80.19	4250	1.0195	0.0943
0.8347	81.13	4300	1.0256	0.0968
0.8244	82.08	4350	1.0264	0.0948
0.8063	83.02	4400	1.0282	0.0931
0.7904	83.96	4450	1.0226	0.0924
0.852	84.91	4500	1.0218	0.0935
0.8013	85.85	4550	1.0202	0.0951
0.8174	86.79	4600	1.0153	0.0934
0.8166	87.74	4650	1.0168	0.0958
0.8036	88.68	4700	1.0182	0.0962
0.7998	89.62	4750	1.0169	0.0962
0.7383	90.57	4800	1.0158	0.0957
0.8606	91.51	4850	1.0148	0.0951
0.7438	92.45	4900	1.0123	0.0944
0.848	93.4	4950	1.0135	0.0956
0.7839	94.34	5000	1.0140	0.0957
0.7708	95.28	5050	1.0130	0.0958
0.8123	96.23	5100	1.0138	0.0957
0.7656	97.17	5150	1.0135	0.0958
0.8085	98.11	5200	1.0136	0.0960
0.7987	99.06	5250	1.0135	0.0958
0.7917	100.0	5300	1.0133	0.0959

Framework versions

• Transformers 4.34.0
• Pytorch 2.0.1
• Datasets 2.14.5
• Tokenizers 0.14.1