tun_wav2vec8

This model was trained from scratch on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 1.2858
• Wer: 0.5831
• Cer: 0.1539

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0001
• train_batch_size: 10
• eval_batch_size: 8
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• lr_scheduler_warmup_steps: 80
• num_epochs: 100

Training results

Training Loss	Epoch	Step	Validation Loss	Wer	Cer
2.2416	5.0	300	1.6570	0.9561	0.3341
0.6397	10.0	600	1.0742	0.8652	0.3005
0.4434	15.0	900	1.2856	0.7743	0.2411
0.3084	20.0	1200	1.0868	0.7335	0.2106
0.2708	25.0	1500	0.9412	0.7367	0.1960
0.2519	30.0	1800	0.8857	0.6959	0.1863
0.1833	35.0	2100	1.2220	0.6740	0.1856
0.1354	40.0	2400	1.1682	0.6520	0.1786
0.1363	45.0	2700	1.1745	0.6865	0.1794
0.129	50.0	3000	1.0153	0.6426	0.1736
0.1036	55.0	3300	1.1114	0.6332	0.1705
0.1011	60.0	3600	1.4662	0.6238	0.1794
0.0902	65.0	3900	1.3797	0.6426	0.1779
0.074	70.0	4200	1.4517	0.6207	0.1813
0.0648	75.0	4500	1.2976	0.6207	0.1694
0.0591	80.0	4800	1.3030	0.5987	0.1690
0.0622	85.0	5100	1.2847	0.5925	0.1636
0.0639	90.0	5400	1.3230	0.5925	0.1659
0.0816	95.0	5700	1.2766	0.5925	0.1582
0.0444	100.0	6000	1.2858	0.5831	0.1539

Framework versions

• Transformers 4.42.3
• Pytorch 2.1.2
• Datasets 2.20.0
• Tokenizers 0.19.1