wav2vec2-large-xlsr-53-tr-fine-tuning-00

This model is a fine-tuned version of facebook/wav2vec2-large-xlsr-53 on the common_voice dataset. It achieves the following results on the evaluation set:

• Loss: 0.3974
• Wer: 0.4784

Training Procedure

Training Hyper-parameters

The following hyperparameters were used during training:

• learning_rate: 0.0003
• train_batch_size: 16
• eval_batch_size: 8
• seed: 42
• gradient_accumulation_steps: 2
• total_train_batch_size: 32
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• lr_scheduler_warmup_steps: 500
• num_epochs: 30

Training Results

Training Loss	Epoch	Step	Validation Loss	Wer
5.0376	4.21	400	2.4295	1.0
0.8375	8.42	800	0.4717	0.6291
0.3246	12.63	1200	0.4066	0.5528
0.216	16.84	1600	0.4022	0.5149
0.1664	21.05	2000	0.3972	0.5013
0.1413	25.26	2400	0.3982	0.4894
0.1197	29.47	2800	0.3974	0.4784

Framework Versions

• Transformers 4.19.1
• Pytorch 1.11.0+cu102
• Datasets 2.2.1
• Tokenizers 0.12.1