xlsr-norm-nmcpc

This model is a fine-tuned version of facebook/wav2vec2-large-xlsr-53 on the None dataset. It achieves the following results on the evaluation set:

• Loss: 0.0024
• Wer: 0.2617

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0004
• train_batch_size: 8
• eval_batch_size: 8
• seed: 42
• gradient_accumulation_steps: 2
• total_train_batch_size: 16
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• lr_scheduler_warmup_steps: 132
• num_epochs: 100
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Wer
4.7523	7.2727	200	3.0692	0.9979
2.8525	14.5455	400	2.2999	1.0
1.4069	21.8182	600	0.3338	0.4723
0.3664	29.0909	800	0.0912	0.3851
0.1875	36.3636	1000	0.0453	0.3255
0.1165	43.6364	1200	0.0302	0.3
0.0959	50.9091	1400	0.0145	0.2766
0.0627	58.1818	1600	0.0107	0.2723
0.0496	65.4545	1800	0.0108	0.2553
0.037	72.7273	2000	0.0034	0.2596
0.0309	80.0	2200	0.0029	0.2617
0.0275	87.2727	2400	0.0026	0.2574
0.0218	94.5455	2600	0.0024	0.2617

Framework versions

• Transformers 4.45.0.dev0
• Pytorch 2.4.0
• Datasets 2.21.0
• Tokenizers 0.19.1