yesj1234/xlsr_cycle1_ko

ko-xlsr

This model is a fine-tuned version of facebook/wav2vec2-large-xlsr-53 on the ./SAMPLE_SPEECH.PY - NA dataset. It achieves the following results on the evaluation set:

• Loss: 0.5156
• Cer: 0.1228

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0003
• train_batch_size: 4
• eval_batch_size: 4
• seed: 42
• distributed_type: multi-GPU
• num_devices: 4
• gradient_accumulation_steps: 8
• total_train_batch_size: 128
• total_eval_batch_size: 16
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• lr_scheduler_warmup_steps: 500
• num_epochs: 40

Training results

Training Loss	Epoch	Step	Validation Loss	Cer
1.778	3.31	1000	1.2773	0.3050
1.1037	6.63	2000	0.7716	0.1888
0.9529	9.94	3000	0.6726	0.1659
0.8424	13.26	4000	0.6138	0.1512
0.767	16.57	5000	0.5885	0.1433
0.7201	19.88	6000	0.5682	0.1378
0.664	23.2	7000	0.5583	0.1333
0.6296	26.51	8000	0.5416	0.1298
0.6021	29.83	9000	0.5377	0.1272
0.568	33.14	10000	0.5241	0.1246
0.5519	36.45	11000	0.5184	0.1228
0.5395	39.77	12000	0.5156	0.1227

Framework versions

• Transformers 4.33.0
• Pytorch 2.0.1+cu117
• Datasets 2.14.4
• Tokenizers 0.13.3