ymoslem/whisper-medium-ga2en-v1.3.0-2k-r

Whisper Medium GA-EN Speech Translation Raw

This model is a fine-tuned version of openai/whisper-medium on the IWSLT-2023, FLEURS, BiteSize, and SpokenWords dataset. It achieves the following results on the evaluation set:

• Bleu: 28.37
• Chrf: 45.85
• Loss: 1.4194
• Wer: 68.1225

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0001
• train_batch_size: 16
• eval_batch_size: 16
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• lr_scheduler_warmup_ratio: 0.03
• training_steps: 2000
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Bleu	Chrf	Validation Loss	Wer
2.5874	0.0539	100	4.9	19.49	2.1785	114.0027
2.3237	0.1079	200	6.48	22.77	2.1129	151.8235
2.192	0.1618	300	7.92	25.9	2.0182	148.6718
1.9861	0.2157	400	10.55	28.55	1.8607	121.0266
1.8893	0.2697	500	16.68	33.64	1.8560	89.7794
1.8526	0.3236	600	8.83	30.12	1.7738	166.9968
1.6537	0.3776	700	10.94	33.83	1.6781	152.2287
1.7103	0.4315	800	16.9	36.4	1.6389	92.2557
1.4837	0.4854	900	13.81	34.5	1.6077	124.2233
1.2784	0.5394	1000	14.79	37.53	1.6103	116.3440
1.111	0.5933	1100	19.31	39.0	1.5579	93.6965
1.167	0.6472	1200	20.88	41.7	1.5210	91.6704
1.2217	0.7012	1300	21.29	41.72	1.4719	84.9167
1.0613	0.7551	1400	28.3	44.37	1.4663	67.1319
0.9256	0.8091	1500	27.5	45.59	1.4258	68.7078
0.8023	0.8630	1600	27.1	46.27	1.4027	72.7600
0.8327	0.9169	1700	27.03	46.19	1.3784	73.0302
0.7019	0.9709	1800	28.91	46.34	1.4127	67.4921
0.2681	1.0248	1900	28.53	47.12	1.3955	68.3026
0.2659	1.0787	2000	28.37	45.85	1.4194	68.1225

Framework versions

• Transformers 4.41.2
• Pytorch 2.2.0+cu121
• Datasets 2.20.0
• Tokenizers 0.19.1