mms-MGB3

This model is a fine-tuned version of facebook/mms-1b-all on the None dataset. It achieves the following results on the evaluation set:

• Loss: 3.5770
• Wer: 99.9986

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 1e-05
• train_batch_size: 14
• eval_batch_size: 8
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: constant_with_warmup
• lr_scheduler_warmup_steps: 50
• num_epochs: 25

Training results

Training Loss	Epoch	Step	Validation Loss	Wer
8.1649	0.83	250	8.8182	100.0211
4.4764	1.66	500	5.1784	100.0254
3.962	2.48	750	4.6853	100.0310
3.7546	3.31	1000	4.4820	101.1220
3.5712	4.14	1250	4.3419	101.5181
5.9242	4.97	1500	4.2276	100.0
6.072	5.79	1750	4.1441	100.0
3.3164	6.62	2000	4.0701	100.0
3.2964	7.45	2250	3.9941	100.0
3.2501	8.28	2500	3.9978	100.0
3.2477	9.11	2750	3.9468	100.0
3.9197	9.93	3000	3.9109	100.0
3.1928	10.76	3250	3.8802	100.0
3.182	11.59	3500	3.8802	100.0
3.181	12.42	3750	3.7959	100.0
4.3975	13.25	4000	3.8292	100.0
3.8885	14.07	4250	3.7765	100.0
4.2643	14.9	4500	3.7765	100.0
3.1381	15.73	4750	3.7338	100.0
3.1197	16.56	5000	3.7391	100.0
3.1345	17.38	5250	3.7267	100.0
4.4275	18.21	5500	3.7405	100.0
4.3669	19.04	5750	3.7220	100.0
3.1225	19.87	6000	3.7093	99.9915
3.8043	20.7	6250	3.6449	99.9958
3.8089	21.52	6500	3.6988	100.0
4.2457	22.35	6750	3.6125	100.0
3.0956	23.18	7000	3.6309	99.9972
3.1013	24.01	7250	3.5845	99.9930
5.8493	24.83	7500	3.5770	99.9986

Framework versions

• Transformers 4.33.2
• Pytorch 2.0.1
• Datasets 2.19.1
• Tokenizers 0.13.3