mms-MDPC

This model is a fine-tuned version of facebook/mms-1b-all on the None dataset. It achieves the following results on the evaluation set:

• Loss: 2.9364
• Wer: 70.0250

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 1e-05
• train_batch_size: 14
• eval_batch_size: 8
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: constant_with_warmup
• lr_scheduler_warmup_steps: 50
• num_epochs: 4

Training results

Training Loss	Epoch	Step	Validation Loss	Wer
19.864	0.06	250	16.1732	100.1101
3.687	0.12	500	3.8920	100.0
3.2582	0.17	750	3.7154	100.0100
2.7525	0.23	1000	3.3154	99.8199
2.2947	0.29	1250	2.9790	95.2376
2.024	0.35	1500	2.7973	91.3057
1.9249	0.4	1750	2.7070	89.1546
1.7769	0.46	2000	2.6510	88.0640
1.7417	0.52	2250	2.6255	86.5633
1.7468	0.58	2500	2.6030	85.4227
1.6818	0.63	2750	2.5864	84.7624
1.6304	0.69	3000	2.5921	84.3522
1.6494	0.75	3250	2.5835	83.5118
1.5068	0.81	3500	2.5737	82.3712
1.6079	0.87	3750	2.5621	81.7509
1.5069	0.92	4000	2.5641	80.9605
1.5596	0.98	4250	2.5636	80.2801
1.5396	1.04	4500	2.5623	79.7899
1.3875	1.1	4750	2.5761	79.5198
1.3952	1.15	5000	2.5841	79.1596
1.3948	1.21	5250	2.5849	78.5493
1.49	1.27	5500	2.5898	78.6693
1.3669	1.33	5750	2.5908	78.5393
1.3488	1.38	6000	2.6110	78.2491
1.3431	1.44	6250	2.6172	77.8589
1.3429	1.5	6500	2.6238	77.5788
1.3683	1.56	6750	2.6229	77.4887
1.4073	1.62	7000	2.6362	76.9485
1.2954	1.67	7250	2.6421	76.8384
1.3793	1.73	7500	2.6392	76.5183
1.3223	1.79	7750	2.6513	76.2281
1.2377	1.85	8000	2.6695	75.8479
1.2889	1.9	8250	2.6720	75.6778
1.2456	1.96	8500	2.6769	75.3877
1.2595	2.02	8750	2.6945	74.9275
1.2332	2.08	9000	2.6904	74.8374
1.2874	2.13	9250	2.7051	74.4372
1.2886	2.19	9500	2.6900	74.2171
1.3229	2.25	9750	2.7075	74.2271
1.245	2.31	10000	2.7114	73.8369
1.2316	2.37	10250	2.7207	73.4767
1.2379	2.42	10500	2.7261	73.7669
1.1906	2.48	10750	2.7471	73.2666
1.3066	2.54	11000	2.7522	73.3667
1.2382	2.6	11250	2.7464	73.0065
1.2262	2.65	11500	2.7626	72.8564
1.3256	2.71	11750	2.7778	72.7264
1.2251	2.77	12000	2.7764	72.4762
1.2187	2.83	12250	2.7751	72.2261
1.2674	2.88	12500	2.7987	72.1861
1.2015	2.94	12750	2.8093	72.0160
1.1485	3.0	13000	2.8161	71.8059
1.1686	3.06	13250	2.8240	71.8659
1.1331	3.11	13500	2.8269	71.6658
1.1589	3.17	13750	2.8500	71.5558
1.2203	3.23	14000	2.8497	71.4557
1.1592	3.29	14250	2.8542	71.4057
1.1957	3.35	14500	2.8789	70.9955
1.202	3.4	14750	2.8694	71.0255
1.1397	3.46	15000	2.8777	70.8354
1.2431	3.52	15250	2.8925	70.5753
1.178	3.58	15500	2.8910	70.6453
1.1716	3.63	15750	2.9085	70.6153
1.1464	3.69	16000	2.8951	70.3752
1.1307	3.75	16250	2.9159	70.5253
1.2094	3.81	16500	2.9170	70.1851
1.1988	3.86	16750	2.9155	70.1151
1.2435	3.92	17000	2.9272	70.1151
1.1212	3.98	17250	2.9364	70.0250

Framework versions

• Transformers 4.33.2
• Pytorch 2.0.1
• Datasets 2.19.1
• Tokenizers 0.13.3