flan-t5-large-P-tuning-cpgQA

This model is a fine-tuned version of google/flan-t5-large on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 0.1423
• Squad: {'exact_match': 59.63302752293578, 'f1': 82.39001389589451}
• Bleu: {'bleu': 0.6044954004940006, 'precisions': [0.6696606786427146, 0.6282194848824189, 0.5842839036755386, 0.5432276657060519], 'brevity_penalty': 1.0, 'length_ratio': 1.3012987012987014, 'translation_length': 1002, 'reference_length': 770}

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 3e-05
• train_batch_size: 8
• eval_batch_size: 8
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 8

Training results

Training Loss	Epoch	Step	Validation Loss	Squad	Bleu
0.3136	1.0	494	0.1411	{'exact_match': 57.79816513761468, 'f1': 81.81283979578464}	{'bleu': 0.5819448175214944, 'precisions': [0.6526946107784432, 0.6080627099664053, 0.5602027883396705, 0.515850144092219], 'brevity_penalty': 1.0, 'length_ratio': 1.363265306122449, 'translation_length': 1002, 'reference_length': 735}
0.28	2.0	988	0.1437	{'exact_match': 57.79816513761468, 'f1': 81.00462660225033}	{'bleu': 0.5717467837073172, 'precisions': [0.6417165668662674, 0.5968645016797313, 0.550063371356147, 0.5072046109510087], 'brevity_penalty': 1.0, 'length_ratio': 1.3707250341997264, 'translation_length': 1002, 'reference_length': 731}
0.2351	3.0	1482	0.1436	{'exact_match': 57.79816513761468, 'f1': 80.92424215489342}	{'bleu': 0.5786949268545348, 'precisions': [0.6477045908183633, 0.6035834266517357, 0.5576679340937896, 0.5144092219020173], 'brevity_penalty': 1.0, 'length_ratio': 1.3431635388739946, 'translation_length': 1002, 'reference_length': 746}
0.2736	4.0	1976	0.1434	{'exact_match': 57.79816513761468, 'f1': 80.98182982715996}	{'bleu': 0.5763280403149159, 'precisions': [0.6467065868263473, 0.6024636058230683, 0.5551330798479087, 0.5100864553314121], 'brevity_penalty': 1.0, 'length_ratio': 1.3449664429530201, 'translation_length': 1002, 'reference_length': 745}
0.2408	5.0	2470	0.1415	{'exact_match': 57.79816513761468, 'f1': 81.54539470265146}	{'bleu': 0.5849077021683632, 'precisions': [0.653692614770459, 0.6103023516237402, 0.5640050697084917, 0.5201729106628242], 'brevity_penalty': 1.0, 'length_ratio': 1.3306772908366533, 'translation_length': 1002, 'reference_length': 753}
0.2513	6.0	2964	0.1426	{'exact_match': 59.63302752293578, 'f1': 81.88361818381664}	{'bleu': 0.5934811323519371, 'precisions': [0.6606786427145709, 0.6181410974244121, 0.5728770595690748, 0.5302593659942363], 'brevity_penalty': 1.0, 'length_ratio': 1.323645970937913, 'translation_length': 1002, 'reference_length': 757}
0.2231	7.0	3458	0.1419	{'exact_match': 59.63302752293578, 'f1': 82.39001389589451}	{'bleu': 0.6044954004940006, 'precisions': [0.6696606786427146, 0.6282194848824189, 0.5842839036755386, 0.5432276657060519], 'brevity_penalty': 1.0, 'length_ratio': 1.3012987012987014, 'translation_length': 1002, 'reference_length': 770}
0.2575	8.0	3952	0.1423	{'exact_match': 59.63302752293578, 'f1': 82.39001389589451}	{'bleu': 0.6044954004940006, 'precisions': [0.6696606786427146, 0.6282194848824189, 0.5842839036755386, 0.5432276657060519], 'brevity_penalty': 1.0, 'length_ratio': 1.3012987012987014, 'translation_length': 1002, 'reference_length': 770}

Framework versions

• Transformers 4.33.0
• Pytorch 2.0.0
• Datasets 2.1.0
• Tokenizers 0.13.3