FLAN-T5-Base-Finetune-Remarks-Trend

This model is a fine-tuned version of google/flan-t5-base on trend remarks dataset. It achieves the following results on the evaluation set:

• Loss: 0.3399
• Rouge1: 36.8734
• Rouge2: 30.0539
• Rougel: 34.3307
• Rougelsum: 34.3313
• Gen Len: 19.0

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 16
• eval_batch_size: 16
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 4

Training results

Training Loss	Epoch	Step	Validation Loss	Rouge1	Rouge2	Rougel	Rougelsum	Gen Len
0.4722	1.0	1124	0.3844	36.439	29.4842	34.2141	34.2138	19.0
0.4071	2.0	2248	0.3552	36.8048	30.0131	34.2166	34.212	19.0
0.3922	3.0	3372	0.3438	36.8067	29.98	34.2816	34.2799	19.0
0.3862	4.0	4496	0.3399	36.8734	30.0539	34.3307	34.3313	19.0

Framework versions

• Transformers 4.44.2
• Pytorch 2.4.0
• Datasets 3.0.0
• Tokenizers 0.19.1