LifePrincipalPegasusLarge

This model is a fine-tuned version of google/pegasus-large on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 5.6547
• Rouge1: 43.3679
• Rouge2: 10.9243
• Rougel: 27.4864
• Rougelsum: 40.3896
• Gen Len: 227.7642

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 1
• eval_batch_size: 1
• seed: 42
• gradient_accumulation_steps: 16
• total_train_batch_size: 16
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• lr_scheduler_warmup_steps: 500
• num_epochs: 1

Training results

Training Loss	Epoch	Step	Validation Loss	Rouge1	Rouge2	Rougel	Rougelsum	Gen Len
6.8795	0.0881	100	6.5014	32.7482	6.8933	21.4622	30.1026	227.7642
6.4726	0.1762	200	6.2112	37.0799	8.9306	24.652	34.5305	227.7642
6.2749	0.2643	300	6.0775	38.9836	9.7478	25.6809	36.2375	227.7642
6.2445	0.3524	400	5.9878	39.9961	10.0625	25.9686	37.4853	227.7642
6.1752	0.4405	500	5.8898	40.1077	10.1194	26.1056	37.4283	227.7642
6.0343	0.5286	600	5.8116	41.7626	10.4065	26.7348	38.9953	227.7642
6.0009	0.6167	700	5.7529	42.0869	10.5523	26.9037	39.3663	227.7642
5.9523	0.7048	800	5.7057	42.4863	10.6188	27.1573	39.6282	227.7642
5.9445	0.7929	900	5.6786	42.4071	10.6897	27.1549	39.5086	227.7642
5.8047	0.8810	1000	5.6646	43.4098	10.9746	27.4646	40.4014	227.7642
5.8718	0.9691	1100	5.6547	43.3679	10.9243	27.4864	40.3896	227.7642

Framework versions

• Transformers 4.41.1
• Pytorch 2.1.2
• Datasets 2.2.1
• Tokenizers 0.19.1