t5-large-finetune-keyword-to-text-generation

This model is a fine-tuned version of t5-large on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 3.1471
• Rouge1: 2.175
• Rouge2: 0.3661
• Rougel: 1.7927
• Rougelsum: 1.7951
• Gen Len: 15.3252

Model description

This model is designed to generate text from a single keyword. This project is intended to be used for generating vocabulary questions for ed-tech applications.

NOTE!: Be sure to use the 'summarize: ' prefix before the word that you would like to un-summarize.

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 8
• eval_batch_size: 8
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 10

Training results

Training Loss	Epoch	Step	Validation Loss	Rouge1	Rouge2	Rougel	Rougelsum	Gen Len
3.3083	1.0	3000	3.1706	2.1498	0.331	1.7579	1.761	16.6826
3.2121	2.0	6000	3.1403	2.1555	0.3409	1.7659	1.769	16.208
3.1286	3.0	9000	3.1300	2.1577	0.3511	1.7703	1.7733	15.9009
3.0567	4.0	12000	3.1282	2.183	0.3584	1.7895	1.7909	15.7135
2.9953	5.0	15000	3.1293	2.1589	0.3525	1.776	1.7781	15.678
2.9483	6.0	18000	3.1308	2.1645	0.3556	1.7824	1.784	15.425
2.9009	7.0	21000	3.1358	2.1622	0.3622	1.7848	1.7877	15.3348
2.8752	8.0	24000	3.1387	2.1716	0.36	1.7936	1.7963	15.5296
2.835	9.0	27000	3.1454	2.1806	0.3658	1.7941	1.7966	15.4625
2.8352	10.0	30000	3.1471	2.175	0.3661	1.7927	1.7951	15.3252

Framework versions

• Transformers 4.21.2
• Pytorch 1.12.1+cu113
• Datasets 2.4.0
• Tokenizers 0.12.1