long-t5-local-base-finetuned-justification-v08

This model is a fine-tuned version of google/long-t5-local-base on the None dataset. It achieves the following results on the evaluation set:

• Loss: nan

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 3e-07
• train_batch_size: 1
• eval_batch_size: 1
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 50

Training results

Training Loss	Epoch	Step	Validation Loss
5.1991	1.0	676	nan
0.0	2.0	1352	nan
0.0	3.0	2028	nan
0.0	4.0	2704	nan
0.0	5.0	3380	nan
0.0	6.0	4056	nan
0.0	7.0	4732	nan
0.0	8.0	5408	nan
0.0	9.0	6084	nan
0.0	10.0	6760	nan
0.0	11.0	7436	nan
0.0	12.0	8112	nan
0.0	13.0	8788	nan
0.0	14.0	9464	nan
0.0	15.0	10140	nan
0.0	16.0	10816	nan
0.0	17.0	11492	nan
0.0	18.0	12168	nan
0.0	19.0	12844	nan
0.0	20.0	13520	nan
0.0	21.0	14196	nan
0.0	22.0	14872	nan
0.0	23.0	15548	nan
0.0	24.0	16224	nan
0.0	25.0	16900	nan
0.0	26.0	17576	nan
0.0	27.0	18252	nan
0.0	28.0	18928	nan
0.0	29.0	19604	nan
0.0	30.0	20280	nan
0.0	31.0	20956	nan
0.0	32.0	21632	nan
0.0	33.0	22308	nan
0.0	34.0	22984	nan
0.0	35.0	23660	nan
0.0	36.0	24336	nan
0.0	37.0	25012	nan
0.0	38.0	25688	nan
0.0	39.0	26364	nan
0.0	40.0	27040	nan
0.0	41.0	27716	nan
0.0	42.0	28392	nan
0.0	43.0	29068	nan
0.0	44.0	29744	nan
0.0	45.0	30420	nan
0.0	46.0	31096	nan
0.0	47.0	31772	nan
0.0	48.0	32448	nan
0.0	49.0	33124	nan
0.0	50.0	33800	nan

Framework versions

• Transformers 4.39.3
• Pytorch 2.2.2+cu121
• Datasets 2.18.0
• Tokenizers 0.15.2