Edit model card

lyric-gpt

This model is a fine-tuned version of gpt2 on the None dataset. It achieves the following results on the evaluation set:

  • Loss: 5.4526

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

  • learning_rate: 0.0001
  • train_batch_size: 8
  • eval_batch_size: 8
  • seed: 42
  • optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
  • lr_scheduler_type: linear
  • num_epochs: 30

Training results

Training Loss Epoch Step Validation Loss
3.5327 1.0 92 3.1885
2.8988 2.0 184 3.1986
3.0192 3.0 276 3.2477
2.5232 4.0 368 3.3318
2.3098 5.0 460 3.3899
2.1412 6.0 552 3.5328
2.3526 7.0 644 3.6322
2.06 8.0 736 3.7834
1.9477 9.0 828 3.8887
1.6156 10.0 920 4.0527
1.5496 11.0 1012 4.2364
1.5933 12.0 1104 4.3163
1.3698 13.0 1196 4.4754
1.308 14.0 1288 4.5911
1.1803 15.0 1380 4.6902
1.0212 16.0 1472 4.7977
0.9838 17.0 1564 4.9419
1.0065 18.0 1656 4.9470
0.8158 19.0 1748 5.0347
0.8352 20.0 1840 5.1256
0.8759 21.0 1932 5.1902
0.7176 22.0 2024 5.2542
0.7457 23.0 2116 5.2947
0.6683 24.0 2208 5.3511
0.6095 25.0 2300 5.3680
0.6247 26.0 2392 5.3916
0.5827 27.0 2484 5.4273
0.5742 28.0 2576 5.4394
0.5334 29.0 2668 5.4477
0.6028 30.0 2760 5.4526

Framework versions

  • Transformers 4.34.1
  • Pytorch 2.1.0+cu118
  • Datasets 2.14.6
  • Tokenizers 0.14.1
Downloads last month
18
Inference Examples
This model does not have enough activity to be deployed to Inference API (serverless) yet. Increase its social visibility and check back later, or deploy to Inference Endpoints (dedicated) instead.

Model tree for ai-maker-space/Transformers-Workshop-GPT-Generation

Finetuned
(1129)
this model

Dataset used to train ai-maker-space/Transformers-Workshop-GPT-Generation

Space using ai-maker-space/Transformers-Workshop-GPT-Generation 1