vi-poem-gpt-neo / README.md
duydatnguyen's picture
Completed Process
60bae80 verified
metadata
base_model: NlpHUST/gpt-neo-vi-small
tags:
  - generated_from_trainer
model-index:
  - name: vi_gpt_poem_
    results: []

vi_gpt_poem_

This model is a fine-tuned version of NlpHUST/gpt-neo-vi-small on the None dataset. It achieves the following results on the evaluation set:

  • Loss: 1.1334

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

  • learning_rate: 1e-05
  • train_batch_size: 42
  • eval_batch_size: 42
  • seed: 42
  • optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
  • lr_scheduler_type: linear
  • num_epochs: 250
  • mixed_precision_training: Native AMP

Training results

Training Loss Epoch Step Validation Loss
6.8634 3.9683 500 6.1900
4.7999 7.9365 1000 3.4039
2.7473 11.9048 1500 2.5766
2.2513 15.8730 2000 2.2051
1.9426 19.8413 2500 1.9113
1.7059 23.8095 3000 1.6723
1.5333 27.7778 3500 1.5196
1.3996 31.7460 4000 1.4060
1.3066 35.7143 4500 1.3193
1.228 39.6825 5000 1.2513
1.1642 43.6508 5500 1.2000
1.1191 47.6190 6000 1.1607
1.0825 51.5873 6500 1.1295
1.0483 55.5556 7000 1.1036
1.0203 59.5238 7500 1.0818
0.9967 63.4921 8000 1.0631
0.9745 67.4603 8500 1.0471
0.9552 71.4286 9000 1.0332
0.9362 75.3968 9500 1.0208
0.9165 79.3651 10000 1.0098
0.8977 83.3333 10500 1.0002
0.8846 87.3016 11000 0.9915
0.8641 91.2698 11500 0.9838
0.8478 95.2381 12000 0.9779
0.8286 99.2063 12500 0.9721
0.811 103.1746 13000 0.9677
0.7916 107.1429 13500 0.9644
0.7721 111.1111 14000 0.9625
0.7513 115.0794 14500 0.9616
0.7292 119.0476 15000 0.9617
0.7066 123.0159 15500 0.9622
0.683 126.9841 16000 0.9639
0.6582 130.9524 16500 0.9661
0.632 134.9206 17000 0.9690
0.6047 138.8889 17500 0.9727
0.5769 142.8571 18000 0.9763
0.548 146.8254 18500 0.9802
0.5169 150.7937 19000 0.9844
0.4863 154.7619 19500 0.9887
0.4536 158.7302 20000 0.9936
0.4223 162.6984 20500 0.9975
0.3891 166.6667 21000 1.0022
0.3571 170.6349 21500 1.0071
0.3256 174.6032 22000 1.0118
0.2946 178.5714 22500 1.0164
0.2642 182.5397 23000 1.0221
0.2345 186.5079 23500 1.0271
0.2069 190.4762 24000 1.0331
0.1806 194.4444 24500 1.0393
0.1565 198.4127 25000 1.0462
0.1351 202.3810 25500 1.0527
0.1153 206.3492 26000 1.0605
0.0984 210.3175 26500 1.0679
0.0842 214.2857 27000 1.0758
0.0721 218.2540 27500 1.0827
0.0627 222.2222 28000 1.0906
0.0555 226.1905 28500 1.0978
0.0495 230.1587 29000 1.1043
0.045 234.1270 29500 1.1107
0.0412 238.0952 30000 1.1166
0.0382 242.0635 30500 1.1228
0.0356 246.0317 31000 1.1275
0.0335 250.0 31500 1.1334

Framework versions

  • Transformers 4.40.0
  • Pytorch 2.1.2
  • Datasets 2.16.1
  • Tokenizers 0.19.1