Hugging Face's logo

wcvz
/
esm2_t130_150M-lora-classifier_2024-04-26_00-25-40

PEFT
Safetensors
Generated from Trainer
Model card Files Community
esm2_t130_150M-lora-classifier_2024-04-26_00-25-40 / README.md
wcvz's picture
wcvz
Model save
51dd844 verified
preview code
|
raw
history blame contribute delete
20.4 kB
metadata
license: mit
library_name: peft
tags:
  - generated_from_trainer
base_model: facebook/esm2_t30_150M_UR50D
metrics:
  - accuracy
model-index:
  - name: esm2_t130_150M-lora-classifier_2024-04-26_00-25-40
    results: []

esm2_t130_150M-lora-classifier_2024-04-26_00-25-40

This model is a fine-tuned version of facebook/esm2_t30_150M_UR50D on the None dataset. It achieves the following results on the evaluation set:

  • Loss: 1.6470
  • Accuracy: 0.8887

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

  • learning_rate: 0.0005701568055793089
  • train_batch_size: 28
  • eval_batch_size: 28
  • seed: 8893
  • optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
  • lr_scheduler_type: cosine
  • num_epochs: 300
  • mixed_precision_training: Native AMP

Training results

Training Loss Epoch Step Validation Loss Accuracy
0.7096 1.0 55 0.6718 0.6055
0.6769 2.0 110 0.6739 0.6055
0.579 3.0 165 0.6608 0.6484
0.5726 4.0 220 0.5777 0.7109
0.6381 5.0 275 0.5020 0.7676
0.183 6.0 330 0.3725 0.8320
0.3701 7.0 385 0.3508 0.8535
0.2147 8.0 440 0.3191 0.8711
0.1654 9.0 495 0.3036 0.875
0.1581 10.0 550 0.3761 0.8516
0.3459 11.0 605 0.3746 0.8594
0.3325 12.0 660 0.3025 0.8867
0.1237 13.0 715 0.2983 0.8770
0.5167 14.0 770 0.3044 0.8887
0.3541 15.0 825 0.2927 0.8906
0.0378 16.0 880 0.3669 0.8906
0.062 17.0 935 0.3298 0.8887
0.1695 18.0 990 0.2912 0.9004
0.0444 19.0 1045 0.3034 0.9004
0.1794 20.0 1100 0.3641 0.8828
0.0634 21.0 1155 0.3521 0.8867
0.0446 22.0 1210 0.3438 0.8887
0.0266 23.0 1265 0.4553 0.8867
0.2637 24.0 1320 0.4715 0.8867
0.159 25.0 1375 0.4323 0.8945
0.2401 26.0 1430 0.6019 0.8809
0.1317 27.0 1485 0.5549 0.8906
0.1223 28.0 1540 0.4819 0.8926
0.0015 29.0 1595 0.6432 0.8711
0.0007 30.0 1650 0.6480 0.8926
0.0774 31.0 1705 0.7596 0.8926
0.1262 32.0 1760 0.7614 0.8809
0.034 33.0 1815 0.7392 0.8789
0.0021 34.0 1870 0.9068 0.8848
0.0003 35.0 1925 0.8724 0.8711
0.0001 36.0 1980 0.9483 0.8867
0.0127 37.0 2035 0.9638 0.8828
0.0001 38.0 2090 0.9105 0.8926
0.0001 39.0 2145 0.9231 0.8809
0.0008 40.0 2200 1.0224 0.8867
0.0001 41.0 2255 1.0666 0.8848
0.0002 42.0 2310 1.1028 0.8848
0.0 43.0 2365 0.9653 0.8906
0.0006 44.0 2420 1.1108 0.8848
0.0001 45.0 2475 1.2919 0.8730
0.0002 46.0 2530 1.0834 0.8926
0.0002 47.0 2585 1.1240 0.8887
0.0135 48.0 2640 1.1466 0.8887
0.0008 49.0 2695 1.2674 0.8691
0.0 50.0 2750 1.1311 0.8887
0.0086 51.0 2805 1.0957 0.8887
0.0 52.0 2860 1.1336 0.8789
0.0007 53.0 2915 1.1494 0.875
0.0002 54.0 2970 1.0790 0.8848
0.0002 55.0 3025 1.1489 0.8809
0.0 56.0 3080 1.1479 0.8867
0.0022 57.0 3135 1.2092 0.8848
0.2415 58.0 3190 1.2060 0.8848
0.7813 59.0 3245 1.3750 0.8613
0.0 60.0 3300 1.1202 0.875
0.0 61.0 3355 1.0502 0.8848
0.0 62.0 3410 1.3270 0.8730
0.0015 63.0 3465 1.0082 0.875
0.0002 64.0 3520 0.9724 0.8867
0.0014 65.0 3575 1.0862 0.8770
0.0002 66.0 3630 1.1366 0.8730
0.1868 67.0 3685 1.1838 0.8770
0.0004 68.0 3740 1.2073 0.875
0.0007 69.0 3795 1.1793 0.8770
0.0 70.0 3850 1.2262 0.8652
0.2838 71.0 3905 1.2415 0.875
0.0 72.0 3960 1.2346 0.8770
0.0041 73.0 4015 1.0830 0.8789
0.0055 74.0 4070 1.0731 0.8867
0.0 75.0 4125 1.4096 0.8652
0.0034 76.0 4180 1.1142 0.8711
0.0 77.0 4235 1.0250 0.8848
0.0002 78.0 4290 1.0700 0.8691
0.0009 79.0 4345 0.9032 0.8789
0.0001 80.0 4400 1.0556 0.8730
0.0001 81.0 4455 1.0740 0.8770
0.0002 82.0 4510 1.2571 0.8691
0.0 83.0 4565 1.2007 0.8809
0.0 84.0 4620 1.2515 0.875
0.0001 85.0 4675 1.0750 0.8828
0.0006 86.0 4730 1.3016 0.8730
0.0001 87.0 4785 1.2393 0.8809
0.0 88.0 4840 1.2232 0.8848
0.0003 89.0 4895 1.2187 0.8789
0.0 90.0 4950 1.2328 0.8730
0.0 91.0 5005 1.3026 0.8848
0.0 92.0 5060 1.3152 0.8770
0.0 93.0 5115 1.4069 0.875
0.0 94.0 5170 1.3988 0.8770
0.0 95.0 5225 1.3675 0.8594
0.0 96.0 5280 1.3366 0.8770
0.0003 97.0 5335 1.2140 0.8848
0.0 98.0 5390 1.3585 0.8711
0.0 99.0 5445 1.1665 0.8672
0.0 100.0 5500 1.0947 0.8809
0.0099 101.0 5555 1.2993 0.8730
0.0 102.0 5610 1.3578 0.8789
0.0 103.0 5665 1.3596 0.8867
0.0006 104.0 5720 1.3164 0.8848
0.0 105.0 5775 1.4100 0.8770
0.0 106.0 5830 1.3459 0.875
0.0005 107.0 5885 1.3783 0.8809
0.0 108.0 5940 1.2698 0.8770
0.0 109.0 5995 1.3933 0.8848
0.0 110.0 6050 1.3813 0.8809
0.0 111.0 6105 1.5747 0.875
0.0001 112.0 6160 1.3368 0.8867
0.0486 113.0 6215 1.3833 0.8828
0.1476 114.0 6270 1.4943 0.8828
0.0002 115.0 6325 1.4725 0.8789
0.0 116.0 6380 1.4614 0.875
0.0047 117.0 6435 1.6313 0.8770
0.0 118.0 6490 1.4459 0.8848
0.0026 119.0 6545 1.4150 0.8730
0.0 120.0 6600 1.6055 0.8555
0.0001 121.0 6655 1.3710 0.8789
0.3319 122.0 6710 1.3940 0.8867
0.0001 123.0 6765 1.2486 0.875
0.0002 124.0 6820 1.2946 0.8711
0.0 125.0 6875 1.2341 0.8711
0.0 126.0 6930 1.1418 0.8887
0.0 127.0 6985 1.0713 0.8926
0.0001 128.0 7040 1.1391 0.8613
0.1624 129.0 7095 1.2195 0.8789
0.0 130.0 7150 1.1576 0.8770
0.0001 131.0 7205 1.2939 0.8730
0.0 132.0 7260 1.1568 0.8867
0.0 133.0 7315 1.2117 0.8848
0.0 134.0 7370 1.1264 0.8926
0.0 135.0 7425 1.1675 0.8848
0.0 136.0 7480 1.1983 0.8828
0.0 137.0 7535 1.2666 0.8770
0.0001 138.0 7590 1.1287 0.8848
0.0 139.0 7645 1.0505 0.8848
0.0 140.0 7700 1.1770 0.8770
0.0 141.0 7755 1.1749 0.8906
0.0 142.0 7810 1.1311 0.8711
0.0 143.0 7865 1.1114 0.8652
0.0 144.0 7920 1.1419 0.8691
0.0 145.0 7975 1.1666 0.8691
0.0 146.0 8030 1.1712 0.8711
0.0 147.0 8085 1.1831 0.8711
0.0 148.0 8140 1.1799 0.8711
0.0 149.0 8195 1.1876 0.8711
0.0 150.0 8250 1.1884 0.8730
0.0 151.0 8305 1.2389 0.8730
0.0 152.0 8360 1.3622 0.875
0.0 153.0 8415 1.2604 0.8789
0.0 154.0 8470 1.3336 0.875
0.0 155.0 8525 1.3496 0.8809
0.0 156.0 8580 1.3882 0.8555
0.1815 157.0 8635 1.3679 0.8789
0.288 158.0 8690 1.3804 0.8691
0.0 159.0 8745 1.2980 0.8770
0.0 160.0 8800 1.4075 0.8789
0.0 161.0 8855 1.4231 0.8789
0.0 162.0 8910 1.4730 0.875
0.0019 163.0 8965 1.5861 0.8672
0.0 164.0 9020 1.4080 0.8809
0.0005 165.0 9075 1.5852 0.8711
0.0 166.0 9130 1.5370 0.875
0.0 167.0 9185 1.5288 0.875
0.0 168.0 9240 1.5516 0.8711
0.0 169.0 9295 1.5268 0.8730
0.0 170.0 9350 1.5061 0.8672
0.0 171.0 9405 1.4843 0.875
0.0 172.0 9460 1.5478 0.8633
0.0 173.0 9515 1.4753 0.8730
0.0 174.0 9570 1.6709 0.8730
0.0 175.0 9625 1.6663 0.875
0.0 176.0 9680 1.6980 0.8672
0.0 177.0 9735 1.5563 0.8770
0.0 178.0 9790 1.6146 0.875
0.0 179.0 9845 1.5599 0.8770
0.0 180.0 9900 1.5558 0.8789
0.0 181.0 9955 1.8485 0.8633
0.0 182.0 10010 1.7223 0.8789
0.0 183.0 10065 1.7169 0.875
0.0 184.0 10120 1.7125 0.8711
0.0 185.0 10175 1.7065 0.8711
0.0 186.0 10230 1.7748 0.8730
0.0 187.0 10285 1.6861 0.8789
0.0 188.0 10340 1.7325 0.8887
0.0 189.0 10395 1.7658 0.8828
0.0 190.0 10450 1.7649 0.8809
0.0 191.0 10505 1.7555 0.8828
0.0162 192.0 10560 1.8313 0.8691
0.0001 193.0 10615 1.8314 0.8574
0.0 194.0 10670 1.7706 0.8672
0.0 195.0 10725 1.6568 0.8730
0.0 196.0 10780 1.6568 0.8770
0.0 197.0 10835 1.6185 0.8848
0.0 198.0 10890 1.6133 0.8848
0.0 199.0 10945 1.6129 0.8848
0.0 200.0 11000 1.6121 0.8848
0.0 201.0 11055 1.6104 0.8828
0.0 202.0 11110 1.6075 0.8828
0.0 203.0 11165 1.6153 0.8867
0.0 204.0 11220 1.6339 0.8828
0.0 205.0 11275 1.6164 0.8867
0.0 206.0 11330 1.6114 0.8848
0.0 207.0 11385 1.6122 0.8867
0.0 208.0 11440 1.6079 0.8867
0.0 209.0 11495 1.6132 0.8867
0.0 210.0 11550 1.6141 0.8867
0.0 211.0 11605 1.6122 0.8867
0.0 212.0 11660 1.6070 0.8867
0.0 213.0 11715 1.6010 0.8867
0.0 214.0 11770 1.6562 0.8789
0.0005 215.0 11825 1.6297 0.8887
0.0 216.0 11880 1.6070 0.8809
0.0 217.0 11935 1.6750 0.8770
0.0 218.0 11990 1.6822 0.8730
0.0 219.0 12045 1.6819 0.8730
0.0 220.0 12100 1.6846 0.8770
0.0 221.0 12155 1.6827 0.875
0.0 222.0 12210 1.6822 0.875
0.0 223.0 12265 1.6780 0.8770
0.0 224.0 12320 1.6813 0.8770
0.0 225.0 12375 1.6770 0.8770
0.0 226.0 12430 1.6878 0.8789
0.0 227.0 12485 1.8890 0.8672
0.0 228.0 12540 1.6978 0.8828
0.0 229.0 12595 1.6945 0.8867
0.0 230.0 12650 1.6960 0.8848
0.0 231.0 12705 1.6972 0.8867
0.0 232.0 12760 1.6929 0.8867
0.0 233.0 12815 1.6911 0.8848
0.0 234.0 12870 1.6887 0.8867
0.0 235.0 12925 1.6999 0.8848
0.0 236.0 12980 1.7000 0.8848
0.0 237.0 13035 1.6877 0.8867
0.0 238.0 13090 1.6858 0.8867
0.0 239.0 13145 1.6859 0.8867
0.0 240.0 13200 1.6842 0.8867
0.0 241.0 13255 1.6829 0.8867
0.0 242.0 13310 1.6800 0.8867
0.0 243.0 13365 1.6870 0.8848
0.0 244.0 13420 1.6856 0.8848
0.0 245.0 13475 1.6831 0.8848
0.0 246.0 13530 1.6864 0.8828
0.0 247.0 13585 1.6896 0.8828
0.0 248.0 13640 1.6900 0.8828
0.0 249.0 13695 1.6906 0.8848
0.0 250.0 13750 1.6928 0.8828
0.0 251.0 13805 1.6943 0.8828
0.0 252.0 13860 1.6902 0.8789
0.0 253.0 13915 1.6638 0.8887
0.0 254.0 13970 1.6632 0.8867
0.0 255.0 14025 1.6627 0.8867
0.0 256.0 14080 1.6631 0.8867
0.0 257.0 14135 1.6626 0.8867
0.0 258.0 14190 1.6629 0.8867
0.0 259.0 14245 1.6617 0.8867
0.0 260.0 14300 1.6606 0.8867
0.0 261.0 14355 1.6598 0.8867
0.0 262.0 14410 1.6559 0.8867
0.0 263.0 14465 1.6564 0.8867
0.0 264.0 14520 1.6555 0.8867
0.0 265.0 14575 1.6588 0.8867
0.0 266.0 14630 1.6565 0.8867
0.0 267.0 14685 1.6558 0.8867
0.0 268.0 14740 1.6564 0.8848
0.0 269.0 14795 1.6578 0.8848
0.0 270.0 14850 1.6566 0.8848
0.0 271.0 14905 1.6560 0.8867
0.0 272.0 14960 1.6587 0.8848
0.0 273.0 15015 1.6575 0.8867
0.0 274.0 15070 1.6575 0.8848
0.0 275.0 15125 1.6570 0.8867
0.0 276.0 15180 1.6586 0.8848
0.0 277.0 15235 1.6572 0.8887
0.0 278.0 15290 1.6577 0.8848
0.0 279.0 15345 1.6570 0.8867
0.0 280.0 15400 1.6567 0.8887
0.0 281.0 15455 1.6548 0.8887
0.0 282.0 15510 1.6558 0.8867
0.0 283.0 15565 1.6505 0.8887
0.0 284.0 15620 1.6515 0.8887
0.0 285.0 15675 1.6513 0.8887
0.0 286.0 15730 1.6456 0.8887
0.0 287.0 15785 1.6471 0.8887
0.0 288.0 15840 1.6451 0.8887
0.0 289.0 15895 1.6468 0.8887
0.0 290.0 15950 1.6470 0.8887
0.0 291.0 16005 1.6448 0.8887
0.0 292.0 16060 1.6478 0.8887
0.0 293.0 16115 1.6475 0.8887
0.0 294.0 16170 1.6471 0.8887
0.0 295.0 16225 1.6476 0.8887
0.0 296.0 16280 1.6475 0.8887
0.0 297.0 16335 1.6460 0.8887
0.0 298.0 16390 1.6471 0.8887
0.0 299.0 16445 1.6469 0.8887
0.0 300.0 16500 1.6470 0.8887

Framework versions

  • PEFT 0.10.0
  • Transformers 4.39.3
  • Pytorch 2.2.1
  • Datasets 2.16.1
  • Tokenizers 0.15.2