Edit model card

bert-large-uncased-sst-2-64-13-30

This model is a fine-tuned version of bert-large-uncased on an unknown dataset. It achieves the following results on the evaluation set:

  • Loss: 0.6366
  • Accuracy: 0.8594

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

  • learning_rate: 1.5e-05
  • train_batch_size: 32
  • eval_batch_size: 32
  • seed: 42
  • optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
  • lr_scheduler_type: linear
  • lr_scheduler_warmup_steps: 5
  • num_epochs: 30

Training results

Training Loss Epoch Step Validation Loss Accuracy
No log 1.0 4 0.6822 0.5781
No log 2.0 8 0.6592 0.6172
0.6625 3.0 12 0.6330 0.6719
0.6625 4.0 16 0.5916 0.75
0.5388 5.0 20 0.5256 0.8047
0.5388 6.0 24 0.4758 0.8125
0.5388 7.0 28 0.4535 0.8203
0.3481 8.0 32 0.4449 0.8203
0.3481 9.0 36 0.4246 0.8203
0.2064 10.0 40 0.3979 0.8359
0.2064 11.0 44 0.3869 0.8594
0.2064 12.0 48 0.3902 0.8672
0.1105 13.0 52 0.4068 0.8516
0.1105 14.0 56 0.4299 0.8516
0.0583 15.0 60 0.4556 0.8516
0.0583 16.0 64 0.4352 0.8594
0.0583 17.0 68 0.4515 0.8672
0.0313 18.0 72 0.4882 0.8672
0.0313 19.0 76 0.5242 0.8516
0.0201 20.0 80 0.5497 0.8594
0.0201 21.0 84 0.5667 0.8594
0.0201 22.0 88 0.5827 0.8594
0.0132 23.0 92 0.6062 0.8516
0.0132 24.0 96 0.6265 0.8516
0.017 25.0 100 0.6330 0.8438
0.017 26.0 104 0.6334 0.8516
0.017 27.0 108 0.6354 0.8516
0.0211 28.0 112 0.6361 0.8594
0.0211 29.0 116 0.6363 0.8594
0.0113 30.0 120 0.6366 0.8594

Framework versions

  • Transformers 4.32.0.dev0
  • Pytorch 2.0.1+cu118
  • Datasets 2.4.0
  • Tokenizers 0.13.3
Downloads last month
8
Inference Examples
This model does not have enough activity to be deployed to Inference API (serverless) yet. Increase its social visibility and check back later, or deploy to Inference Endpoints (dedicated) instead.

Model tree for simonycl/bert-large-uncased-sst-2-64-13-30

Finetuned
(103)
this model