rubert-tiny2-1-4 / README.md
Adammz's picture
End of training
65f1701
|
raw
history blame
2.33 kB
metadata
license: mit
base_model: cointegrated/rubert-tiny2
tags:
  - generated_from_trainer
metrics:
  - accuracy
model-index:
  - name: rubert-tiny2-1-4
    results: []

rubert-tiny2-1-4

This model is a fine-tuned version of cointegrated/rubert-tiny2 on the None dataset. It achieves the following results on the evaluation set:

  • Loss: 0.3882
  • Accuracy: 0.9001

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

  • learning_rate: 1e-05
  • train_batch_size: 32
  • eval_batch_size: 32
  • seed: 42
  • optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
  • lr_scheduler_type: linear
  • lr_scheduler_warmup_steps: 500
  • num_epochs: 15
  • mixed_precision_training: Native AMP

Training results

Training Loss Epoch Step Validation Loss Accuracy
1.9597 1.0 1500 1.1052 0.7613
0.9583 2.0 3000 0.8140 0.8157
0.7343 3.0 4500 0.6514 0.8502
0.6076 4.0 6000 0.5656 0.867
0.5257 5.0 7500 0.5115 0.8771
0.4694 6.0 9000 0.4748 0.8826
0.4296 7.0 10500 0.4477 0.8885
0.4006 8.0 12000 0.4295 0.8938
0.3753 9.0 13500 0.4159 0.896
0.358 10.0 15000 0.4066 0.8979
0.3417 11.0 16500 0.3994 0.8992
0.3296 12.0 18000 0.3943 0.8993
0.3203 13.0 19500 0.3914 0.8993
0.3158 14.0 21000 0.3889 0.9001
0.3126 15.0 22500 0.3882 0.9001

Framework versions

  • Transformers 4.35.2
  • Pytorch 2.1.0+cu121
  • Datasets 2.16.0
  • Tokenizers 0.15.0