metadata
tags:
- generated_from_trainer
datasets:
- uonlp/CulturaX
metrics:
- accuracy
model-index:
- name: gpt2+ts_cx-en_00000-00009_50k
results:
- task:
name: Causal Language Modeling
type: text-generation
dataset:
name: uonlp/CulturaX en
type: uonlp/CulturaX
args: en
metrics:
- name: Accuracy
type: accuracy
value: 0.3894698710798747
license: mit
language:
- en
gpt2+ts_cx-en_00000-00009_50k
This model is a fine-tuned version of on the uonlp/CulturaX en dataset. It achieves the following results on the evaluation set:
- Loss: 3.4121
- Accuracy: 0.3895
Model description
More information needed
Intended uses & limitations
More information needed
Training and evaluation data
More information needed
Training procedure
Training hyperparameters
The following hyperparameters were used during training:
- learning_rate: 5e-05
- train_batch_size: 64
- eval_batch_size: 64
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- num_epochs: 1.0
Training results
| Training Loss | Epoch | Step | Validation Loss | Accuracy |
|---|---|---|---|---|
| 4.375 | 0.04 | 10000 | 4.2815 | 0.3111 |
| 4.0754 | 0.08 | 20000 | 3.9984 | 0.3341 |
| 3.9409 | 0.11 | 30000 | 3.8615 | 0.3457 |
| 3.8554 | 0.15 | 40000 | 3.7798 | 0.3531 |
| 3.7973 | 0.19 | 50000 | 3.7210 | 0.3584 |
| 3.7421 | 0.23 | 60000 | 3.6750 | 0.3630 |
| 3.7097 | 0.27 | 70000 | 3.6378 | 0.3664 |
| 3.6741 | 0.3 | 80000 | 3.6061 | 0.3694 |
| 3.6599 | 0.34 | 90000 | 3.5803 | 0.3718 |
| 3.6356 | 0.38 | 100000 | 3.5584 | 0.3741 |
| 3.6131 | 0.42 | 110000 | 3.5423 | 0.3758 |
| 3.5991 | 0.46 | 120000 | 3.5254 | 0.3776 |
| 3.591 | 0.49 | 130000 | 3.5108 | 0.3790 |
| 3.574 | 0.53 | 140000 | 3.4966 | 0.3805 |
| 3.5606 | 0.57 | 150000 | 3.4866 | 0.3815 |
| 3.5516 | 0.61 | 160000 | 3.4739 | 0.3828 |
| 3.5423 | 0.64 | 170000 | 3.4650 | 0.3838 |
| 3.5298 | 0.68 | 180000 | 3.4560 | 0.3847 |
| 3.5287 | 0.72 | 190000 | 3.4479 | 0.3857 |
| 3.5187 | 0.76 | 200000 | 3.4408 | 0.3863 |
| 3.5157 | 0.8 | 210000 | 3.4339 | 0.3870 |
| 3.5042 | 0.83 | 220000 | 3.4286 | 0.3876 |
| 3.5033 | 0.87 | 230000 | 3.4229 | 0.3883 |
| 3.501 | 0.91 | 240000 | 3.4188 | 0.3888 |
| 3.4946 | 0.95 | 250000 | 3.4149 | 0.3892 |
| 3.4971 | 0.99 | 260000 | 3.4126 | 0.3894 |
Framework versions
- Transformers 4.37.1
- Pytorch 2.1.2+cu121
- Datasets 2.16.1
- Tokenizers 0.15.1