metadata
license: apache-2.0
base_model: t5-small
tags:
- generated_from_trainer
metrics:
- bleu
model-index:
- name: ft-t5-small-on-info-lg-formal
results: []
ft-t5-small-on-info-lg-formal
This model is a fine-tuned version of t5-small on the Luganda Formal Data dataset. It achieves the following results on the evaluation set:
- Loss: 0.1997
- Bleu: 15.1631
- Gen Len: 12.3001
Model description
More information needed
Intended uses & limitations
More information needed
Training and evaluation data
More information needed
Training procedure
Training hyperparameters
The following hyperparameters were used during training:
- learning_rate: 0.0001
- train_batch_size: 16
- eval_batch_size: 16
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- num_epochs: 15
Training results
| Training Loss | Epoch | Step | Validation Loss | Bleu | Gen Len |
|---|---|---|---|---|---|
| 0.5515 | 1.0 | 846 | 0.3160 | 1.5468 | 12.8596 |
| 0.3157 | 2.0 | 1692 | 0.2813 | 3.6176 | 13.2076 |
| 0.2801 | 3.0 | 2538 | 0.2576 | 5.4512 | 12.9075 |
| 0.2655 | 4.0 | 3384 | 0.2432 | 7.2817 | 12.6248 |
| 0.2468 | 5.0 | 4230 | 0.2307 | 8.9484 | 12.6766 |
| 0.2339 | 6.0 | 5076 | 0.2228 | 10.7122 | 12.475 |
| 0.225 | 7.0 | 5922 | 0.2165 | 11.5595 | 12.4025 |
| 0.2152 | 8.0 | 6768 | 0.2119 | 12.6485 | 12.3446 |
| 0.2094 | 9.0 | 7614 | 0.2079 | 12.9033 | 12.5243 |
| 0.2053 | 10.0 | 8460 | 0.2048 | 13.9509 | 12.4165 |
| 0.2019 | 11.0 | 9306 | 0.2033 | 14.4313 | 12.2947 |
| 0.1975 | 12.0 | 10152 | 0.2014 | 14.6399 | 12.2768 |
| 0.195 | 13.0 | 10998 | 0.2004 | 14.9622 | 12.3094 |
| 0.1927 | 14.0 | 11844 | 0.2000 | 14.8282 | 12.3087 |
| 0.1917 | 15.0 | 12690 | 0.1997 | 15.1631 | 12.3001 |
Framework versions
- Transformers 4.42.3
- Pytorch 2.1.2
- Datasets 2.20.0
- Tokenizers 0.19.1