---
language:
- ko
- ja
base_model: facebook/mbart-large-50-many-to-many-mmt
tags:
- generated_from_trainer
metrics:
- bleu
model-index:
- name: mbart-mmt_mid3_ko-ja
  results: []
---

<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->

# mbart-mmt_mid3_ko-ja

This model is a fine-tuned version of [facebook/mbart-large-50-many-to-many-mmt](https://huggingface.co/facebook/mbart-large-50-many-to-many-mmt) on an unknown dataset.
It achieves the following results on the evaluation set:
- Loss: 0.8652
- Bleu: 10.1883
- Gen Len: 17.2057

## Model description

More information needed

## Intended uses & limitations

More information needed

## Training and evaluation data

More information needed

## Training procedure

### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 5e-05
- train_batch_size: 4
- eval_batch_size: 4
- seed: 42
- distributed_type: multi-GPU
- num_devices: 4
- gradient_accumulation_steps: 2
- total_train_batch_size: 32
- total_eval_batch_size: 16
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- lr_scheduler_warmup_steps: 500
- num_epochs: 35

### Training results

| Training Loss | Epoch | Step  | Validation Loss | Bleu    | Gen Len |
|:-------------:|:-----:|:-----:|:---------------:|:-------:|:-------:|
| 1.6216        | 0.23  | 1500  | 1.5229          | 2.686   | 17.599  |
| 1.3587        | 0.46  | 3000  | 1.3061          | 4.0749  | 17.3772 |
| 1.2279        | 0.68  | 4500  | 1.1881          | 5.2878  | 17.3642 |
| 1.1408        | 0.91  | 6000  | 1.0994          | 5.4783  | 17.4093 |
| 0.9977        | 1.14  | 7500  | 1.0313          | 7.6015  | 17.36   |
| 0.9582        | 1.37  | 9000  | 0.9918          | 8.2303  | 17.3526 |
| 0.9525        | 1.59  | 10500 | 0.9811          | 8.2837  | 17.2597 |
| 0.9415        | 1.82  | 12000 | 0.9589          | 8.1592  | 17.2241 |
| 0.856         | 2.05  | 13500 | 0.9462          | 7.8401  | 17.4066 |
| 0.8273        | 2.28  | 15000 | 0.9336          | 8.6082  | 17.1918 |
| 0.8066        | 2.5   | 16500 | 0.9220          | 9.7751  | 17.5198 |
| 0.784         | 2.73  | 18000 | 0.8949          | 10.292  | 17.4097 |
| 0.8016        | 2.96  | 19500 | 0.8958          | 9.0262  | 17.4097 |
| 0.6872        | 3.19  | 21000 | 0.9043          | 9.7549  | 17.2672 |
| 0.7107        | 3.42  | 22500 | 0.8994          | 10.3016 | 17.0973 |
| 0.6726        | 3.64  | 24000 | 0.8747          | 10.5183 | 17.2871 |
| 0.6699        | 3.87  | 25500 | 0.8652          | 10.1883 | 17.2057 |
| 0.612         | 4.1   | 27000 | 0.8949          | 9.5697  | 17.2443 |
| 0.621         | 4.33  | 28500 | 0.8904          | 10.8592 | 17.329  |
| 0.6219        | 4.55  | 30000 | 0.8772          | 10.925  | 17.482  |
| 0.6164        | 4.78  | 31500 | 0.8694          | 11.8749 | 17.1624 |


### Framework versions

- Transformers 4.34.0
- Pytorch 2.1.0+cu121
- Datasets 2.14.5
- Tokenizers 0.14.1