metadata

license: mit
base_model: xlm-roberta-large
tags:
  - generated_from_trainer
model-index:
  - name: xlm-roberta-large_ALL_BCE_new_data_multihead_19_shuffled_special_tokens
    results: []

xlm-roberta-large_ALL_BCE_new_data_multihead_19_shuffled_special_tokens

This model is a fine-tuned version of xlm-roberta-large on the None dataset. It achieves the following results on the evaluation set:

Loss: 0.8445
F1 Macro 0.1: 0.0895
F1 Macro 0.15: 0.1160
F1 Macro 0.2: 0.1402
F1 Macro 0.25: 0.1634
F1 Macro 0.3: 0.1847
F1 Macro 0.35: 0.2040
F1 Macro 0.4: 0.2229
F1 Macro 0.45: 0.2406
F1 Macro 0.5: 0.2583
F1 Macro 0.55: 0.2763
F1 Macro 0.6: 0.2924
F1 Macro 0.65: 0.3101
F1 Macro 0.7: 0.3251
F1 Macro 0.75: 0.3405
F1 Macro 0.8: 0.3547
F1 Macro 0.85: 0.3634
F1 Macro 0.9: 0.3572
F1 Macro 0.95: 0.2839
Threshold 0: 0.8
Threshold 1: 0.85
Threshold 2: 0.9
Threshold 3: 0.9
Threshold 4: 0.8
Threshold 5: 0.85
Threshold 6: 0.8
Threshold 7: 0.9
Threshold 8: 0.9
Threshold 9: 0.8
Threshold 10: 0.95
Threshold 11: 0.85
Threshold 12: 0.9
Threshold 13: 0.8
Threshold 14: 0.9
Threshold 15: 0.85
Threshold 16: 0.85
Threshold 17: 0.85
Threshold 18: 0.9
0: 0.1543
1: 0.2738
2: 0.3791
3: 0.2915
4: 0.4439
5: 0.4944
6: 0.4463
7: 0.3216
8: 0.3402
9: 0.5410
10: 0.5665
11: 0.5310
12: 0.2331
13: 0.1319
14: 0.3899
15: 0.3173
16: 0.4432
17: 0.6120
18: 0.2342
Max F1: 0.3634
Mean F1: 0.3761

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

learning_rate: 5e-06
train_batch_size: 8
eval_batch_size: 8
seed: 2024
optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
lr_scheduler_type: linear
lr_scheduler_warmup_ratio: 0.1
num_epochs: 3
mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	F1 Macro 0.1	F1 Macro 0.15	F1 Macro 0.2	F1 Macro 0.25	F1 Macro 0.3	F1 Macro 0.35	F1 Macro 0.4	F1 Macro 0.45	F1 Macro 0.5	F1 Macro 0.55	F1 Macro 0.6	F1 Macro 0.65	F1 Macro 0.7	F1 Macro 0.75	F1 Macro 0.8	F1 Macro 0.85	F1 Macro 0.9	F1 Macro 0.95	Threshold 0	Threshold 1	Threshold 2	Threshold 3	Threshold 4	Threshold 5	Threshold 6	Threshold 7	Threshold 8	Threshold 9	Threshold 10	Threshold 11	Threshold 12	Threshold 13	Threshold 14	Threshold 15	Threshold 16	Threshold 17	Threshold 18	0	1	2	3	4	5	6	7	8	9	10	11	12	13	14	15	16	17	18	Max F1	Mean F1
1.2949	1.0	5595	0.9920	0.0638	0.0742	0.0860	0.0994	0.1129	0.1278	0.1430	0.1589	0.1751	0.1903	0.2064	0.2235	0.2373	0.2479	0.2512	0.2275	0.1775	0.0876	0.75	0.8	0.75	0.85	0.65	0.8	0.75	0.85	0.8	0.7	0.9	0.75	0.8	0.8	0.85	0.8	0.85	0.9	0.85	0.0863	0.1572	0.2169	0.0959	0.2903	0.3523	0.3723	0.1624	0.2313	0.4610	0.3852	0.4756	0.1678	0.1154	0.2816	0.1848	0.3673	0.5307	0.1168	0.2512	0.2658
0.9147	2.0	11190	0.9023	0.0813	0.1044	0.1275	0.1498	0.1706	0.1898	0.2088	0.2261	0.2449	0.2624	0.2798	0.2951	0.3107	0.3233	0.3328	0.3348	0.3156	0.2286	0.75	0.8	0.85	0.9	0.75	0.85	0.8	0.85	0.8	0.8	0.9	0.85	0.9	0.65	0.9	0.9	0.85	0.9	0.95	0.1231	0.2517	0.3359	0.2514	0.4106	0.4565	0.4166	0.2556	0.3152	0.5241	0.5686	0.5085	0.2177	0.1176	0.3757	0.3059	0.4286	0.5881	0.2143	0.3348	0.3508
0.732	3.0	16785	0.8445	0.0895	0.1160	0.1402	0.1634	0.1847	0.2040	0.2229	0.2406	0.2583	0.2763	0.2924	0.3101	0.3251	0.3405	0.3547	0.3634	0.3572	0.2839	0.8	0.85	0.9	0.9	0.8	0.85	0.8	0.9	0.9	0.8	0.95	0.85	0.9	0.8	0.9	0.85	0.85	0.85	0.9	0.1543	0.2738	0.3791	0.2915	0.4439	0.4944	0.4463	0.3216	0.3402	0.5410	0.5665	0.5310	0.2331	0.1319	0.3899	0.3173	0.4432	0.6120	0.2342	0.3634	0.3761

Framework versions

Transformers 4.36.1
Pytorch 2.1.0+cu121
Datasets 2.13.1
Tokenizers 0.15.0