lilt-xlm-roberta-base-finetuned-DocLayNet-base_paragraphs_ml512-v1

This model is a fine-tuned version of nielsr/lilt-xlm-roberta-base on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 0.4110
• Precision: 0.8507
• Recall: 0.8507
• F1: 0.8507
• Accuracy: 0.8507

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 8
• eval_batch_size: 16
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 1
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Precision	Recall	F1	Accuracy
No log	0.0533	100	0.9209	0.6685	0.6685	0.6685	0.6685
No log	0.1066	200	0.6489	0.8154	0.8154	0.8154	0.8154
No log	0.1599	300	0.7198	0.7420	0.7420	0.7420	0.7420
No log	0.2132	400	0.5487	0.7961	0.7961	0.7961	0.7961
0.736	0.2665	500	0.4430	0.8669	0.8669	0.8669	0.8669
0.736	0.3198	600	0.4406	0.8614	0.8614	0.8614	0.8614
0.736	0.3731	700	0.4112	0.8651	0.8651	0.8651	0.8651
0.736	0.4264	800	0.4101	0.8711	0.8711	0.8711	0.8711
0.736	0.4797	900	0.4691	0.8398	0.8398	0.8398	0.8398
0.4575	0.5330	1000	0.4894	0.8009	0.8009	0.8009	0.8009
0.4575	0.5864	1100	0.4275	0.8553	0.8553	0.8553	0.8553
0.4575	0.6397	1200	0.3614	0.8857	0.8857	0.8857	0.8857
0.4575	0.6930	1300	0.4571	0.8432	0.8432	0.8432	0.8432
0.4575	0.7463	1400	0.4478	0.8377	0.8377	0.8377	0.8377
0.3718	0.7996	1500	0.4381	0.8324	0.8324	0.8324	0.8324
0.3718	0.8529	1600	0.3267	0.8963	0.8963	0.8963	0.8963
0.3718	0.9062	1700	0.3470	0.8890	0.8890	0.8890	0.8890
0.3718	0.9595	1800	0.4110	0.8507	0.8507	0.8507	0.8507

Framework versions

• Transformers 4.41.2
• Pytorch 2.3.0+cu121
• Datasets 2.19.1
• Tokenizers 0.19.1