EElayoutlmv3_jordyvl_rvl_cdip_100_examples_per_class_2023-09-02_txt_vis_concat_enc_8_ramp

This model is a fine-tuned version of microsoft/layoutlmv3-base on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 1.3797
• Accuracy: 0.7775
• Exit 0 Accuracy: 0.0975
• Exit 1 Accuracy: 0.7825

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 2
• eval_batch_size: 1
• seed: 42
• gradient_accumulation_steps: 24
• total_train_batch_size: 48
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 60

Training results

Training Loss	Epoch	Step	Validation Loss	Accuracy	Exit 0 Accuracy	Exit 1 Accuracy
No log	0.96	16	2.6923	0.125	0.0425	0.0625
No log	1.98	33	2.5475	0.215	0.0675	0.0625
No log	3.0	50	2.4085	0.2725	0.075	0.0625
No log	3.96	66	2.1946	0.365	0.0775	0.0625
No log	4.98	83	1.9525	0.5	0.08	0.0625
No log	6.0	100	1.6874	0.5775	0.08	0.0625
No log	6.96	116	1.4652	0.6475	0.0825	0.0625
No log	7.98	133	1.3144	0.67	0.0825	0.0625
No log	9.0	150	1.1765	0.71	0.0825	0.0625
No log	9.96	166	1.0615	0.7325	0.085	0.0625
No log	10.98	183	1.0225	0.7425	0.085	0.0625
No log	12.0	200	0.9310	0.7775	0.085	0.09
No log	12.96	216	0.9445	0.7475	0.0875	0.25
No log	13.98	233	0.9819	0.74	0.085	0.5325
No log	15.0	250	0.9187	0.755	0.085	0.6775
No log	15.96	266	0.9655	0.75	0.09	0.705
No log	16.98	283	1.0417	0.7525	0.08	0.7125
No log	18.0	300	0.9947	0.7675	0.08	0.7325
No log	18.96	316	1.0721	0.7475	0.085	0.725
No log	19.98	333	1.0403	0.765	0.0825	0.74
No log	21.0	350	1.0728	0.76	0.085	0.7475
No log	21.96	366	1.1415	0.75	0.09	0.745
No log	22.98	383	1.0932	0.765	0.09	0.7775
No log	24.0	400	1.1408	0.77	0.095	0.775
No log	24.96	416	1.1579	0.775	0.0975	0.7675
No log	25.98	433	1.1688	0.7725	0.0925	0.78
No log	27.0	450	1.1945	0.7675	0.09	0.77
No log	27.96	466	1.1929	0.7675	0.09	0.7775
No log	28.98	483	1.2495	0.77	0.0925	0.775
1.457	30.0	500	1.1974	0.7775	0.0975	0.79
1.457	30.96	516	1.2452	0.7725	0.09	0.785
1.457	31.98	533	1.2672	0.7775	0.095	0.7825
1.457	33.0	550	1.2877	0.7725	0.095	0.78
1.457	33.96	566	1.2928	0.7775	0.0975	0.785
1.457	34.98	583	1.2982	0.775	0.095	0.7825
1.457	36.0	600	1.3094	0.775	0.095	0.7875
1.457	36.96	616	1.3342	0.77	0.095	0.78
1.457	37.98	633	1.3218	0.77	0.0925	0.7825
1.457	39.0	650	1.3302	0.77	0.095	0.79
1.457	39.96	666	1.3409	0.7725	0.095	0.7825
1.457	40.98	683	1.3496	0.7725	0.0975	0.7825
1.457	42.0	700	1.3411	0.7775	0.095	0.7825
1.457	42.96	716	1.3441	0.775	0.0975	0.785
1.457	43.98	733	1.3500	0.775	0.095	0.785
1.457	45.0	750	1.3569	0.775	0.095	0.785
1.457	45.96	766	1.3555	0.775	0.0975	0.7875
1.457	46.98	783	1.3589	0.775	0.0975	0.7825
1.457	48.0	800	1.3597	0.7725	0.0925	0.78
1.457	48.96	816	1.3666	0.7725	0.0975	0.785
1.457	49.98	833	1.3718	0.7675	0.095	0.7825
1.457	51.0	850	1.3767	0.7725	0.0975	0.785
1.457	51.96	866	1.3846	0.775	0.0975	0.7825
1.457	52.98	883	1.3835	0.775	0.0975	0.7825
1.457	54.0	900	1.3832	0.775	0.0975	0.785
1.457	54.96	916	1.3807	0.775	0.0975	0.785
1.457	55.98	933	1.3793	0.7775	0.0975	0.785
1.457	57.0	950	1.3796	0.7775	0.0975	0.7825
1.457	57.6	960	1.3797	0.7775	0.0975	0.7825

Framework versions

• Transformers 4.31.0
• Pytorch 2.0.1+cu117
• Datasets 2.13.1
• Tokenizers 0.13.3