EElayoutlmv3_jordyvl_rvl_cdip_100_examples_per_class_2023-09-01_txt_vis_concat_enc_6_ramp

This model is a fine-tuned version of microsoft/layoutlmv3-base on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 1.4817
• Accuracy: 0.7625
• Exit 0 Accuracy: 0.09
• Exit 1 Accuracy: 0.7475

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 2
• eval_batch_size: 1
• seed: 42
• gradient_accumulation_steps: 24
• total_train_batch_size: 48
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 60

Training results

Training Loss	Epoch	Step	Validation Loss	Accuracy	Exit 0 Accuracy	Exit 1 Accuracy
No log	0.96	16	2.6831	0.1275	0.0525	0.0625
No log	1.98	33	2.5250	0.2525	0.035	0.0975
No log	3.0	50	2.3236	0.3375	0.0625	0.2775
No log	3.96	66	2.0234	0.4475	0.0675	0.415
No log	4.98	83	1.7420	0.5775	0.0825	0.49
No log	6.0	100	1.5328	0.635	0.0775	0.5225
No log	6.96	116	1.3275	0.6825	0.0725	0.5875
No log	7.98	133	1.1960	0.705	0.0725	0.6075
No log	9.0	150	1.0898	0.725	0.0775	0.6125
No log	9.96	166	1.0232	0.7325	0.0825	0.6325
No log	10.98	183	0.9708	0.745	0.075	0.6575
No log	12.0	200	0.9516	0.7525	0.08	0.6475
No log	12.96	216	0.9288	0.7575	0.08	0.675
No log	13.98	233	1.0144	0.725	0.08	0.66
No log	15.0	250	0.9685	0.75	0.08	0.6825
No log	15.96	266	0.9704	0.7425	0.085	0.6775
No log	16.98	283	0.9901	0.7725	0.085	0.685
No log	18.0	300	1.0792	0.75	0.085	0.6675
No log	18.96	316	1.0894	0.745	0.0825	0.6975
No log	19.98	333	1.0638	0.7475	0.085	0.715
No log	21.0	350	1.1147	0.76	0.085	0.71
No log	21.96	366	1.1803	0.745	0.0875	0.725
No log	22.98	383	1.1308	0.7525	0.085	0.7425
No log	24.0	400	1.2403	0.7525	0.085	0.7475
No log	24.96	416	1.2687	0.745	0.0925	0.745
No log	25.98	433	1.1862	0.7425	0.09	0.72
No log	27.0	450	1.1882	0.7775	0.09	0.745
No log	27.96	466	1.2807	0.735	0.09	0.7375
No log	28.98	483	1.3265	0.7475	0.09	0.73
1.3459	30.0	500	1.2653	0.7625	0.09	0.735
1.3459	30.96	516	1.2625	0.7575	0.0875	0.7525
1.3459	31.98	533	1.3163	0.7525	0.0925	0.7525
1.3459	33.0	550	1.3334	0.7625	0.0925	0.73
1.3459	33.96	566	1.3998	0.7575	0.0925	0.735
1.3459	34.98	583	1.3609	0.76	0.09	0.75
1.3459	36.0	600	1.3793	0.755	0.0875	0.745
1.3459	36.96	616	1.3695	0.76	0.0875	0.7525
1.3459	37.98	633	1.3866	0.7575	0.0875	0.745
1.3459	39.0	650	1.3995	0.7625	0.09	0.7425
1.3459	39.96	666	1.4202	0.755	0.0925	0.75
1.3459	40.98	683	1.4166	0.755	0.095	0.735
1.3459	42.0	700	1.4389	0.745	0.0875	0.745
1.3459	42.96	716	1.4526	0.7625	0.0925	0.75
1.3459	43.98	733	1.4500	0.76	0.0875	0.7475
1.3459	45.0	750	1.4613	0.765	0.0875	0.74
1.3459	45.96	766	1.4589	0.76	0.0925	0.7475
1.3459	46.98	783	1.4711	0.76	0.09	0.745
1.3459	48.0	800	1.4707	0.76	0.0875	0.7475
1.3459	48.96	816	1.4805	0.76	0.09	0.7475
1.3459	49.98	833	1.4813	0.76	0.0875	0.7475
1.3459	51.0	850	1.4795	0.76	0.09	0.7525
1.3459	51.96	866	1.4826	0.76	0.09	0.75
1.3459	52.98	883	1.4825	0.76	0.09	0.7525
1.3459	54.0	900	1.4820	0.7625	0.09	0.75
1.3459	54.96	916	1.4803	0.7625	0.09	0.75
1.3459	55.98	933	1.4814	0.7625	0.09	0.7475
1.3459	57.0	950	1.4816	0.7625	0.09	0.7475
1.3459	57.6	960	1.4817	0.7625	0.09	0.7475

Framework versions

• Transformers 4.31.0
• Pytorch 2.0.1+cu117
• Datasets 2.13.1
• Tokenizers 0.13.3