EElayoutlmv3_jordyvl_rvl_cdip_100_examples_per_class_2023-09-02_txt_vis_concat_enc_7_ramp

This model is a fine-tuned version of microsoft/layoutlmv3-base on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 1.3516
• Accuracy: 0.7825
• Exit 0 Accuracy: 0.0825
• Exit 1 Accuracy: 0.7525

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 2
• eval_batch_size: 1
• seed: 42
• gradient_accumulation_steps: 24
• total_train_batch_size: 48
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 60

Training results

Training Loss	Epoch	Step	Validation Loss	Accuracy	Exit 0 Accuracy	Exit 1 Accuracy
No log	0.96	16	2.6991	0.1275	0.0475	0.0625
No log	1.98	33	2.5453	0.2225	0.0375	0.0775
No log	3.0	50	2.3689	0.2975	0.045	0.165
No log	3.96	66	2.1411	0.41	0.0525	0.4175
No log	4.98	83	1.8781	0.5325	0.055	0.5175
No log	6.0	100	1.6031	0.64	0.065	0.5725
No log	6.96	116	1.4385	0.655	0.065	0.595
No log	7.98	133	1.2796	0.6875	0.065	0.635
No log	9.0	150	1.1812	0.7225	0.065	0.6575
No log	9.96	166	1.0855	0.735	0.0675	0.68
No log	10.98	183	1.0009	0.755	0.0675	0.6925
No log	12.0	200	1.0138	0.73	0.0675	0.685
No log	12.96	216	0.9252	0.7525	0.0675	0.68
No log	13.98	233	0.9161	0.7525	0.07	0.69
No log	15.0	250	1.0396	0.73	0.065	0.6925
No log	15.96	266	1.0077	0.7425	0.07	0.6925
No log	16.98	283	0.9578	0.7625	0.0675	0.71
No log	18.0	300	0.9925	0.7625	0.07	0.72
No log	18.96	316	1.0645	0.75	0.075	0.725
No log	19.98	333	1.0794	0.7525	0.07	0.7325
No log	21.0	350	1.0704	0.76	0.07	0.7325
No log	21.96	366	1.0809	0.7675	0.075	0.7425
No log	22.98	383	1.1286	0.7675	0.0725	0.745
No log	24.0	400	1.0816	0.78	0.08	0.7475
No log	24.96	416	1.1370	0.765	0.0825	0.7525
No log	25.98	433	1.2095	0.7625	0.0825	0.7475
No log	27.0	450	1.2139	0.77	0.08	0.755
No log	27.96	466	1.2498	0.77	0.08	0.7475
No log	28.98	483	1.2696	0.7575	0.085	0.7425
1.3298	30.0	500	1.2227	0.7825	0.09	0.7575
1.3298	30.96	516	1.2820	0.77	0.0825	0.7525
1.3298	31.98	533	1.2411	0.7825	0.0825	0.7525
1.3298	33.0	550	1.2531	0.775	0.085	0.7425
1.3298	33.96	566	1.2552	0.77	0.085	0.7375
1.3298	34.98	583	1.2638	0.78	0.0825	0.7525
1.3298	36.0	600	1.2798	0.775	0.0775	0.7525
1.3298	36.96	616	1.2777	0.7875	0.0775	0.76
1.3298	37.98	633	1.2900	0.78	0.0775	0.7525
1.3298	39.0	650	1.3050	0.78	0.08	0.7575
1.3298	39.96	666	1.3243	0.7825	0.08	0.75
1.3298	40.98	683	1.3234	0.7775	0.0825	0.7525
1.3298	42.0	700	1.3262	0.78	0.0775	0.7525
1.3298	42.96	716	1.3246	0.7875	0.0825	0.755
1.3298	43.98	733	1.3223	0.7825	0.08	0.7525
1.3298	45.0	750	1.3297	0.7825	0.0775	0.7525
1.3298	45.96	766	1.3311	0.7875	0.0825	0.7475
1.3298	46.98	783	1.3421	0.7775	0.0825	0.7525
1.3298	48.0	800	1.3358	0.7825	0.08	0.7525
1.3298	48.96	816	1.3365	0.78	0.0825	0.7525
1.3298	49.98	833	1.3442	0.785	0.08	0.755
1.3298	51.0	850	1.3480	0.785	0.0825	0.755
1.3298	51.96	866	1.3450	0.785	0.0825	0.755
1.3298	52.98	883	1.3472	0.7825	0.0825	0.75
1.3298	54.0	900	1.3486	0.78	0.0825	0.75
1.3298	54.96	916	1.3438	0.7825	0.0825	0.75
1.3298	55.98	933	1.3507	0.7825	0.0825	0.75
1.3298	57.0	950	1.3515	0.7825	0.0825	0.7525
1.3298	57.6	960	1.3516	0.7825	0.0825	0.7525

Framework versions

• Transformers 4.31.0
• Pytorch 2.0.1+cu117
• Datasets 2.13.1
• Tokenizers 0.13.3