EElayoutlmv3_jordyvl_rvl_cdip_100_examples_per_class_2023-09-03_txt_vis_concat_enc_12_ramp

This model is a fine-tuned version of microsoft/layoutlmv3-base on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 1.1463
• Accuracy: 0.805
• Exit 0 Accuracy: 0.08
• Exit 1 Accuracy: 0.8075

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 2
• eval_batch_size: 1
• seed: 42
• gradient_accumulation_steps: 24
• total_train_batch_size: 48
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 60

Training results

Training Loss	Epoch	Step	Validation Loss	Accuracy	Exit 0 Accuracy	Exit 1 Accuracy
No log	0.96	16	2.6926	0.115	0.03	0.0975
No log	1.98	33	2.5577	0.22	0.06	0.1975
No log	3.0	50	2.3770	0.305	0.065	0.305
No log	3.96	66	2.1060	0.43	0.0675	0.43
No log	4.98	83	1.8310	0.5775	0.065	0.5275
No log	6.0	100	1.6004	0.615	0.065	0.595
No log	6.96	116	1.3907	0.6925	0.065	0.6375
No log	7.98	133	1.2449	0.71	0.065	0.71
No log	9.0	150	1.1319	0.74	0.0675	0.7325
No log	9.96	166	0.9893	0.7575	0.065	0.75
No log	10.98	183	0.9431	0.7575	0.0675	0.7475
No log	12.0	200	0.8968	0.76	0.0675	0.7575
No log	12.96	216	0.8665	0.7725	0.065	0.765
No log	13.98	233	0.9219	0.735	0.0675	0.745
No log	15.0	250	0.8944	0.7475	0.065	0.755
No log	15.96	266	0.8463	0.79	0.075	0.7825
No log	16.98	283	0.9329	0.75	0.07	0.7475
No log	18.0	300	0.9706	0.76	0.065	0.76
No log	18.96	316	1.0194	0.745	0.065	0.75
No log	19.98	333	0.9081	0.785	0.07	0.79
No log	21.0	350	0.9894	0.785	0.075	0.7775
No log	21.96	366	1.0477	0.74	0.075	0.7425
No log	22.98	383	0.9729	0.7825	0.075	0.7825
No log	24.0	400	1.0044	0.79	0.08	0.7925
No log	24.96	416	1.0300	0.78	0.08	0.7875
No log	25.98	433	0.9863	0.7975	0.0775	0.7975
No log	27.0	450	0.9913	0.7975	0.075	0.8
No log	27.96	466	1.0085	0.8	0.0775	0.8025
No log	28.98	483	1.0336	0.8	0.0775	0.8025
1.2663	30.0	500	1.0423	0.7925	0.08	0.8
1.2663	30.96	516	1.0509	0.7925	0.0775	0.8025
1.2663	31.98	533	1.0561	0.7925	0.08	0.795
1.2663	33.0	550	1.0546	0.8	0.08	0.8
1.2663	33.96	566	1.0632	0.8	0.08	0.8
1.2663	34.98	583	1.0605	0.805	0.075	0.81
1.2663	36.0	600	1.1232	0.795	0.0775	0.7975
1.2663	36.96	616	1.0872	0.805	0.0775	0.8025
1.2663	37.98	633	1.0939	0.81	0.0775	0.81
1.2663	39.0	650	1.0951	0.8125	0.0775	0.81
1.2663	39.96	666	1.1014	0.81	0.08	0.81
1.2663	40.98	683	1.1039	0.81	0.08	0.81
1.2663	42.0	700	1.1108	0.81	0.08	0.8075
1.2663	42.96	716	1.1139	0.81	0.08	0.8125
1.2663	43.98	733	1.1177	0.81	0.08	0.8125
1.2663	45.0	750	1.1245	0.8075	0.08	0.81
1.2663	45.96	766	1.1309	0.8075	0.08	0.8075
1.2663	46.98	783	1.1336	0.805	0.08	0.8075
1.2663	48.0	800	1.1336	0.8075	0.08	0.8075
1.2663	48.96	816	1.1368	0.8075	0.08	0.8075
1.2663	49.98	833	1.1348	0.805	0.08	0.8075
1.2663	51.0	850	1.1389	0.805	0.08	0.805
1.2663	51.96	866	1.1414	0.805	0.08	0.805
1.2663	52.98	883	1.1422	0.805	0.08	0.805
1.2663	54.0	900	1.1435	0.805	0.08	0.8075
1.2663	54.96	916	1.1442	0.805	0.08	0.8075
1.2663	55.98	933	1.1461	0.805	0.08	0.8075
1.2663	57.0	950	1.1462	0.805	0.08	0.8075
1.2663	57.6	960	1.1463	0.805	0.08	0.8075

Framework versions

• Transformers 4.31.0
• Pytorch 2.0.1+cu117
• Datasets 2.13.1
• Tokenizers 0.13.3