EElayoutlmv3_jordyvl_rvl_cdip_100_examples_per_class_2023-09-02_txt_vis_concat_enc_10_ramp

This model is a fine-tuned version of microsoft/layoutlmv3-base on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 1.3065
• Accuracy: 0.78
• Exit 0 Accuracy: 0.08
• Exit 1 Accuracy: 0.78

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 2
• eval_batch_size: 1
• seed: 42
• gradient_accumulation_steps: 24
• total_train_batch_size: 48
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 60

Training results

Training Loss	Epoch	Step	Validation Loss	Accuracy	Exit 0 Accuracy	Exit 1 Accuracy
No log	0.96	16	2.6831	0.1225	0.0425	0.0625
No log	1.98	33	2.5269	0.2475	0.0475	0.0625
No log	3.0	50	2.3314	0.3	0.065	0.0625
No log	3.96	66	2.1407	0.3875	0.065	0.0625
No log	4.98	83	1.9319	0.495	0.065	0.0625
No log	6.0	100	1.6262	0.6075	0.0675	0.0625
No log	6.96	116	1.4189	0.6525	0.0675	0.0625
No log	7.98	133	1.2238	0.705	0.0675	0.0625
No log	9.0	150	1.1216	0.725	0.065	0.0625
No log	9.96	166	1.0243	0.745	0.0675	0.0625
No log	10.98	183	0.9489	0.7725	0.0675	0.0625
No log	12.0	200	0.9294	0.755	0.065	0.0625
No log	12.96	216	0.9293	0.765	0.0675	0.0625
No log	13.98	233	0.9327	0.76	0.07	0.0625
No log	15.0	250	0.9204	0.785	0.065	0.55
No log	15.96	266	0.9853	0.765	0.0725	0.7075
No log	16.98	283	0.9700	0.77	0.0675	0.745
No log	18.0	300	1.0333	0.755	0.0675	0.7475
No log	18.96	316	1.0310	0.765	0.0675	0.7675
No log	19.98	333	0.9923	0.785	0.07	0.7925
No log	21.0	350	1.0907	0.7825	0.07	0.78
No log	21.96	366	1.0952	0.7775	0.07	0.78
No log	22.98	383	1.1303	0.7675	0.07	0.775
No log	24.0	400	1.0843	0.78	0.0725	0.78
No log	24.96	416	1.1523	0.7775	0.075	0.7775
No log	25.98	433	1.1420	0.77	0.07	0.765
No log	27.0	450	1.1594	0.7775	0.0675	0.7775
No log	27.96	466	1.1929	0.775	0.07	0.7775
No log	28.98	483	1.1958	0.78	0.0725	0.785
1.4332	30.0	500	1.1998	0.7775	0.0775	0.785
1.4332	30.96	516	1.2055	0.7825	0.0725	0.7825
1.4332	31.98	533	1.2077	0.7825	0.0775	0.78
1.4332	33.0	550	1.2200	0.78	0.0775	0.7825
1.4332	33.96	566	1.2262	0.78	0.0775	0.7825
1.4332	34.98	583	1.2393	0.775	0.0725	0.78
1.4332	36.0	600	1.2447	0.7775	0.075	0.78
1.4332	36.96	616	1.2493	0.7775	0.0725	0.7775
1.4332	37.98	633	1.2579	0.775	0.0775	0.7775
1.4332	39.0	650	1.2564	0.7775	0.0775	0.78
1.4332	39.96	666	1.2599	0.7775	0.0775	0.78
1.4332	40.98	683	1.2615	0.7775	0.08	0.785
1.4332	42.0	700	1.2718	0.78	0.075	0.78
1.4332	42.96	716	1.2750	0.78	0.0775	0.78
1.4332	43.98	733	1.2776	0.78	0.075	0.78
1.4332	45.0	750	1.2833	0.7825	0.075	0.78
1.4332	45.96	766	1.2873	0.7825	0.0775	0.78
1.4332	46.98	783	1.2951	0.78	0.0775	0.7775
1.4332	48.0	800	1.2968	0.78	0.075	0.7775
1.4332	48.96	816	1.2979	0.78	0.0775	0.78
1.4332	49.98	833	1.2994	0.78	0.075	0.78
1.4332	51.0	850	1.3009	0.78	0.0775	0.7775
1.4332	51.96	866	1.3009	0.7775	0.0775	0.78
1.4332	52.98	883	1.3019	0.78	0.0775	0.7825
1.4332	54.0	900	1.3036	0.78	0.08	0.78
1.4332	54.96	916	1.3048	0.78	0.08	0.78
1.4332	55.98	933	1.3058	0.78	0.08	0.78
1.4332	57.0	950	1.3063	0.78	0.08	0.78
1.4332	57.6	960	1.3065	0.78	0.08	0.78

Framework versions

• Transformers 4.31.0
• Pytorch 2.0.1+cu117
• Datasets 2.13.1
• Tokenizers 0.13.3