EElayoutlmv3_jordyvl_rvl_cdip_100_examples_per_class_2023-09-01_txt_vis_concat_enc_5_ramp

This model is a fine-tuned version of microsoft/layoutlmv3-base on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 1.7881
• Accuracy: 0.7275
• Exit 0 Accuracy: 0.09
• Exit 1 Accuracy: 0.705

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 2
• eval_batch_size: 1
• seed: 42
• gradient_accumulation_steps: 24
• total_train_batch_size: 48
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 60

Training results

Training Loss	Epoch	Step	Validation Loss	Accuracy	Exit 0 Accuracy	Exit 1 Accuracy
No log	0.96	16	2.7022	0.1225	0.0525	0.1
No log	1.98	33	2.5401	0.2375	0.0425	0.1775
No log	3.0	50	2.3562	0.32	0.06	0.3125
No log	3.96	66	2.0733	0.41	0.0675	0.3375
No log	4.98	83	1.8281	0.53	0.0675	0.4325
No log	6.0	100	1.5814	0.605	0.07	0.495
No log	6.96	116	1.4011	0.655	0.065	0.525
No log	7.98	133	1.2755	0.6925	0.0725	0.57
No log	9.0	150	1.1738	0.71	0.065	0.5975
No log	9.96	166	1.1008	0.7025	0.0675	0.5775
No log	10.98	183	1.1162	0.7025	0.0725	0.6275
No log	12.0	200	1.1360	0.6725	0.0725	0.6175
No log	12.96	216	1.0388	0.7325	0.0675	0.6475
No log	13.98	233	1.1008	0.705	0.0675	0.665
No log	15.0	250	1.1237	0.7125	0.0725	0.6575
No log	15.96	266	1.1345	0.7125	0.065	0.67
No log	16.98	283	1.1696	0.7125	0.075	0.6575
No log	18.0	300	1.2075	0.705	0.0775	0.655
No log	18.96	316	1.3137	0.705	0.0775	0.665
No log	19.98	333	1.3152	0.7	0.075	0.685
No log	21.0	350	1.3460	0.7	0.08	0.6725
No log	21.96	366	1.3561	0.7175	0.0825	0.6825
No log	22.98	383	1.4231	0.7075	0.085	0.685
No log	24.0	400	1.4084	0.72	0.0925	0.695
No log	24.96	416	1.4287	0.72	0.0875	0.705
No log	25.98	433	1.4479	0.7175	0.085	0.6925
No log	27.0	450	1.5538	0.715	0.085	0.6975
No log	27.96	466	1.5187	0.72	0.085	0.69
No log	28.98	483	1.5472	0.71	0.0875	0.6775
1.395	30.0	500	1.6103	0.705	0.0875	0.6875
1.395	30.96	516	1.6125	0.715	0.085	0.715
1.395	31.98	533	1.5962	0.7225	0.085	0.7025
1.395	33.0	550	1.6054	0.7225	0.0875	0.695
1.395	33.96	566	1.5790	0.72	0.0875	0.6975
1.395	34.98	583	1.5978	0.72	0.0875	0.71
1.395	36.0	600	1.6560	0.7125	0.09	0.7025
1.395	36.96	616	1.6633	0.7175	0.09	0.6975
1.395	37.98	633	1.6619	0.72	0.0875	0.6925
1.395	39.0	650	1.6841	0.72	0.09	0.6975
1.395	39.96	666	1.7132	0.7175	0.09	0.71
1.395	40.98	683	1.7284	0.7175	0.09	0.7025
1.395	42.0	700	1.7035	0.7275	0.0875	0.7025
1.395	42.96	716	1.7357	0.7225	0.09	0.71
1.395	43.98	733	1.7345	0.725	0.09	0.705
1.395	45.0	750	1.7187	0.7275	0.09	0.705
1.395	45.96	766	1.7534	0.7225	0.0925	0.7025
1.395	46.98	783	1.7550	0.7275	0.09	0.695
1.395	48.0	800	1.7578	0.73	0.09	0.7125
1.395	48.96	816	1.7672	0.73	0.09	0.7025
1.395	49.98	833	1.7894	0.725	0.09	0.69
1.395	51.0	850	1.7910	0.725	0.09	0.7075
1.395	51.96	866	1.7902	0.7225	0.09	0.705
1.395	52.98	883	1.7817	0.725	0.09	0.7025
1.395	54.0	900	1.7853	0.7275	0.09	0.695
1.395	54.96	916	1.7874	0.7275	0.09	0.7
1.395	55.98	933	1.7896	0.7275	0.09	0.705
1.395	57.0	950	1.7882	0.7275	0.09	0.705
1.395	57.6	960	1.7881	0.7275	0.09	0.705

Framework versions

• Transformers 4.31.0
• Pytorch 2.0.1+cu117
• Datasets 2.13.1
• Tokenizers 0.13.3