EElayoutlmv3_jordyvl_rvl_cdip_100_examples_per_class_2023-09-01_txt_vis_concat_enc_4_ramp

This model is a fine-tuned version of microsoft/layoutlmv3-base on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 1.6270
• Accuracy: 0.7175
• Exit 0 Accuracy: 0.09
• Exit 1 Accuracy: 0.7225

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 2
• eval_batch_size: 1
• seed: 42
• gradient_accumulation_steps: 24
• total_train_batch_size: 48
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 60

Training results

Training Loss	Epoch	Step	Validation Loss	Accuracy	Exit 0 Accuracy	Exit 1 Accuracy
No log	0.96	16	2.6970	0.1225	0.05	0.0625
No log	1.98	33	2.5370	0.2275	0.0375	0.0875
No log	3.0	50	2.3578	0.2975	0.06	0.1175
No log	3.96	66	2.1969	0.3925	0.0625	0.13
No log	4.98	83	2.0006	0.4775	0.075	0.2325
No log	6.0	100	1.8354	0.5175	0.0775	0.27
No log	6.96	116	1.6223	0.5975	0.0775	0.3675
No log	7.98	133	1.4348	0.65	0.0675	0.4525
No log	9.0	150	1.2723	0.6775	0.07	0.4975
No log	9.96	166	1.1623	0.6975	0.07	0.515
No log	10.98	183	1.0860	0.7175	0.07	0.575
No log	12.0	200	1.0837	0.6925	0.0725	0.57
No log	12.96	216	1.0867	0.685	0.075	0.5975
No log	13.98	233	1.0405	0.7125	0.075	0.5925
No log	15.0	250	1.1247	0.7025	0.0775	0.6125
No log	15.96	266	1.0507	0.7225	0.0825	0.615
No log	16.98	283	1.1754	0.6875	0.08	0.6
No log	18.0	300	1.1605	0.685	0.0775	0.63
No log	18.96	316	1.1766	0.7025	0.0775	0.645
No log	19.98	333	1.1271	0.7125	0.08	0.6375
No log	21.0	350	1.1904	0.73	0.0825	0.6475
No log	21.96	366	1.2511	0.7025	0.08	0.64
No log	22.98	383	1.3078	0.7175	0.08	0.66
No log	24.0	400	1.2960	0.7025	0.0775	0.6475
No log	24.96	416	1.3926	0.695	0.08	0.6575
No log	25.98	433	1.4649	0.69	0.0825	0.67
No log	27.0	450	1.4266	0.7075	0.08	0.6925
No log	27.96	466	1.4971	0.7	0.08	0.6775
No log	28.98	483	1.3950	0.715	0.085	0.7
1.5293	30.0	500	1.4578	0.7125	0.0875	0.6925
1.5293	30.96	516	1.4085	0.7175	0.08	0.6925
1.5293	31.98	533	1.4643	0.705	0.0825	0.7025
1.5293	33.0	550	1.4807	0.7175	0.0825	0.705
1.5293	33.96	566	1.5091	0.73	0.0825	0.7025
1.5293	34.98	583	1.4994	0.7275	0.0825	0.7025
1.5293	36.0	600	1.5193	0.7275	0.0825	0.7125
1.5293	36.96	616	1.5334	0.7275	0.085	0.73
1.5293	37.98	633	1.5487	0.7075	0.08	0.7225
1.5293	39.0	650	1.5068	0.7225	0.0775	0.7175
1.5293	39.96	666	1.5550	0.7225	0.0825	0.72
1.5293	40.98	683	1.5202	0.7175	0.0825	0.7225
1.5293	42.0	700	1.6623	0.695	0.0875	0.705
1.5293	42.96	716	1.5383	0.725	0.09	0.725
1.5293	43.98	733	1.5419	0.7275	0.0875	0.715
1.5293	45.0	750	1.6383	0.715	0.085	0.7175
1.5293	45.96	766	1.6017	0.725	0.0875	0.7175
1.5293	46.98	783	1.5820	0.7325	0.085	0.715
1.5293	48.0	800	1.6027	0.72	0.0875	0.73
1.5293	48.96	816	1.6122	0.72	0.0925	0.7175
1.5293	49.98	833	1.6359	0.715	0.09	0.73
1.5293	51.0	850	1.6087	0.72	0.09	0.73
1.5293	51.96	866	1.6284	0.715	0.09	0.7175
1.5293	52.98	883	1.6058	0.72	0.09	0.72
1.5293	54.0	900	1.6177	0.7225	0.09	0.725
1.5293	54.96	916	1.6188	0.725	0.09	0.73
1.5293	55.98	933	1.6162	0.725	0.09	0.7275
1.5293	57.0	950	1.6261	0.7175	0.09	0.7225
1.5293	57.6	960	1.6270	0.7175	0.09	0.7225

Framework versions

• Transformers 4.31.0
• Pytorch 2.0.1+cu117
• Datasets 2.13.1
• Tokenizers 0.13.3