layoutlm-funsd

This model is a fine-tuned version of microsoft/layoutlm-base-uncased on the funsd dataset. It achieves the following results on the evaluation set:

• Loss: 0.6847
• Answer: {'precision': 0.7144432194046306, 'recall': 0.8009888751545118, 'f1': 0.7552447552447553, 'number': 809}
• Header: {'precision': 0.30952380952380953, 'recall': 0.3277310924369748, 'f1': 0.31836734693877555, 'number': 119}
• Question: {'precision': 0.7912966252220248, 'recall': 0.8366197183098592, 'f1': 0.81332724783204, 'number': 1065}
• Overall Precision: 0.7309
• Overall Recall: 0.7918
• Overall F1: 0.7601
• Overall Accuracy: 0.8132

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 3e-05
• train_batch_size: 16
• eval_batch_size: 8
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 15

Training results

Training Loss	Epoch	Step	Validation Loss	Answer	Header	Question	Overall Precision	Overall Recall	Overall F1	Overall Accuracy
1.8057	1.0	10	1.5966	{'precision': 0.008733624454148471, 'recall': 0.009888751545117428, 'f1': 0.009275362318840578, 'number': 809}	{'precision': 0.0, 'recall': 0.0, 'f1': 0.0, 'number': 119}	{'precision': 0.14909090909090908, 'recall': 0.11549295774647887, 'f1': 0.13015873015873014, 'number': 1065}	0.0752	0.0657	0.0702	0.3764
1.4635	2.0	20	1.2374	{'precision': 0.14137483787289234, 'recall': 0.13473423980222496, 'f1': 0.1379746835443038, 'number': 809}	{'precision': 0.0, 'recall': 0.0, 'f1': 0.0, 'number': 119}	{'precision': 0.42204995693367786, 'recall': 0.460093896713615, 'f1': 0.440251572327044, 'number': 1065}	0.3100	0.3006	0.3052	0.6035
1.1031	3.0	30	0.9623	{'precision': 0.4551451187335092, 'recall': 0.4264524103831891, 'f1': 0.44033184428844924, 'number': 809}	{'precision': 0.13157894736842105, 'recall': 0.04201680672268908, 'f1': 0.06369426751592357, 'number': 119}	{'precision': 0.630297565374211, 'recall': 0.6563380281690141, 'f1': 0.6430542778288868, 'number': 1065}	0.5507	0.5263	0.5382	0.7016
0.8514	4.0	40	0.7967	{'precision': 0.6146682188591386, 'recall': 0.6526576019777504, 'f1': 0.6330935251798562, 'number': 809}	{'precision': 0.23333333333333334, 'recall': 0.11764705882352941, 'f1': 0.1564245810055866, 'number': 119}	{'precision': 0.6810422282120395, 'recall': 0.711737089201878, 'f1': 0.6960514233241506, 'number': 1065}	0.6398	0.6523	0.6460	0.7495
0.6854	5.0	50	0.7228	{'precision': 0.6617647058823529, 'recall': 0.723114956736712, 'f1': 0.6910809214412286, 'number': 809}	{'precision': 0.25806451612903225, 'recall': 0.20168067226890757, 'f1': 0.22641509433962265, 'number': 119}	{'precision': 0.697751873438801, 'recall': 0.7868544600938967, 'f1': 0.7396293027360988, 'number': 1065}	0.6644	0.7260	0.6938	0.7818
0.5608	6.0	60	0.6733	{'precision': 0.6585879873551106, 'recall': 0.7725587144622992, 'f1': 0.7110352673492606, 'number': 809}	{'precision': 0.25, 'recall': 0.17647058823529413, 'f1': 0.20689655172413793, 'number': 119}	{'precision': 0.7112561174551386, 'recall': 0.8187793427230047, 'f1': 0.7612396333478829, 'number': 1065}	0.6720	0.7617	0.7140	0.7976
0.486	7.0	70	0.6683	{'precision': 0.670514165792235, 'recall': 0.7898640296662547, 'f1': 0.7253121452894438, 'number': 809}	{'precision': 0.25688073394495414, 'recall': 0.23529411764705882, 'f1': 0.24561403508771928, 'number': 119}	{'precision': 0.7351398601398601, 'recall': 0.7896713615023474, 'f1': 0.7614305115436849, 'number': 1065}	0.6836	0.7566	0.7183	0.7992
0.4391	8.0	80	0.6590	{'precision': 0.6809623430962343, 'recall': 0.8046971569839307, 'f1': 0.7376770538243627, 'number': 809}	{'precision': 0.2641509433962264, 'recall': 0.23529411764705882, 'f1': 0.24888888888888888, 'number': 119}	{'precision': 0.7585324232081911, 'recall': 0.8347417840375587, 'f1': 0.7948144836835047, 'number': 1065}	0.7019	0.7868	0.7419	0.8042
0.3834	9.0	90	0.6569	{'precision': 0.7043189368770764, 'recall': 0.7861557478368356, 'f1': 0.7429906542056073, 'number': 809}	{'precision': 0.2619047619047619, 'recall': 0.2773109243697479, 'f1': 0.2693877551020408, 'number': 119}	{'precision': 0.7637457044673539, 'recall': 0.8347417840375587, 'f1': 0.7976671152983401, 'number': 1065}	0.7104	0.7817	0.7444	0.8099
0.3489	10.0	100	0.6655	{'precision': 0.6984649122807017, 'recall': 0.7873918417799752, 'f1': 0.7402672864613596, 'number': 809}	{'precision': 0.2714285714285714, 'recall': 0.31932773109243695, 'f1': 0.29343629343629346, 'number': 119}	{'precision': 0.7745614035087719, 'recall': 0.8291079812206573, 'f1': 0.800907029478458, 'number': 1065}	0.7108	0.7817	0.7446	0.8126
0.3103	11.0	110	0.6682	{'precision': 0.6981934112646121, 'recall': 0.8121137206427689, 'f1': 0.7508571428571429, 'number': 809}	{'precision': 0.3173076923076923, 'recall': 0.2773109243697479, 'f1': 0.29596412556053814, 'number': 119}	{'precision': 0.7878521126760564, 'recall': 0.8403755868544601, 'f1': 0.8132666969559291, 'number': 1065}	0.7267	0.7953	0.7595	0.8148
0.293	12.0	120	0.6739	{'precision': 0.7123893805309734, 'recall': 0.796044499381953, 'f1': 0.7518972562755399, 'number': 809}	{'precision': 0.328, 'recall': 0.3445378151260504, 'f1': 0.33606557377049184, 'number': 119}	{'precision': 0.7863475177304965, 'recall': 0.8328638497652582, 'f1': 0.8089375284997721, 'number': 1065}	0.7288	0.7888	0.7576	0.8167
0.2761	13.0	130	0.6783	{'precision': 0.705945945945946, 'recall': 0.8071693448702101, 'f1': 0.7531718569780853, 'number': 809}	{'precision': 0.3467741935483871, 'recall': 0.36134453781512604, 'f1': 0.35390946502057613, 'number': 119}	{'precision': 0.7935656836461126, 'recall': 0.8338028169014085, 'f1': 0.8131868131868133, 'number': 1065}	0.7306	0.7948	0.7614	0.8137
0.2633	14.0	140	0.6849	{'precision': 0.7085590465872156, 'recall': 0.8084054388133498, 'f1': 0.7551963048498845, 'number': 809}	{'precision': 0.31746031746031744, 'recall': 0.33613445378151263, 'f1': 0.32653061224489793, 'number': 119}	{'precision': 0.7883082373782108, 'recall': 0.8356807511737089, 'f1': 0.8113035551504102, 'number': 1065}	0.7273	0.7948	0.7595	0.8125
0.2632	15.0	150	0.6847	{'precision': 0.7144432194046306, 'recall': 0.8009888751545118, 'f1': 0.7552447552447553, 'number': 809}	{'precision': 0.30952380952380953, 'recall': 0.3277310924369748, 'f1': 0.31836734693877555, 'number': 119}	{'precision': 0.7912966252220248, 'recall': 0.8366197183098592, 'f1': 0.81332724783204, 'number': 1065}	0.7309	0.7918	0.7601	0.8132

Framework versions

• Transformers 4.30.2
• Pytorch 2.0.1+cu118
• Datasets 2.13.1
• Tokenizers 0.13.3