layoutlm-funsd

This model is a fine-tuned version of microsoft/layoutlm-base-uncased on the funsd dataset. It achieves the following results on the evaluation set:

• Loss: 1.1017
• Answer: {'precision': 0.40439158279963405, 'recall': 0.546353522867738, 'f1': 0.46477392218717145, 'number': 809}
• Header: {'precision': 0.3368421052631579, 'recall': 0.2689075630252101, 'f1': 0.29906542056074764, 'number': 119}
• Question: {'precision': 0.5619128949615713, 'recall': 0.6178403755868545, 'f1': 0.5885509838998211, 'number': 1065}
• Overall Precision: 0.4799
• Overall Recall: 0.5680
• Overall F1: 0.5202
• Overall Accuracy: 0.6339

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 3e-05
• train_batch_size: 32
• eval_batch_size: 16
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 15

Training results

Training Loss	Epoch	Step	Validation Loss	Answer	Header	Question	Overall Precision	Overall Recall	Overall F1	Overall Accuracy
1.3745	1.0	5	1.1446	{'precision': 0.24631396357328708, 'recall': 0.3510506798516687, 'f1': 0.28950050968399593, 'number': 809}	{'precision': 0.20930232558139536, 'recall': 0.226890756302521, 'f1': 0.21774193548387097, 'number': 119}	{'precision': 0.4135151890886547, 'recall': 0.6262910798122066, 'f1': 0.4981329350261389, 'number': 1065}	0.3378	0.4907	0.4002	0.5475
1.0195	2.0	10	1.0518	{'precision': 0.29006968641114983, 'recall': 0.411619283065513, 'f1': 0.340316811446091, 'number': 809}	{'precision': 0.2815533980582524, 'recall': 0.24369747899159663, 'f1': 0.26126126126126126, 'number': 119}	{'precision': 0.42618741976893454, 'recall': 0.6234741784037559, 'f1': 0.5062905070529927, 'number': 1065}	0.3653	0.5148	0.4273	0.5967
0.8996	3.0	15	1.0952	{'precision': 0.3147887323943662, 'recall': 0.5525339925834364, 'f1': 0.4010767160161508, 'number': 809}	{'precision': 0.25301204819277107, 'recall': 0.17647058823529413, 'f1': 0.20792079207920794, 'number': 119}	{'precision': 0.4714285714285714, 'recall': 0.5267605633802817, 'f1': 0.4975609756097561, 'number': 1065}	0.3821	0.5163	0.4392	0.5831
0.8294	4.0	20	1.0418	{'precision': 0.3429571303587052, 'recall': 0.484548825710754, 'f1': 0.4016393442622951, 'number': 809}	{'precision': 0.32, 'recall': 0.20168067226890757, 'f1': 0.24742268041237112, 'number': 119}	{'precision': 0.49588815789473684, 'recall': 0.5661971830985916, 'f1': 0.5287154756685665, 'number': 1065}	0.4187	0.5113	0.4604	0.6110
0.773	5.0	25	1.0412	{'precision': 0.34150772025431425, 'recall': 0.4647713226205192, 'f1': 0.393717277486911, 'number': 809}	{'precision': 0.2857142857142857, 'recall': 0.20168067226890757, 'f1': 0.23645320197044337, 'number': 119}	{'precision': 0.4541223404255319, 'recall': 0.6413145539906103, 'f1': 0.5317244063838069, 'number': 1065}	0.4028	0.5434	0.4626	0.6114
0.731	6.0	30	1.0832	{'precision': 0.352991452991453, 'recall': 0.5105067985166872, 'f1': 0.4173825164224356, 'number': 809}	{'precision': 0.2708333333333333, 'recall': 0.2184873949579832, 'f1': 0.24186046511627907, 'number': 119}	{'precision': 0.5029686174724343, 'recall': 0.5568075117370892, 'f1': 0.5285204991087344, 'number': 1065}	0.4221	0.5178	0.4651	0.6014
0.6884	7.0	35	1.1304	{'precision': 0.3588709677419355, 'recall': 0.5500618046971569, 'f1': 0.4343582235236701, 'number': 809}	{'precision': 0.36619718309859156, 'recall': 0.2184873949579832, 'f1': 0.2736842105263158, 'number': 119}	{'precision': 0.5510204081632653, 'recall': 0.5577464788732395, 'f1': 0.5543630424638357, 'number': 1065}	0.4458	0.5344	0.4861	0.6078
0.6731	8.0	40	1.0667	{'precision': 0.3651096282173499, 'recall': 0.47342398022249693, 'f1': 0.41227125941872983, 'number': 809}	{'precision': 0.29, 'recall': 0.24369747899159663, 'f1': 0.2648401826484018, 'number': 119}	{'precision': 0.49964912280701756, 'recall': 0.6685446009389672, 'f1': 0.5718875502008032, 'number': 1065}	0.4367	0.5640	0.4922	0.6205
0.6441	9.0	45	1.0893	{'precision': 0.3948576675849403, 'recall': 0.5315203955500618, 'f1': 0.45310853530031614, 'number': 809}	{'precision': 0.3238095238095238, 'recall': 0.2857142857142857, 'f1': 0.30357142857142855, 'number': 119}	{'precision': 0.5439367311072056, 'recall': 0.5812206572769953, 'f1': 0.5619609623241035, 'number': 1065}	0.4644	0.5434	0.5008	0.6241
0.6139	10.0	50	1.0987	{'precision': 0.37037037037037035, 'recall': 0.5562422744128553, 'f1': 0.44466403162055335, 'number': 809}	{'precision': 0.313953488372093, 'recall': 0.226890756302521, 'f1': 0.2634146341463415, 'number': 119}	{'precision': 0.533678756476684, 'recall': 0.5802816901408451, 'f1': 0.5560053981106613, 'number': 1065}	0.4453	0.5494	0.4919	0.6253
0.6007	11.0	55	1.0803	{'precision': 0.40096618357487923, 'recall': 0.5129789864029666, 'f1': 0.45010845986984815, 'number': 809}	{'precision': 0.29591836734693877, 'recall': 0.24369747899159663, 'f1': 0.26728110599078336, 'number': 119}	{'precision': 0.5409054805401112, 'recall': 0.6394366197183099, 'f1': 0.5860585197934596, 'number': 1065}	0.4703	0.5645	0.5131	0.6317
0.5985	12.0	60	1.0997	{'precision': 0.4080846968238691, 'recall': 0.5241038318912238, 'f1': 0.45887445887445893, 'number': 809}	{'precision': 0.31683168316831684, 'recall': 0.2689075630252101, 'f1': 0.29090909090909095, 'number': 119}	{'precision': 0.5536303630363036, 'recall': 0.6300469483568075, 'f1': 0.5893719806763285, 'number': 1065}	0.4792	0.5655	0.5188	0.6323
0.5828	13.0	65	1.0996	{'precision': 0.40275229357798165, 'recall': 0.5426452410383189, 'f1': 0.46234860452869925, 'number': 809}	{'precision': 0.33695652173913043, 'recall': 0.2605042016806723, 'f1': 0.29383886255924174, 'number': 119}	{'precision': 0.5685936151855048, 'recall': 0.6187793427230047, 'f1': 0.5926258992805755, 'number': 1065}	0.4823	0.5665	0.5210	0.6345
0.5656	14.0	70	1.1065	{'precision': 0.40542986425339367, 'recall': 0.553770086526576, 'f1': 0.46812957157784746, 'number': 809}	{'precision': 0.32967032967032966, 'recall': 0.25210084033613445, 'f1': 0.28571428571428575, 'number': 119}	{'precision': 0.5730735163861824, 'recall': 0.6075117370892019, 'f1': 0.5897903372835004, 'number': 1065}	0.4839	0.5645	0.5211	0.6338
0.5625	15.0	75	1.1017	{'precision': 0.40439158279963405, 'recall': 0.546353522867738, 'f1': 0.46477392218717145, 'number': 809}	{'precision': 0.3368421052631579, 'recall': 0.2689075630252101, 'f1': 0.29906542056074764, 'number': 119}	{'precision': 0.5619128949615713, 'recall': 0.6178403755868545, 'f1': 0.5885509838998211, 'number': 1065}	0.4799	0.5680	0.5202	0.6339

Framework versions

• Transformers 4.33.0
• Pytorch 2.0.1+cu118
• Datasets 2.14.5
• Tokenizers 0.13.3