layoutlm-funsd

This model is a fine-tuned version of microsoft/layoutlm-base-uncased on the funsd dataset. It achieves the following results on the evaluation set:

• Loss: 0.6909
• Answer: {'precision': 0.7051835853131749, 'recall': 0.8071693448702101, 'f1': 0.7527377521613834, 'number': 809}
• Header: {'precision': 0.3418803418803419, 'recall': 0.33613445378151263, 'f1': 0.3389830508474576, 'number': 119}
• Question: {'precision': 0.7631352282515074, 'recall': 0.831924882629108, 'f1': 0.7960467205750225, 'number': 1065}
• Overall Precision: 0.7164
• Overall Recall: 0.7923
• Overall F1: 0.7524
• Overall Accuracy: 0.8064

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 3e-05
• train_batch_size: 16
• eval_batch_size: 8
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 15

Training results

Training Loss	Epoch	Step	Validation Loss	Answer	Header	Question	Overall Precision	Overall Recall	Overall F1	Overall Accuracy
1.7913	1.0	10	1.5806	{'precision': 0.02405857740585774, 'recall': 0.02843016069221261, 'f1': 0.026062322946175637, 'number': 809}	{'precision': 0.0, 'recall': 0.0, 'f1': 0.0, 'number': 119}	{'precision': 0.17197452229299362, 'recall': 0.15211267605633802, 'f1': 0.16143497757847533, 'number': 1065}	0.0975	0.0928	0.0951	0.3662
1.4607	2.0	20	1.2580	{'precision': 0.22879464285714285, 'recall': 0.25339925834363414, 'f1': 0.2404692082111437, 'number': 809}	{'precision': 0.0, 'recall': 0.0, 'f1': 0.0, 'number': 119}	{'precision': 0.41384499623777277, 'recall': 0.5164319248826291, 'f1': 0.4594820384294069, 'number': 1065}	0.3393	0.3788	0.3580	0.5702
1.104	3.0	30	0.9936	{'precision': 0.4552058111380145, 'recall': 0.4647713226205192, 'f1': 0.4599388379204893, 'number': 809}	{'precision': 0.14705882352941177, 'recall': 0.04201680672268908, 'f1': 0.06535947712418301, 'number': 119}	{'precision': 0.5559471365638766, 'recall': 0.5924882629107981, 'f1': 0.5736363636363637, 'number': 1065}	0.5073	0.5078	0.5075	0.6862
0.8426	4.0	40	0.8075	{'precision': 0.5957918050941307, 'recall': 0.6650185414091471, 'f1': 0.6285046728971962, 'number': 809}	{'precision': 0.3220338983050847, 'recall': 0.15966386554621848, 'f1': 0.21348314606741572, 'number': 119}	{'precision': 0.6645739910313901, 'recall': 0.6957746478873239, 'f1': 0.6798165137614679, 'number': 1065}	0.6249	0.6513	0.6378	0.7554
0.6743	5.0	50	0.7167	{'precision': 0.6370370370370371, 'recall': 0.7441285537700866, 'f1': 0.6864310148232612, 'number': 809}	{'precision': 0.35365853658536583, 'recall': 0.24369747899159663, 'f1': 0.2885572139303482, 'number': 119}	{'precision': 0.6849192100538599, 'recall': 0.7164319248826291, 'f1': 0.700321248279027, 'number': 1065}	0.6511	0.6994	0.6744	0.7781
0.5571	6.0	60	0.6785	{'precision': 0.6492146596858639, 'recall': 0.7663782447466008, 'f1': 0.7029478458049887, 'number': 809}	{'precision': 0.36585365853658536, 'recall': 0.25210084033613445, 'f1': 0.29850746268656714, 'number': 119}	{'precision': 0.6846275752773375, 'recall': 0.8112676056338028, 'f1': 0.742587021916631, 'number': 1065}	0.6585	0.7597	0.7055	0.7929
0.4858	7.0	70	0.6678	{'precision': 0.6611740473738414, 'recall': 0.7935723114956736, 'f1': 0.7213483146067416, 'number': 809}	{'precision': 0.39080459770114945, 'recall': 0.2857142857142857, 'f1': 0.33009708737864074, 'number': 119}	{'precision': 0.7212543554006968, 'recall': 0.7774647887323943, 'f1': 0.7483054676909172, 'number': 1065}	0.6818	0.7546	0.7164	0.7961
0.4397	8.0	80	0.6626	{'precision': 0.6826608505997819, 'recall': 0.7737948084054388, 'f1': 0.7253765932792584, 'number': 809}	{'precision': 0.32673267326732675, 'recall': 0.2773109243697479, 'f1': 0.30000000000000004, 'number': 119}	{'precision': 0.742437337942956, 'recall': 0.8065727699530516, 'f1': 0.7731773177317731, 'number': 1065}	0.6979	0.7617	0.7284	0.8015
0.393	9.0	90	0.6611	{'precision': 0.6856223175965666, 'recall': 0.7898640296662547, 'f1': 0.7340608845491098, 'number': 809}	{'precision': 0.30833333333333335, 'recall': 0.31092436974789917, 'f1': 0.3096234309623431, 'number': 119}	{'precision': 0.7425658453695837, 'recall': 0.8206572769953052, 'f1': 0.7796610169491525, 'number': 1065}	0.6954	0.7777	0.7342	0.8020
0.351	10.0	100	0.6665	{'precision': 0.6994535519125683, 'recall': 0.7911001236093943, 'f1': 0.7424593967517401, 'number': 809}	{'precision': 0.33043478260869563, 'recall': 0.31932773109243695, 'f1': 0.32478632478632474, 'number': 119}	{'precision': 0.7415254237288136, 'recall': 0.8215962441314554, 'f1': 0.7795100222717148, 'number': 1065}	0.7027	0.7792	0.7390	0.8054
0.3187	11.0	110	0.6752	{'precision': 0.6963123644251626, 'recall': 0.7935723114956736, 'f1': 0.7417677642980935, 'number': 809}	{'precision': 0.3275862068965517, 'recall': 0.31932773109243695, 'f1': 0.3234042553191489, 'number': 119}	{'precision': 0.7708516242317822, 'recall': 0.8244131455399061, 'f1': 0.7967332123411976, 'number': 1065}	0.7157	0.7817	0.7472	0.8076
0.3034	12.0	120	0.6826	{'precision': 0.6970998925886144, 'recall': 0.8022249690976514, 'f1': 0.7459770114942528, 'number': 809}	{'precision': 0.3486238532110092, 'recall': 0.31932773109243695, 'f1': 0.3333333333333333, 'number': 119}	{'precision': 0.7675814751286449, 'recall': 0.8403755868544601, 'f1': 0.8023307933662035, 'number': 1065}	0.7171	0.7938	0.7535	0.8080
0.2825	13.0	130	0.6909	{'precision': 0.6901408450704225, 'recall': 0.7873918417799752, 'f1': 0.7355658198614318, 'number': 809}	{'precision': 0.3228346456692913, 'recall': 0.3445378151260504, 'f1': 0.3333333333333333, 'number': 119}	{'precision': 0.7626086956521739, 'recall': 0.8234741784037559, 'f1': 0.7918735891647856, 'number': 1065}	0.7068	0.7802	0.7417	0.8055
0.2745	14.0	140	0.6884	{'precision': 0.7039827771797632, 'recall': 0.8084054388133498, 'f1': 0.7525891829689298, 'number': 809}	{'precision': 0.33620689655172414, 'recall': 0.3277310924369748, 'f1': 0.33191489361702126, 'number': 119}	{'precision': 0.7651122625215889, 'recall': 0.831924882629108, 'f1': 0.7971210076473234, 'number': 1065}	0.7167	0.7923	0.7526	0.8070
0.2711	15.0	150	0.6909	{'precision': 0.7051835853131749, 'recall': 0.8071693448702101, 'f1': 0.7527377521613834, 'number': 809}	{'precision': 0.3418803418803419, 'recall': 0.33613445378151263, 'f1': 0.3389830508474576, 'number': 119}	{'precision': 0.7631352282515074, 'recall': 0.831924882629108, 'f1': 0.7960467205750225, 'number': 1065}	0.7164	0.7923	0.7524	0.8064

Framework versions

• Transformers 4.26.0
• Pytorch 1.12.1
• Datasets 2.9.0
• Tokenizers 0.13.2