layoutlm-funsd

This model is a fine-tuned version of microsoft/layoutlm-base-uncased on the funsd dataset. It achieves the following results on the evaluation set:

• Loss: 0.6689
• Answer: {'precision': 0.7029063509149623, 'recall': 0.8071693448702101, 'f1': 0.7514384349827388, 'number': 809}
• Header: {'precision': 0.3412698412698413, 'recall': 0.36134453781512604, 'f1': 0.35102040816326535, 'number': 119}
• Question: {'precision': 0.7777777777777778, 'recall': 0.828169014084507, 'f1': 0.8021828103683492, 'number': 1065}
• Overall Precision: 0.7209
• Overall Recall: 0.7918
• Overall F1: 0.7547
• Overall Accuracy: 0.8158

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 3e-05
• train_batch_size: 16
• eval_batch_size: 8
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 15

Training results

Training Loss	Epoch	Step	Validation Loss	Answer	Header	Question	Overall Precision	Overall Recall	Overall F1	Overall Accuracy
1.8306	1.0	10	1.6060	{'precision': 0.026582278481012658, 'recall': 0.02595797280593325, 'f1': 0.026266416510318948, 'number': 809}	{'precision': 0.0, 'recall': 0.0, 'f1': 0.0, 'number': 119}	{'precision': 0.21528861154446177, 'recall': 0.1295774647887324, 'f1': 0.16178194607268465, 'number': 1065}	0.1111	0.0798	0.0929	0.3733
1.4787	2.0	20	1.2612	{'precision': 0.20019627085377822, 'recall': 0.2521631644004944, 'f1': 0.22319474835886213, 'number': 809}	{'precision': 0.0, 'recall': 0.0, 'f1': 0.0, 'number': 119}	{'precision': 0.419710544452102, 'recall': 0.571830985915493, 'f1': 0.4841017488076311, 'number': 1065}	0.3291	0.4079	0.3643	0.5976
1.1115	3.0	30	0.9517	{'precision': 0.466, 'recall': 0.5760197775030902, 'f1': 0.5152017689331123, 'number': 809}	{'precision': 0.0, 'recall': 0.0, 'f1': 0.0, 'number': 119}	{'precision': 0.5697115384615384, 'recall': 0.6676056338028169, 'f1': 0.6147859922178989, 'number': 1065}	0.5201	0.5906	0.5531	0.6834
0.8531	4.0	40	0.8275	{'precision': 0.5730337078651685, 'recall': 0.7564894932014833, 'f1': 0.65210442194992, 'number': 809}	{'precision': 0.06521739130434782, 'recall': 0.025210084033613446, 'f1': 0.03636363636363636, 'number': 119}	{'precision': 0.6735751295336787, 'recall': 0.7323943661971831, 'f1': 0.7017543859649122, 'number': 1065}	0.6140	0.6999	0.6542	0.7393
0.7059	5.0	50	0.7345	{'precision': 0.6333687566418703, 'recall': 0.7367119901112484, 'f1': 0.6811428571428572, 'number': 809}	{'precision': 0.2, 'recall': 0.14285714285714285, 'f1': 0.16666666666666666, 'number': 119}	{'precision': 0.6966386554621848, 'recall': 0.7784037558685446, 'f1': 0.7352549889135255, 'number': 1065}	0.6507	0.7235	0.6852	0.7712
0.5949	6.0	60	0.6931	{'precision': 0.6376050420168067, 'recall': 0.7503090234857849, 'f1': 0.689381033503691, 'number': 809}	{'precision': 0.20430107526881722, 'recall': 0.15966386554621848, 'f1': 0.1792452830188679, 'number': 119}	{'precision': 0.6931637519872814, 'recall': 0.8187793427230047, 'f1': 0.7507533362031856, 'number': 1065}	0.6505	0.7516	0.6974	0.7836
0.5143	7.0	70	0.6674	{'precision': 0.6688172043010753, 'recall': 0.7688504326328801, 'f1': 0.7153536515238643, 'number': 809}	{'precision': 0.23478260869565218, 'recall': 0.226890756302521, 'f1': 0.23076923076923078, 'number': 119}	{'precision': 0.7146341463414634, 'recall': 0.8253521126760563, 'f1': 0.7660130718954249, 'number': 1065}	0.6716	0.7667	0.7160	0.7933
0.4641	8.0	80	0.6507	{'precision': 0.667016806722689, 'recall': 0.7849196538936959, 'f1': 0.7211811470755252, 'number': 809}	{'precision': 0.3142857142857143, 'recall': 0.2773109243697479, 'f1': 0.29464285714285715, 'number': 119}	{'precision': 0.7347280334728034, 'recall': 0.8244131455399061, 'f1': 0.7769911504424779, 'number': 1065}	0.6865	0.7757	0.7284	0.8029
0.4063	9.0	90	0.6671	{'precision': 0.6574074074074074, 'recall': 0.7898640296662547, 'f1': 0.7175743964065132, 'number': 809}	{'precision': 0.3114754098360656, 'recall': 0.31932773109243695, 'f1': 0.3153526970954357, 'number': 119}	{'precision': 0.747008547008547, 'recall': 0.8206572769953052, 'f1': 0.782102908277405, 'number': 1065}	0.6851	0.7782	0.7287	0.8017
0.3643	10.0	100	0.6603	{'precision': 0.6851063829787234, 'recall': 0.796044499381953, 'f1': 0.7364208118925099, 'number': 809}	{'precision': 0.3669724770642202, 'recall': 0.33613445378151263, 'f1': 0.3508771929824562, 'number': 119}	{'precision': 0.7674825174825175, 'recall': 0.8244131455399061, 'f1': 0.7949298325033951, 'number': 1065}	0.7123	0.7837	0.7463	0.8069
0.3331	11.0	110	0.6691	{'precision': 0.6928879310344828, 'recall': 0.7948084054388134, 'f1': 0.740356937248129, 'number': 809}	{'precision': 0.30158730158730157, 'recall': 0.31932773109243695, 'f1': 0.310204081632653, 'number': 119}	{'precision': 0.7666666666666667, 'recall': 0.8206572769953052, 'f1': 0.7927437641723357, 'number': 1065}	0.7088	0.7802	0.7428	0.8071
0.3193	12.0	120	0.6597	{'precision': 0.6932059447983014, 'recall': 0.8071693448702101, 'f1': 0.7458595088520845, 'number': 809}	{'precision': 0.3416666666666667, 'recall': 0.3445378151260504, 'f1': 0.34309623430962344, 'number': 119}	{'precision': 0.7721739130434783, 'recall': 0.8338028169014085, 'f1': 0.801805869074492, 'number': 1065}	0.7152	0.7938	0.7524	0.8112
0.2972	13.0	130	0.6679	{'precision': 0.7011866235167206, 'recall': 0.8034610630407911, 'f1': 0.7488479262672811, 'number': 809}	{'precision': 0.344, 'recall': 0.36134453781512604, 'f1': 0.3524590163934426, 'number': 119}	{'precision': 0.7716814159292036, 'recall': 0.8187793427230047, 'f1': 0.7945330296127562, 'number': 1065}	0.7172	0.7852	0.7497	0.8145
0.2833	14.0	140	0.6684	{'precision': 0.703023758099352, 'recall': 0.8046971569839307, 'f1': 0.7504322766570604, 'number': 809}	{'precision': 0.3412698412698413, 'recall': 0.36134453781512604, 'f1': 0.35102040816326535, 'number': 119}	{'precision': 0.7769973661106233, 'recall': 0.8309859154929577, 'f1': 0.8030852994555354, 'number': 1065}	0.7207	0.7923	0.7548	0.8163
0.2765	15.0	150	0.6689	{'precision': 0.7029063509149623, 'recall': 0.8071693448702101, 'f1': 0.7514384349827388, 'number': 809}	{'precision': 0.3412698412698413, 'recall': 0.36134453781512604, 'f1': 0.35102040816326535, 'number': 119}	{'precision': 0.7777777777777778, 'recall': 0.828169014084507, 'f1': 0.8021828103683492, 'number': 1065}	0.7209	0.7918	0.7547	0.8158

Framework versions

• Transformers 4.31.0
• Pytorch 2.0.1+cu118
• Datasets 2.14.3
• Tokenizers 0.13.3