layoutlm-funsd

This model is a fine-tuned version of microsoft/layoutlm-base-uncased on the funsd dataset. It achieves the following results on the evaluation set:

• Loss: 0.6832
• Answer: {'precision': 0.7075575027382256, 'recall': 0.7985166872682324, 'f1': 0.7502903600464577, 'number': 809}
• Header: {'precision': 0.31932773109243695, 'recall': 0.31932773109243695, 'f1': 0.31932773109243695, 'number': 119}
• Question: {'precision': 0.784366576819407, 'recall': 0.819718309859155, 'f1': 0.8016528925619835, 'number': 1065}
• Overall Precision: 0.7259
• Overall Recall: 0.7812
• Overall F1: 0.7525
• Overall Accuracy: 0.8047

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 3e-05
• train_batch_size: 16
• eval_batch_size: 8
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 15
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Answer	Header	Question	Overall Precision	Overall Recall	Overall F1	Overall Accuracy
1.8109	1.0	10	1.6140	{'precision': 0.013268998793727383, 'recall': 0.013597033374536464, 'f1': 0.013431013431013432, 'number': 809}	{'precision': 0.0, 'recall': 0.0, 'f1': 0.0, 'number': 119}	{'precision': 0.10519645120405577, 'recall': 0.07793427230046948, 'f1': 0.08953613807982741, 'number': 1065}	0.0581	0.0472	0.0521	0.3634
1.468	2.0	20	1.2385	{'precision': 0.13105413105413105, 'recall': 0.11372064276885044, 'f1': 0.12177365982792852, 'number': 809}	{'precision': 0.0, 'recall': 0.0, 'f1': 0.0, 'number': 119}	{'precision': 0.4378029079159935, 'recall': 0.5089201877934272, 'f1': 0.4706904038211029, 'number': 1065}	0.3268	0.3181	0.3224	0.5793
1.0973	3.0	30	0.9328	{'precision': 0.41563275434243174, 'recall': 0.41409147095179233, 'f1': 0.41486068111455104, 'number': 809}	{'precision': 0.03125, 'recall': 0.008403361344537815, 'f1': 0.013245033112582781, 'number': 119}	{'precision': 0.6483720930232558, 'recall': 0.6544600938967137, 'f1': 0.6514018691588785, 'number': 1065}	0.5400	0.5183	0.5289	0.7016
0.8233	4.0	40	0.7582	{'precision': 0.6106290672451193, 'recall': 0.695920889987639, 'f1': 0.6504910456383594, 'number': 809}	{'precision': 0.17543859649122806, 'recall': 0.08403361344537816, 'f1': 0.11363636363636363, 'number': 119}	{'precision': 0.6941176470588235, 'recall': 0.72018779342723, 'f1': 0.7069124423963133, 'number': 1065}	0.6430	0.6724	0.6573	0.7595
0.6573	5.0	50	0.6894	{'precision': 0.6411332633788038, 'recall': 0.7552533992583437, 'f1': 0.6935300794551646, 'number': 809}	{'precision': 0.2696629213483146, 'recall': 0.20168067226890757, 'f1': 0.23076923076923078, 'number': 119}	{'precision': 0.7267080745341615, 'recall': 0.7690140845070422, 'f1': 0.7472627737226277, 'number': 1065}	0.6704	0.7296	0.6987	0.7838
0.5504	6.0	60	0.6623	{'precision': 0.6652675760755509, 'recall': 0.7836835599505563, 'f1': 0.7196367763904654, 'number': 809}	{'precision': 0.19736842105263158, 'recall': 0.12605042016806722, 'f1': 0.15384615384615385, 'number': 119}	{'precision': 0.731626754748142, 'recall': 0.831924882629108, 'f1': 0.7785588752196836, 'number': 1065}	0.6853	0.7702	0.7253	0.7933
0.4731	7.0	70	0.6464	{'precision': 0.6681127982646421, 'recall': 0.761433868974042, 'f1': 0.7117273252455227, 'number': 809}	{'precision': 0.22641509433962265, 'recall': 0.20168067226890757, 'f1': 0.21333333333333335, 'number': 119}	{'precision': 0.7656794425087108, 'recall': 0.8253521126760563, 'f1': 0.7943967464979665, 'number': 1065}	0.6981	0.7622	0.7287	0.8003
0.428	8.0	80	0.6407	{'precision': 0.6865671641791045, 'recall': 0.796044499381953, 'f1': 0.7372638809387521, 'number': 809}	{'precision': 0.22321428571428573, 'recall': 0.21008403361344538, 'f1': 0.21645021645021645, 'number': 119}	{'precision': 0.7692307692307693, 'recall': 0.8262910798122066, 'f1': 0.7967406066093254, 'number': 1065}	0.7060	0.7772	0.7399	0.8053
0.3776	9.0	90	0.6475	{'precision': 0.7108843537414966, 'recall': 0.7750309023485785, 'f1': 0.7415730337078651, 'number': 809}	{'precision': 0.23770491803278687, 'recall': 0.24369747899159663, 'f1': 0.24066390041493776, 'number': 119}	{'precision': 0.7615780445969125, 'recall': 0.8338028169014085, 'f1': 0.796055580457194, 'number': 1065}	0.7115	0.7747	0.7418	0.8022
0.3434	10.0	100	0.6694	{'precision': 0.6895074946466809, 'recall': 0.796044499381953, 'f1': 0.7389558232931727, 'number': 809}	{'precision': 0.2831858407079646, 'recall': 0.2689075630252101, 'f1': 0.27586206896551724, 'number': 119}	{'precision': 0.7693661971830986, 'recall': 0.8206572769953052, 'f1': 0.79418446160836, 'number': 1065}	0.7100	0.7777	0.7423	0.8007
0.3082	11.0	110	0.6749	{'precision': 0.6961206896551724, 'recall': 0.7985166872682324, 'f1': 0.7438111686816349, 'number': 809}	{'precision': 0.2905982905982906, 'recall': 0.2857142857142857, 'f1': 0.288135593220339, 'number': 119}	{'precision': 0.7794779477947795, 'recall': 0.8131455399061033, 'f1': 0.7959558823529411, 'number': 1065}	0.7171	0.7757	0.7452	0.7985
0.2933	12.0	120	0.6635	{'precision': 0.7130242825607064, 'recall': 0.7985166872682324, 'f1': 0.7533527696793003, 'number': 809}	{'precision': 0.28448275862068967, 'recall': 0.2773109243697479, 'f1': 0.28085106382978725, 'number': 119}	{'precision': 0.78125, 'recall': 0.8215962441314554, 'f1': 0.8009153318077803, 'number': 1065}	0.7255	0.7797	0.7516	0.8056
0.278	13.0	130	0.6760	{'precision': 0.7122381477398015, 'recall': 0.7985166872682324, 'f1': 0.752913752913753, 'number': 809}	{'precision': 0.3170731707317073, 'recall': 0.3277310924369748, 'f1': 0.32231404958677684, 'number': 119}	{'precision': 0.7897111913357401, 'recall': 0.8215962441314554, 'f1': 0.8053382420616658, 'number': 1065}	0.7297	0.7827	0.7553	0.8049
0.2699	14.0	140	0.6824	{'precision': 0.7041484716157205, 'recall': 0.7972805933250927, 'f1': 0.7478260869565218, 'number': 809}	{'precision': 0.3305785123966942, 'recall': 0.33613445378151263, 'f1': 0.33333333333333337, 'number': 119}	{'precision': 0.7845601436265709, 'recall': 0.8206572769953052, 'f1': 0.8022028453419, 'number': 1065}	0.7248	0.7822	0.7524	0.8045
0.2645	15.0	150	0.6832	{'precision': 0.7075575027382256, 'recall': 0.7985166872682324, 'f1': 0.7502903600464577, 'number': 809}	{'precision': 0.31932773109243695, 'recall': 0.31932773109243695, 'f1': 0.31932773109243695, 'number': 119}	{'precision': 0.784366576819407, 'recall': 0.819718309859155, 'f1': 0.8016528925619835, 'number': 1065}	0.7259	0.7812	0.7525	0.8047

Framework versions

• Transformers 4.21.3
• Pytorch 1.12.1+cu102
• Datasets 2.4.0
• Tokenizers 0.12.1