layoutlm-funsd

This model is a fine-tuned version of microsoft/layoutlm-base-uncased on the funsd dataset. It achieves the following results on the evaluation set:

• Loss: 0.6940
• Answer: {'precision': 0.721978021978022, 'recall': 0.8121137206427689, 'f1': 0.7643979057591623, 'number': 809}
• Header: {'precision': 0.2662337662337662, 'recall': 0.3445378151260504, 'f1': 0.30036630036630035, 'number': 119}
• Question: {'precision': 0.7816091954022989, 'recall': 0.8300469483568075, 'f1': 0.8051001821493625, 'number': 1065}
• Overall Precision: 0.7207
• Overall Recall: 0.7938
• Overall F1: 0.7555
• Overall Accuracy: 0.8073

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 3e-05
• train_batch_size: 16
• eval_batch_size: 8
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 15

Training results

Training Loss	Epoch	Step	Validation Loss	Answer	Header	Question	Overall Precision	Overall Recall	Overall F1	Overall Accuracy
1.755	1.0	10	1.5815	{'precision': 0.026919242273180457, 'recall': 0.03337453646477132, 'f1': 0.02980132450331126, 'number': 809}	{'precision': 0.0, 'recall': 0.0, 'f1': 0.0, 'number': 119}	{'precision': 0.20780487804878048, 'recall': 0.2, 'f1': 0.20382775119617225, 'number': 1065}	0.1183	0.1204	0.1194	0.3885
1.4375	2.0	20	1.2088	{'precision': 0.28227848101265823, 'recall': 0.27564894932014833, 'f1': 0.2789243277048155, 'number': 809}	{'precision': 0.0, 'recall': 0.0, 'f1': 0.0, 'number': 119}	{'precision': 0.4782964782964783, 'recall': 0.5483568075117371, 'f1': 0.5109361329833771, 'number': 1065}	0.4013	0.4049	0.4031	0.6223
1.0595	3.0	30	0.9379	{'precision': 0.503954802259887, 'recall': 0.5512978986402967, 'f1': 0.526564344746163, 'number': 809}	{'precision': 0.0425531914893617, 'recall': 0.01680672268907563, 'f1': 0.024096385542168672, 'number': 119}	{'precision': 0.6126205083260298, 'recall': 0.6563380281690141, 'f1': 0.6337262012692656, 'number': 1065}	0.5533	0.5755	0.5642	0.7194
0.8139	4.0	40	0.7735	{'precision': 0.6280041797283177, 'recall': 0.7428924598269468, 'f1': 0.680634201585504, 'number': 809}	{'precision': 0.13432835820895522, 'recall': 0.07563025210084033, 'f1': 0.09677419354838708, 'number': 119}	{'precision': 0.6600688468158348, 'recall': 0.72018779342723, 'f1': 0.688819039066008, 'number': 1065}	0.6299	0.6909	0.6590	0.7636
0.664	5.0	50	0.7245	{'precision': 0.6519453207150369, 'recall': 0.7663782447466008, 'f1': 0.7045454545454546, 'number': 809}	{'precision': 0.24719101123595505, 'recall': 0.18487394957983194, 'f1': 0.21153846153846156, 'number': 119}	{'precision': 0.7090909090909091, 'recall': 0.7690140845070422, 'f1': 0.7378378378378379, 'number': 1065}	0.6656	0.7331	0.6977	0.7757
0.5505	6.0	60	0.6956	{'precision': 0.6834061135371179, 'recall': 0.7737948084054388, 'f1': 0.7257971014492753, 'number': 809}	{'precision': 0.28205128205128205, 'recall': 0.18487394957983194, 'f1': 0.2233502538071066, 'number': 119}	{'precision': 0.723421926910299, 'recall': 0.8178403755868544, 'f1': 0.7677390921110622, 'number': 1065}	0.6911	0.7622	0.7249	0.7888
0.4759	7.0	70	0.6712	{'precision': 0.6844396082698585, 'recall': 0.7775030902348579, 'f1': 0.7280092592592592, 'number': 809}	{'precision': 0.2727272727272727, 'recall': 0.2773109243697479, 'f1': 0.27499999999999997, 'number': 119}	{'precision': 0.7472527472527473, 'recall': 0.8300469483568075, 'f1': 0.786476868327402, 'number': 1065}	0.6955	0.7757	0.7334	0.7975
0.4276	8.0	80	0.6765	{'precision': 0.6889375684556407, 'recall': 0.7775030902348579, 'f1': 0.7305458768873403, 'number': 809}	{'precision': 0.28205128205128205, 'recall': 0.2773109243697479, 'f1': 0.2796610169491525, 'number': 119}	{'precision': 0.7527333894028595, 'recall': 0.8403755868544601, 'f1': 0.7941437444543035, 'number': 1065}	0.7017	0.7812	0.7393	0.8021
0.3788	9.0	90	0.6653	{'precision': 0.7081930415263749, 'recall': 0.7799752781211372, 'f1': 0.7423529411764707, 'number': 809}	{'precision': 0.2647058823529412, 'recall': 0.3025210084033613, 'f1': 0.2823529411764706, 'number': 119}	{'precision': 0.7667238421955404, 'recall': 0.8394366197183099, 'f1': 0.8014343343792021, 'number': 1065}	0.7118	0.7832	0.7458	0.8049
0.3466	10.0	100	0.6838	{'precision': 0.7005464480874317, 'recall': 0.792336217552534, 'f1': 0.7436194895591649, 'number': 809}	{'precision': 0.2706766917293233, 'recall': 0.3025210084033613, 'f1': 0.28571428571428564, 'number': 119}	{'precision': 0.7728055077452668, 'recall': 0.8431924882629108, 'f1': 0.8064660978895375, 'number': 1065}	0.7127	0.7903	0.7495	0.8047
0.3142	11.0	110	0.6795	{'precision': 0.6997816593886463, 'recall': 0.792336217552534, 'f1': 0.7431884057971013, 'number': 809}	{'precision': 0.2857142857142857, 'recall': 0.3025210084033613, 'f1': 0.2938775510204082, 'number': 119}	{'precision': 0.7994628469113697, 'recall': 0.8384976525821596, 'f1': 0.8185151237396883, 'number': 1065}	0.7272	0.7878	0.7563	0.8067
0.2978	12.0	120	0.6922	{'precision': 0.6927194860813705, 'recall': 0.799752781211372, 'f1': 0.7423981640849111, 'number': 809}	{'precision': 0.2585034013605442, 'recall': 0.31932773109243695, 'f1': 0.2857142857142857, 'number': 119}	{'precision': 0.7768090671316478, 'recall': 0.8366197183098592, 'f1': 0.8056057866184448, 'number': 1065}	0.7074	0.7908	0.7467	0.8026
0.2824	13.0	130	0.6960	{'precision': 0.7184357541899441, 'recall': 0.7948084054388134, 'f1': 0.754694835680751, 'number': 809}	{'precision': 0.2611464968152866, 'recall': 0.3445378151260504, 'f1': 0.2971014492753623, 'number': 119}	{'precision': 0.7757255936675461, 'recall': 0.828169014084507, 'f1': 0.8010899182561309, 'number': 1065}	0.7154	0.7858	0.7489	0.8045
0.2696	14.0	140	0.6917	{'precision': 0.7164667393675027, 'recall': 0.8121137206427689, 'f1': 0.7612977983777521, 'number': 809}	{'precision': 0.2708333333333333, 'recall': 0.3277310924369748, 'f1': 0.2965779467680608, 'number': 119}	{'precision': 0.7833775419982316, 'recall': 0.831924882629108, 'f1': 0.8069216757741348, 'number': 1065}	0.7217	0.7938	0.7560	0.8067
0.2674	15.0	150	0.6940	{'precision': 0.721978021978022, 'recall': 0.8121137206427689, 'f1': 0.7643979057591623, 'number': 809}	{'precision': 0.2662337662337662, 'recall': 0.3445378151260504, 'f1': 0.30036630036630035, 'number': 119}	{'precision': 0.7816091954022989, 'recall': 0.8300469483568075, 'f1': 0.8051001821493625, 'number': 1065}	0.7207	0.7938	0.7555	0.8073

Framework versions

• Transformers 4.30.2
• Pytorch 2.0.1+cu118
• Datasets 2.13.1
• Tokenizers 0.13.3