layoutlm-morning

This model is a fine-tuned version of microsoft/layoutlm-base-uncased on the 01_load_data_set dataset. It achieves the following results on the evaluation set:

• Loss: 0.6956
• Answer: {'precision': 0.6905016008537886, 'recall': 0.799752781211372, 'f1': 0.7411225658648338, 'number': 809}
• Header: {'precision': 0.3157894736842105, 'recall': 0.3025210084033613, 'f1': 0.30901287553648066, 'number': 119}
• Question: {'precision': 0.7729296527159395, 'recall': 0.8150234741784037, 'f1': 0.793418647166362, 'number': 1065}
• Overall Precision: 0.7134
• Overall Recall: 0.7782
• Overall F1: 0.7444
• Overall Accuracy: 0.8070

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 3e-05
• train_batch_size: 16
• eval_batch_size: 8
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 15
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Answer	Header	Question	Overall Precision	Overall Recall	Overall F1	Overall Accuracy
1.8333	1.0	10	1.6190	{'precision': 0.0047694753577106515, 'recall': 0.003708281829419036, 'f1': 0.004172461752433936, 'number': 809}	{'precision': 0.0, 'recall': 0.0, 'f1': 0.0, 'number': 119}	{'precision': 0.19169329073482427, 'recall': 0.11267605633802817, 'f1': 0.1419278533412182, 'number': 1065}	0.0980	0.0617	0.0757	0.3380
1.4863	2.0	20	1.2878	{'precision': 0.18296892980437285, 'recall': 0.1965389369592089, 'f1': 0.1895113230035757, 'number': 809}	{'precision': 0.0, 'recall': 0.0, 'f1': 0.0, 'number': 119}	{'precision': 0.42759795570698467, 'recall': 0.47136150234741786, 'f1': 0.4484144707458687, 'number': 1065}	0.3235	0.3317	0.3276	0.5761
1.1342	3.0	30	0.9369	{'precision': 0.4929444967074318, 'recall': 0.6477132262051916, 'f1': 0.5598290598290597, 'number': 809}	{'precision': 0.0, 'recall': 0.0, 'f1': 0.0, 'number': 119}	{'precision': 0.5810372771474879, 'recall': 0.6732394366197183, 'f1': 0.6237494562853414, 'number': 1065}	0.5340	0.6227	0.5749	0.7069
0.8661	4.0	40	0.7953	{'precision': 0.5635864592863677, 'recall': 0.761433868974042, 'f1': 0.6477392218717138, 'number': 809}	{'precision': 0.06779661016949153, 'recall': 0.03361344537815126, 'f1': 0.0449438202247191, 'number': 119}	{'precision': 0.6269035532994924, 'recall': 0.6957746478873239, 'f1': 0.6595460614152203, 'number': 1065}	0.5831	0.6829	0.6291	0.7567
0.6917	5.0	50	0.7124	{'precision': 0.6372045220966084, 'recall': 0.7663782447466008, 'f1': 0.6958473625140292, 'number': 809}	{'precision': 0.19767441860465115, 'recall': 0.14285714285714285, 'f1': 0.16585365853658537, 'number': 119}	{'precision': 0.673469387755102, 'recall': 0.7746478873239436, 'f1': 0.7205240174672489, 'number': 1065}	0.6401	0.7336	0.6837	0.7887
0.591	6.0	60	0.7081	{'precision': 0.6201171875, 'recall': 0.7849196538936959, 'f1': 0.6928532460447353, 'number': 809}	{'precision': 0.11009174311926606, 'recall': 0.10084033613445378, 'f1': 0.10526315789473684, 'number': 119}	{'precision': 0.7084063047285464, 'recall': 0.7596244131455399, 'f1': 0.7331218849116448, 'number': 1065}	0.64	0.7306	0.6823	0.7810
0.515	7.0	70	0.6769	{'precision': 0.665625, 'recall': 0.7898640296662547, 'f1': 0.7224420576596947, 'number': 809}	{'precision': 0.21238938053097345, 'recall': 0.20168067226890757, 'f1': 0.20689655172413793, 'number': 119}	{'precision': 0.730836236933798, 'recall': 0.787793427230047, 'f1': 0.7582467239042024, 'number': 1065}	0.6763	0.7536	0.7129	0.7947
0.4607	8.0	80	0.6785	{'precision': 0.6605128205128206, 'recall': 0.796044499381953, 'f1': 0.7219730941704037, 'number': 809}	{'precision': 0.20967741935483872, 'recall': 0.2184873949579832, 'f1': 0.2139917695473251, 'number': 119}	{'precision': 0.731686541737649, 'recall': 0.8065727699530516, 'f1': 0.7673068334077713, 'number': 1065}	0.6727	0.7672	0.7168	0.7987
0.408	9.0	90	0.6652	{'precision': 0.6800847457627118, 'recall': 0.7935723114956736, 'f1': 0.7324586423274386, 'number': 809}	{'precision': 0.25806451612903225, 'recall': 0.2689075630252101, 'f1': 0.26337448559670784, 'number': 119}	{'precision': 0.7521588946459413, 'recall': 0.8178403755868544, 'f1': 0.783625730994152, 'number': 1065}	0.6941	0.7752	0.7324	0.8068
0.3936	10.0	100	0.6655	{'precision': 0.6747114375655824, 'recall': 0.7948084054388134, 'f1': 0.7298524404086267, 'number': 809}	{'precision': 0.336734693877551, 'recall': 0.2773109243697479, 'f1': 0.30414746543778803, 'number': 119}	{'precision': 0.7747747747747747, 'recall': 0.8075117370892019, 'f1': 0.7908045977011493, 'number': 1065}	0.7108	0.7707	0.7395	0.8122
0.3423	11.0	110	0.6888	{'precision': 0.7035398230088495, 'recall': 0.7861557478368356, 'f1': 0.7425569176882663, 'number': 809}	{'precision': 0.27702702702702703, 'recall': 0.3445378151260504, 'f1': 0.30711610486891383, 'number': 119}	{'precision': 0.7586805555555556, 'recall': 0.8206572769953052, 'f1': 0.7884528642309426, 'number': 1065}	0.7037	0.7782	0.7391	0.8060
0.3293	12.0	120	0.6758	{'precision': 0.679324894514768, 'recall': 0.796044499381953, 'f1': 0.7330677290836654, 'number': 809}	{'precision': 0.3018867924528302, 'recall': 0.2689075630252101, 'f1': 0.28444444444444444, 'number': 119}	{'precision': 0.7754919499105546, 'recall': 0.8140845070422535, 'f1': 0.7943197434722858, 'number': 1065}	0.7104	0.7742	0.7409	0.8123
0.3062	13.0	130	0.6923	{'precision': 0.698051948051948, 'recall': 0.7972805933250927, 'f1': 0.7443739180611655, 'number': 809}	{'precision': 0.2890625, 'recall': 0.31092436974789917, 'f1': 0.29959514170040485, 'number': 119}	{'precision': 0.7679509632224168, 'recall': 0.8234741784037559, 'f1': 0.79474399637517, 'number': 1065}	0.7106	0.7822	0.7447	0.8091
0.2858	14.0	140	0.6966	{'precision': 0.6915688367129136, 'recall': 0.8009888751545118, 'f1': 0.7422680412371134, 'number': 809}	{'precision': 0.32142857142857145, 'recall': 0.3025210084033613, 'f1': 0.3116883116883117, 'number': 119}	{'precision': 0.7813911472448057, 'recall': 0.812206572769953, 'f1': 0.796500920810313, 'number': 1065}	0.7185	0.7772	0.7467	0.8060
0.286	15.0	150	0.6956	{'precision': 0.6905016008537886, 'recall': 0.799752781211372, 'f1': 0.7411225658648338, 'number': 809}	{'precision': 0.3157894736842105, 'recall': 0.3025210084033613, 'f1': 0.30901287553648066, 'number': 119}	{'precision': 0.7729296527159395, 'recall': 0.8150234741784037, 'f1': 0.793418647166362, 'number': 1065}	0.7134	0.7782	0.7444	0.8070

Framework versions

• Transformers 4.45.1
• Pytorch 2.4.1+cu121
• Datasets 3.0.1
• Tokenizers 0.20.0