README.md

metadata

license: mit
base_model: microsoft/layoutlm-base-uncased
tags:
  - generated_from_trainer
model-index:
  - name: layoutlm-funsd
    results: []

layoutlm-funsd

This model is a fine-tuned version of microsoft/layoutlm-base-uncased on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 1.1105
• Answer: {'precision': 0.7390326209223848, 'recall': 0.8121137206427689, 'f1': 0.773851590106007, 'number': 809}
• Header: {'precision': 0.4206896551724138, 'recall': 0.5126050420168067, 'f1': 0.46212121212121215, 'number': 119}
• Question: {'precision': 0.8176795580110497, 'recall': 0.8338028169014085, 'f1': 0.8256624825662483, 'number': 1065}
• Overall Precision: 0.7575
• Overall Recall: 0.8058
• Overall F1: 0.7809
• Overall Accuracy: 0.8108

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 3e-05
• train_batch_size: 16
• eval_batch_size: 8
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 50
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Answer	Header	Question	Overall Precision	Overall Recall	Overall F1	Overall Accuracy
1.7888	1.0	10	1.5431	{'precision': 0.023681377825618945, 'recall': 0.027194066749072928, 'f1': 0.02531645569620253, 'number': 809}	{'precision': 0.0, 'recall': 0.0, 'f1': 0.0, 'number': 119}	{'precision': 0.23876404494382023, 'recall': 0.1596244131455399, 'f1': 0.19133370849746764, 'number': 1065}	0.1170	0.0963	0.1057	0.3970
1.3827	2.0	20	1.1583	{'precision': 0.2289156626506024, 'recall': 0.21137206427688504, 'f1': 0.21979434447300772, 'number': 809}	{'precision': 0.0, 'recall': 0.0, 'f1': 0.0, 'number': 119}	{'precision': 0.46695652173913044, 'recall': 0.504225352112676, 'f1': 0.48487584650112864, 'number': 1065}	0.3732	0.3552	0.3640	0.6072
1.0226	3.0	30	0.8747	{'precision': 0.521594684385382, 'recall': 0.5822002472187886, 'f1': 0.5502336448598131, 'number': 809}	{'precision': 0.0, 'recall': 0.0, 'f1': 0.0, 'number': 119}	{'precision': 0.6095238095238096, 'recall': 0.6610328638497652, 'f1': 0.6342342342342343, 'number': 1065}	0.5595	0.5896	0.5742	0.7279
0.7648	4.0	40	0.7658	{'precision': 0.5875706214689266, 'recall': 0.7713226205191595, 'f1': 0.667022982362373, 'number': 809}	{'precision': 0.22058823529411764, 'recall': 0.12605042016806722, 'f1': 0.16042780748663102, 'number': 119}	{'precision': 0.6698357821953328, 'recall': 0.7276995305164319, 'f1': 0.6975697569756976, 'number': 1065}	0.6183	0.7095	0.6607	0.7635
0.6327	5.0	50	0.6972	{'precision': 0.6389473684210526, 'recall': 0.7503090234857849, 'f1': 0.6901648664013643, 'number': 809}	{'precision': 0.3448275862068966, 'recall': 0.16806722689075632, 'f1': 0.22598870056497172, 'number': 119}	{'precision': 0.7047619047619048, 'recall': 0.7643192488262911, 'f1': 0.7333333333333334, 'number': 1065}	0.6662	0.7230	0.6935	0.7853
0.5253	6.0	60	0.6675	{'precision': 0.667027027027027, 'recall': 0.7626699629171817, 'f1': 0.7116493656286043, 'number': 809}	{'precision': 0.2248062015503876, 'recall': 0.24369747899159663, 'f1': 0.2338709677419355, 'number': 119}	{'precision': 0.6939586645468998, 'recall': 0.819718309859155, 'f1': 0.751614291863969, 'number': 1065}	0.6570	0.7622	0.7057	0.7984
0.4339	7.0	70	0.6615	{'precision': 0.7005524861878453, 'recall': 0.7836835599505563, 'f1': 0.7397899649941657, 'number': 809}	{'precision': 0.24342105263157895, 'recall': 0.31092436974789917, 'f1': 0.27306273062730624, 'number': 119}	{'precision': 0.7237386269644334, 'recall': 0.8215962441314554, 'f1': 0.7695690413368513, 'number': 1065}	0.6823	0.7757	0.7260	0.8022
0.3775	8.0	80	0.6598	{'precision': 0.7138009049773756, 'recall': 0.7799752781211372, 'f1': 0.7454223272297696, 'number': 809}	{'precision': 0.27007299270072993, 'recall': 0.31092436974789917, 'f1': 0.2890625, 'number': 119}	{'precision': 0.7389491242702252, 'recall': 0.831924882629108, 'f1': 0.7826855123674912, 'number': 1065}	0.7	0.7797	0.7377	0.8078
0.3219	9.0	90	0.6811	{'precision': 0.7013422818791947, 'recall': 0.7750309023485785, 'f1': 0.7363476218438051, 'number': 809}	{'precision': 0.30158730158730157, 'recall': 0.31932773109243695, 'f1': 0.310204081632653, 'number': 119}	{'precision': 0.7541268462206777, 'recall': 0.8150234741784037, 'f1': 0.7833935018050542, 'number': 1065}	0.7061	0.7692	0.7363	0.8004
0.2732	10.0	100	0.6725	{'precision': 0.7259668508287292, 'recall': 0.8121137206427689, 'f1': 0.766627771295216, 'number': 809}	{'precision': 0.3053435114503817, 'recall': 0.33613445378151263, 'f1': 0.32000000000000006, 'number': 119}	{'precision': 0.7798573975044564, 'recall': 0.8215962441314554, 'f1': 0.800182898948331, 'number': 1065}	0.7285	0.7888	0.7574	0.8113
0.2275	11.0	110	0.7076	{'precision': 0.7237569060773481, 'recall': 0.8096415327564895, 'f1': 0.764294049008168, 'number': 809}	{'precision': 0.3381294964028777, 'recall': 0.3949579831932773, 'f1': 0.3643410852713178, 'number': 119}	{'precision': 0.7820284697508897, 'recall': 0.8253521126760563, 'f1': 0.8031064412973961, 'number': 1065}	0.7292	0.7933	0.7599	0.8109
0.195	12.0	120	0.7281	{'precision': 0.7301231802911534, 'recall': 0.8059332509270705, 'f1': 0.7661574618096357, 'number': 809}	{'precision': 0.3141025641025641, 'recall': 0.4117647058823529, 'f1': 0.3563636363636363, 'number': 119}	{'precision': 0.7923497267759563, 'recall': 0.8169014084507042, 'f1': 0.8044382801664355, 'number': 1065}	0.7317	0.7883	0.7589	0.8085
0.1765	13.0	130	0.7390	{'precision': 0.7318181818181818, 'recall': 0.796044499381953, 'f1': 0.7625814091178211, 'number': 809}	{'precision': 0.358974358974359, 'recall': 0.47058823529411764, 'f1': 0.40727272727272734, 'number': 119}	{'precision': 0.781195079086116, 'recall': 0.8347417840375587, 'f1': 0.8070812528370404, 'number': 1065}	0.7309	0.7973	0.7627	0.8164
0.1463	14.0	140	0.7731	{'precision': 0.7437923250564334, 'recall': 0.8145859085290482, 'f1': 0.7775811209439528, 'number': 809}	{'precision': 0.3368421052631579, 'recall': 0.5378151260504201, 'f1': 0.4142394822006472, 'number': 119}	{'precision': 0.7963800904977375, 'recall': 0.8262910798122066, 'f1': 0.8110599078341013, 'number': 1065}	0.7350	0.8043	0.7681	0.8163
0.1292	15.0	150	0.7745	{'precision': 0.738933030646992, 'recall': 0.8046971569839307, 'f1': 0.770414201183432, 'number': 809}	{'precision': 0.3719512195121951, 'recall': 0.5126050420168067, 'f1': 0.431095406360424, 'number': 119}	{'precision': 0.7852650494159928, 'recall': 0.8206572769953052, 'f1': 0.8025711662075299, 'number': 1065}	0.7349	0.7958	0.7642	0.8111
0.1209	16.0	160	0.8221	{'precision': 0.7181719260065288, 'recall': 0.8158220024721878, 'f1': 0.7638888888888888, 'number': 809}	{'precision': 0.3333333333333333, 'recall': 0.4789915966386555, 'f1': 0.393103448275862, 'number': 119}	{'precision': 0.7963302752293578, 'recall': 0.8150234741784037, 'f1': 0.805568445475638, 'number': 1065}	0.7271	0.7953	0.7596	0.8063
0.1095	17.0	170	0.8437	{'precision': 0.7384441939120632, 'recall': 0.8096415327564895, 'f1': 0.7724056603773586, 'number': 809}	{'precision': 0.3673469387755102, 'recall': 0.453781512605042, 'f1': 0.406015037593985, 'number': 119}	{'precision': 0.7715030408340573, 'recall': 0.8338028169014085, 'f1': 0.8014440433212997, 'number': 1065}	0.7309	0.8013	0.7645	0.8098
0.0888	18.0	180	0.8539	{'precision': 0.7426556991774383, 'recall': 0.7812113720642769, 'f1': 0.7614457831325301, 'number': 809}	{'precision': 0.3716216216216216, 'recall': 0.46218487394957986, 'f1': 0.41198501872659177, 'number': 119}	{'precision': 0.784070796460177, 'recall': 0.831924882629108, 'f1': 0.8072892938496583, 'number': 1065}	0.7388	0.7893	0.7632	0.8063
0.0796	19.0	190	0.8697	{'precision': 0.7249154453213078, 'recall': 0.7948084054388134, 'f1': 0.7582547169811321, 'number': 809}	{'precision': 0.39416058394160586, 'recall': 0.453781512605042, 'f1': 0.421875, 'number': 119}	{'precision': 0.7835232252410167, 'recall': 0.8394366197183099, 'f1': 0.8105167724388033, 'number': 1065}	0.7349	0.7983	0.7653	0.8091
0.0747	20.0	200	0.9098	{'precision': 0.7414965986394558, 'recall': 0.8084054388133498, 'f1': 0.7735068007096393, 'number': 809}	{'precision': 0.33695652173913043, 'recall': 0.5210084033613446, 'f1': 0.40924092409240925, 'number': 119}	{'precision': 0.8124419684308264, 'recall': 0.8215962441314554, 'f1': 0.8169934640522875, 'number': 1065}	0.7424	0.7983	0.7693	0.8074
0.0654	21.0	210	0.9180	{'precision': 0.7411225658648339, 'recall': 0.799752781211372, 'f1': 0.7693222354340071, 'number': 809}	{'precision': 0.3905325443786982, 'recall': 0.5546218487394958, 'f1': 0.4583333333333333, 'number': 119}	{'precision': 0.8036866359447005, 'recall': 0.8187793427230047, 'f1': 0.8111627906976744, 'number': 1065}	0.7452	0.7953	0.7694	0.8082
0.0545	22.0	220	0.9456	{'precision': 0.7479768786127168, 'recall': 0.799752781211372, 'f1': 0.7729988052568698, 'number': 809}	{'precision': 0.34502923976608185, 'recall': 0.4957983193277311, 'f1': 0.4068965517241379, 'number': 119}	{'precision': 0.7932910244786945, 'recall': 0.8215962441314554, 'f1': 0.8071955719557196, 'number': 1065}	0.7391	0.7933	0.7652	0.8064
0.0526	23.0	230	0.9552	{'precision': 0.7383720930232558, 'recall': 0.7849196538936959, 'f1': 0.7609346914319952, 'number': 809}	{'precision': 0.36075949367088606, 'recall': 0.4789915966386555, 'f1': 0.41155234657039713, 'number': 119}	{'precision': 0.7947794779477948, 'recall': 0.8291079812206573, 'f1': 0.8115808823529413, 'number': 1065}	0.7398	0.7903	0.7642	0.8048
0.0465	24.0	240	0.9799	{'precision': 0.7411111111111112, 'recall': 0.8244746600741656, 'f1': 0.780573434757168, 'number': 809}	{'precision': 0.44274809160305345, 'recall': 0.48739495798319327, 'f1': 0.464, 'number': 119}	{'precision': 0.7863475177304965, 'recall': 0.8328638497652582, 'f1': 0.8089375284997721, 'number': 1065}	0.7466	0.8088	0.7765	0.8082
0.0402	25.0	250	0.9705	{'precision': 0.7559591373439274, 'recall': 0.823238566131026, 'f1': 0.7881656804733728, 'number': 809}	{'precision': 0.40559440559440557, 'recall': 0.48739495798319327, 'f1': 0.4427480916030534, 'number': 119}	{'precision': 0.785204991087344, 'recall': 0.8272300469483568, 'f1': 0.8056698673982624, 'number': 1065}	0.7479	0.8053	0.7755	0.8091
0.0426	26.0	260	0.9772	{'precision': 0.7350917431192661, 'recall': 0.792336217552534, 'f1': 0.7626412849494348, 'number': 809}	{'precision': 0.40816326530612246, 'recall': 0.5042016806722689, 'f1': 0.45112781954887216, 'number': 119}	{'precision': 0.8149498632634458, 'recall': 0.8394366197183099, 'f1': 0.8270120259019426, 'number': 1065}	0.7538	0.8003	0.7763	0.8103
0.0377	27.0	270	0.9820	{'precision': 0.7414965986394558, 'recall': 0.8084054388133498, 'f1': 0.7735068007096393, 'number': 809}	{'precision': 0.4405594405594406, 'recall': 0.5294117647058824, 'f1': 0.48091603053435117, 'number': 119}	{'precision': 0.8223866790009251, 'recall': 0.8347417840375587, 'f1': 0.8285181733457596, 'number': 1065}	0.7626	0.8058	0.7836	0.8106
0.031	28.0	280	1.0041	{'precision': 0.734717416378316, 'recall': 0.7873918417799752, 'f1': 0.7601431980906921, 'number': 809}	{'precision': 0.4064516129032258, 'recall': 0.5294117647058824, 'f1': 0.4598540145985402, 'number': 119}	{'precision': 0.8052064631956912, 'recall': 0.8422535211267606, 'f1': 0.8233134465351079, 'number': 1065}	0.7477	0.8013	0.7736	0.8075
0.0322	29.0	290	1.0408	{'precision': 0.7292817679558011, 'recall': 0.8158220024721878, 'f1': 0.7701283547257876, 'number': 809}	{'precision': 0.4027777777777778, 'recall': 0.48739495798319327, 'f1': 0.44106463878326996, 'number': 119}	{'precision': 0.8104693140794224, 'recall': 0.8431924882629108, 'f1': 0.8265071329958583, 'number': 1065}	0.7492	0.8108	0.7788	0.8062
0.0292	30.0	300	1.0359	{'precision': 0.7351598173515982, 'recall': 0.796044499381953, 'f1': 0.7643916913946588, 'number': 809}	{'precision': 0.3727810650887574, 'recall': 0.5294117647058824, 'f1': 0.4375, 'number': 119}	{'precision': 0.8218283582089553, 'recall': 0.8272300469483568, 'f1': 0.8245203556387459, 'number': 1065}	0.7501	0.7968	0.7727	0.8045
0.0259	31.0	310	1.0452	{'precision': 0.7275784753363229, 'recall': 0.8022249690976514, 'f1': 0.7630805408583187, 'number': 809}	{'precision': 0.41732283464566927, 'recall': 0.44537815126050423, 'f1': 0.43089430894308944, 'number': 119}	{'precision': 0.8238532110091743, 'recall': 0.8431924882629108, 'f1': 0.8334106728538283, 'number': 1065}	0.7587	0.8028	0.7801	0.8082
0.0268	32.0	320	1.0653	{'precision': 0.7338618346545867, 'recall': 0.8009888751545118, 'f1': 0.7659574468085106, 'number': 809}	{'precision': 0.39473684210526316, 'recall': 0.5042016806722689, 'f1': 0.4428044280442805, 'number': 119}	{'precision': 0.7909252669039146, 'recall': 0.8347417840375587, 'f1': 0.8122430333485611, 'number': 1065}	0.7397	0.8013	0.7693	0.8040
0.0258	33.0	330	1.0603	{'precision': 0.7429537767756482, 'recall': 0.8145859085290482, 'f1': 0.777122641509434, 'number': 809}	{'precision': 0.3885350318471338, 'recall': 0.5126050420168067, 'f1': 0.4420289855072464, 'number': 119}	{'precision': 0.7969314079422383, 'recall': 0.8291079812206573, 'f1': 0.8127013345605153, 'number': 1065}	0.7449	0.8043	0.7735	0.8077
0.0236	34.0	340	1.0683	{'precision': 0.7353951890034365, 'recall': 0.7935723114956736, 'f1': 0.7633769322235434, 'number': 809}	{'precision': 0.36470588235294116, 'recall': 0.5210084033613446, 'f1': 0.4290657439446367, 'number': 119}	{'precision': 0.8066298342541437, 'recall': 0.8225352112676056, 'f1': 0.814504881450488, 'number': 1065}	0.7421	0.7928	0.7666	0.8057
0.0219	35.0	350	1.0757	{'precision': 0.7254464285714286, 'recall': 0.8034610630407911, 'f1': 0.7624633431085045, 'number': 809}	{'precision': 0.43609022556390975, 'recall': 0.48739495798319327, 'f1': 0.46031746031746035, 'number': 119}	{'precision': 0.7964444444444444, 'recall': 0.8413145539906103, 'f1': 0.8182648401826484, 'number': 1065}	0.7447	0.8048	0.7736	0.8054
0.0229	36.0	360	1.1042	{'precision': 0.7369020501138952, 'recall': 0.799752781211372, 'f1': 0.7670420865441612, 'number': 809}	{'precision': 0.3584905660377358, 'recall': 0.4789915966386555, 'f1': 0.41007194244604317, 'number': 119}	{'precision': 0.809040590405904, 'recall': 0.8234741784037559, 'f1': 0.8161935784085622, 'number': 1065}	0.7454	0.7933	0.7686	0.8040
0.0205	37.0	370	1.0980	{'precision': 0.7352614015572859, 'recall': 0.8170580964153276, 'f1': 0.7740046838407495, 'number': 809}	{'precision': 0.36423841059602646, 'recall': 0.46218487394957986, 'f1': 0.4074074074074074, 'number': 119}	{'precision': 0.8087431693989071, 'recall': 0.8338028169014085, 'f1': 0.8210818307905686, 'number': 1065}	0.7467	0.8048	0.7747	0.8085
0.0194	38.0	380	1.0869	{'precision': 0.740909090909091, 'recall': 0.8059332509270705, 'f1': 0.7720544701006513, 'number': 809}	{'precision': 0.3972602739726027, 'recall': 0.48739495798319327, 'f1': 0.43773584905660373, 'number': 119}	{'precision': 0.7982062780269058, 'recall': 0.8356807511737089, 'f1': 0.8165137614678898, 'number': 1065}	0.7473	0.8028	0.7741	0.8052
0.0217	39.0	390	1.0871	{'precision': 0.7366071428571429, 'recall': 0.8158220024721878, 'f1': 0.7741935483870966, 'number': 809}	{'precision': 0.39864864864864863, 'recall': 0.4957983193277311, 'f1': 0.44194756554307113, 'number': 119}	{'precision': 0.8102189781021898, 'recall': 0.8338028169014085, 'f1': 0.8218417399352151, 'number': 1065}	0.7509	0.8063	0.7776	0.8078
0.0186	40.0	400	1.0944	{'precision': 0.7315436241610739, 'recall': 0.8084054388133498, 'f1': 0.7680563711098063, 'number': 809}	{'precision': 0.41216216216216217, 'recall': 0.5126050420168067, 'f1': 0.4569288389513108, 'number': 119}	{'precision': 0.7991031390134529, 'recall': 0.8366197183098592, 'f1': 0.8174311926605504, 'number': 1065}	0.7446	0.8058	0.7740	0.8069
0.0172	41.0	410	1.0907	{'precision': 0.7425968109339408, 'recall': 0.8059332509270705, 'f1': 0.7729697688203911, 'number': 809}	{'precision': 0.41843971631205673, 'recall': 0.4957983193277311, 'f1': 0.4538461538461538, 'number': 119}	{'precision': 0.8132474701011959, 'recall': 0.8300469483568075, 'f1': 0.8215613382899627, 'number': 1065}	0.7574	0.8003	0.7782	0.8115
0.0163	42.0	420	1.1016	{'precision': 0.7427293064876958, 'recall': 0.8207663782447466, 'f1': 0.7798003523194362, 'number': 809}	{'precision': 0.3945578231292517, 'recall': 0.48739495798319327, 'f1': 0.43609022556390975, 'number': 119}	{'precision': 0.8060109289617486, 'recall': 0.8309859154929577, 'f1': 0.8183079056865464, 'number': 1065}	0.7513	0.8063	0.7778	0.8099
0.0165	43.0	430	1.1055	{'precision': 0.7410714285714286, 'recall': 0.8207663782447466, 'f1': 0.7788856304985337, 'number': 809}	{'precision': 0.39864864864864863, 'recall': 0.4957983193277311, 'f1': 0.44194756554307113, 'number': 119}	{'precision': 0.8113553113553114, 'recall': 0.831924882629108, 'f1': 0.8215113583681039, 'number': 1065}	0.7533	0.8073	0.7794	0.8090
0.0157	44.0	440	1.1047	{'precision': 0.7334826427771557, 'recall': 0.8096415327564895, 'f1': 0.7696827262044654, 'number': 809}	{'precision': 0.4166666666666667, 'recall': 0.5042016806722689, 'f1': 0.4562737642585551, 'number': 119}	{'precision': 0.8155963302752294, 'recall': 0.8347417840375587, 'f1': 0.8250580046403712, 'number': 1065}	0.7541	0.8048	0.7786	0.8103
0.0147	45.0	450	1.1064	{'precision': 0.7329608938547486, 'recall': 0.8108776266996292, 'f1': 0.7699530516431925, 'number': 809}	{'precision': 0.41216216216216217, 'recall': 0.5126050420168067, 'f1': 0.4569288389513108, 'number': 119}	{'precision': 0.8107370336669699, 'recall': 0.8366197183098592, 'f1': 0.8234750462107209, 'number': 1065}	0.7507	0.8068	0.7778	0.8106
0.0136	46.0	460	1.1085	{'precision': 0.7334826427771557, 'recall': 0.8096415327564895, 'f1': 0.7696827262044654, 'number': 809}	{'precision': 0.4206896551724138, 'recall': 0.5126050420168067, 'f1': 0.46212121212121215, 'number': 119}	{'precision': 0.8111313868613139, 'recall': 0.8347417840375587, 'f1': 0.822767237390097, 'number': 1065}	0.7521	0.8053	0.7778	0.8098
0.0168	47.0	470	1.1110	{'precision': 0.7368421052631579, 'recall': 0.8133498145859085, 'f1': 0.7732079905992949, 'number': 809}	{'precision': 0.41496598639455784, 'recall': 0.5126050420168067, 'f1': 0.4586466165413534, 'number': 119}	{'precision': 0.810958904109589, 'recall': 0.8338028169014085, 'f1': 0.8222222222222222, 'number': 1065}	0.7527	0.8063	0.7786	0.8106
0.0137	48.0	480	1.1121	{'precision': 0.7370786516853932, 'recall': 0.8108776266996292, 'f1': 0.7722189523248971, 'number': 809}	{'precision': 0.39869281045751637, 'recall': 0.5126050420168067, 'f1': 0.4485294117647059, 'number': 119}	{'precision': 0.8113553113553114, 'recall': 0.831924882629108, 'f1': 0.8215113583681039, 'number': 1065}	0.7508	0.8043	0.7766	0.8103
0.0137	49.0	490	1.1106	{'precision': 0.7382022471910112, 'recall': 0.8121137206427689, 'f1': 0.7733961153619776, 'number': 809}	{'precision': 0.4178082191780822, 'recall': 0.5126050420168067, 'f1': 0.460377358490566, 'number': 119}	{'precision': 0.8143382352941176, 'recall': 0.831924882629108, 'f1': 0.8230376219228982, 'number': 1065}	0.7552	0.8048	0.7792	0.8104
0.0134	50.0	500	1.1105	{'precision': 0.7390326209223848, 'recall': 0.8121137206427689, 'f1': 0.773851590106007, 'number': 809}	{'precision': 0.4206896551724138, 'recall': 0.5126050420168067, 'f1': 0.46212121212121215, 'number': 119}	{'precision': 0.8176795580110497, 'recall': 0.8338028169014085, 'f1': 0.8256624825662483, 'number': 1065}	0.7575	0.8058	0.7809	0.8108

Framework versions

• Transformers 4.35.2
• Pytorch 2.1.0+cu121
• Tokenizers 0.15.0