nyt-ingredient-tagger-gte-base

This model is a fine-tuned version of thenlper/gte-base on the nyt_ingredients dataset. It achieves the following results on the evaluation set:

• Loss: 0.8715
• Comment: {'precision': 0.6872832369942197, 'recall': 0.8166208791208791, 'f1': 0.7463904582548649, 'number': 7280}
• Name: {'precision': 0.8038948801172652, 'recall': 0.8268360973508507, 'f1': 0.8152041195519455, 'number': 9286}
• Qty: {'precision': 0.9873535676251332, 'recall': 0.9860409465567668, 'f1': 0.9866968205401091, 'number': 7522}
• Range End: {'precision': 0.6285714285714286, 'recall': 0.9361702127659575, 'f1': 0.7521367521367521, 'number': 94}
• Unit: {'precision': 0.9322007236117665, 'recall': 0.9777264477808942, 'f1': 0.9544210017716219, 'number': 6061}
• Overall Precision: 0.8399
• Overall Recall: 0.8946
• Overall F1: 0.8664
• Overall Accuracy: 0.8432

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 32
• eval_batch_size: 32
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 5
• label_smoothing_factor: 0.1

Training results

Training Loss	Epoch	Step	Validation Loss	Comment	Name	Qty	Range End	Unit	Overall Precision	Overall Recall	Overall F1	Overall Accuracy
0.944	0.2	1000	0.9202	{'precision': 0.6120679415607554, 'recall': 0.7544655929721815, 'f1': 0.675847596563709, 'number': 6830}	{'precision': 0.7844396082698586, 'recall': 0.8165137614678899, 'f1': 0.8001553915311616, 'number': 8829}	{'precision': 0.9741153793198403, 'recall': 0.9908963585434174, 'f1': 0.9824342150940777, 'number': 7140}	{'precision': 0.6214285714285714, 'recall': 0.925531914893617, 'f1': 0.7435897435897435, 'number': 94}	{'precision': 0.9131349077968295, 'recall': 0.9861984626135569, 'f1': 0.9482613808163951, 'number': 5724}	0.8069	0.8795	0.8416	0.8220
0.9154	0.4	2000	0.9014	{'precision': 0.6404628414648694, 'recall': 0.7860907759882869, 'f1': 0.7058436863209098, 'number': 6830}	{'precision': 0.7931676338552502, 'recall': 0.8204779703250651, 'f1': 0.8065916935753257, 'number': 8829}	{'precision': 0.9789502838941975, 'recall': 0.9900560224089636, 'f1': 0.9844718334377829, 'number': 7140}	{'precision': 0.5947712418300654, 'recall': 0.9680851063829787, 'f1': 0.7368421052631579, 'number': 94}	{'precision': 0.9199088393293179, 'recall': 0.9872466806429071, 'f1': 0.9523889778377012, 'number': 5724}	0.8193	0.8884	0.8524	0.8306
0.9086	0.59	3000	0.8954	{'precision': 0.6559519604380077, 'recall': 0.8156661786237189, 'f1': 0.7271422045291392, 'number': 6830}	{'precision': 0.7948717948717948, 'recall': 0.8216106014271152, 'f1': 0.8080200501253132, 'number': 8829}	{'precision': 0.975865397876155, 'recall': 0.9910364145658264, 'f1': 0.9833923980265444, 'number': 7140}	{'precision': 0.6376811594202898, 'recall': 0.9361702127659575, 'f1': 0.7586206896551724, 'number': 94}	{'precision': 0.9211516440372975, 'recall': 0.9837526205450734, 'f1': 0.9514235025766664, 'number': 5724}	0.8232	0.8953	0.8577	0.8333
0.8962	0.79	4000	0.8914	{'precision': 0.6682174338883448, 'recall': 0.799121522693997, 'f1': 0.7278303773836512, 'number': 6830}	{'precision': 0.7972380534853135, 'recall': 0.8238758636312153, 'f1': 0.8103381050520805, 'number': 8829}	{'precision': 0.9793685959567987, 'recall': 0.9906162464985995, 'f1': 0.984960311934271, 'number': 7140}	{'precision': 0.6484375, 'recall': 0.8829787234042553, 'f1': 0.7477477477477479, 'number': 94}	{'precision': 0.9248070924314562, 'recall': 0.9841020265548568, 'f1': 0.9535336436732967, 'number': 5724}	0.8304	0.8918	0.8600	0.8360
0.8891	0.99	5000	0.8838	{'precision': 0.680119581464873, 'recall': 0.7994143484626647, 'f1': 0.7349575986000808, 'number': 6830}	{'precision': 0.7980727113447219, 'recall': 0.8254615471740854, 'f1': 0.8115361060074606, 'number': 8829}	{'precision': 0.9788352469221192, 'recall': 0.9910364145658264, 'f1': 0.9848980444011414, 'number': 7140}	{'precision': 0.6615384615384615, 'recall': 0.9148936170212766, 'f1': 0.7678571428571428, 'number': 94}	{'precision': 0.9254220619570562, 'recall': 0.9863731656184487, 'f1': 0.9549260042283298, 'number': 5724}	0.8346	0.8930	0.8628	0.8389
0.8641	1.19	6000	0.8874	{'precision': 0.6708014106773683, 'recall': 0.8076134699853587, 'f1': 0.732877167342058, 'number': 6830}	{'precision': 0.8106220201796208, 'recall': 0.8280665987088005, 'f1': 0.8192514567458539, 'number': 8829}	{'precision': 0.9736480922316771, 'recall': 0.9935574229691877, 'f1': 0.9835020102592541, 'number': 7140}	{'precision': 0.625, 'recall': 0.9574468085106383, 'f1': 0.7563025210084033, 'number': 94}	{'precision': 0.9219081849371018, 'recall': 0.9858490566037735, 'f1': 0.9528070915998311, 'number': 5724}	0.8331	0.8965	0.8636	0.8376
0.8716	1.39	7000	0.8784	{'precision': 0.6831488314883148, 'recall': 0.8131771595900439, 'f1': 0.7425133689839571, 'number': 6830}	{'precision': 0.8091653752490591, 'recall': 0.8279533355985955, 'f1': 0.8184515478922914, 'number': 8829}	{'precision': 0.9784470848300636, 'recall': 0.9918767507002801, 'f1': 0.9851161496731118, 'number': 7140}	{'precision': 0.6717557251908397, 'recall': 0.9361702127659575, 'f1': 0.7822222222222222, 'number': 94}	{'precision': 0.9253166639249877, 'recall': 0.9827044025157232, 'f1': 0.9531475048716428, 'number': 5724}	0.8382	0.8966	0.8664	0.8420
0.8613	1.58	8000	0.8823	{'precision': 0.6763180118228979, 'recall': 0.8207906295754026, 'f1': 0.7415834380580726, 'number': 6830}	{'precision': 0.8007234462347912, 'recall': 0.8273870200475705, 'f1': 0.8138368983957219, 'number': 8829}	{'precision': 0.9789793942746509, 'recall': 0.9914565826330533, 'f1': 0.9851784844478463, 'number': 7140}	{'precision': 0.6164383561643836, 'recall': 0.9574468085106383, 'f1': 0.7500000000000001, 'number': 94}	{'precision': 0.9251152073732719, 'recall': 0.9820055904961565, 'f1': 0.9527118644067797, 'number': 5724}	0.8327	0.8981	0.8642	0.8402
0.8744	1.78	9000	0.8788	{'precision': 0.6842490842490843, 'recall': 0.820497803806735, 'f1': 0.7462050599201066, 'number': 6830}	{'precision': 0.8025666337611056, 'recall': 0.8287461773700305, 'f1': 0.8154463390170511, 'number': 8829}	{'precision': 0.9823513062812673, 'recall': 0.9900560224089636, 'f1': 0.9861886160714286, 'number': 7140}	{'precision': 0.6818181818181818, 'recall': 0.9574468085106383, 'f1': 0.7964601769911505, 'number': 94}	{'precision': 0.922613229064842, 'recall': 0.9893431167016072, 'f1': 0.954813690777272, 'number': 5724}	0.8365	0.8996	0.8669	0.8423
0.8644	1.98	10000	0.8753	{'precision': 0.6871112216969395, 'recall': 0.8086383601756955, 'f1': 0.7429378531073446, 'number': 6830}	{'precision': 0.8013368397983782, 'recall': 0.8282931249292106, 'f1': 0.8145920356446671, 'number': 8829}	{'precision': 0.9827490261547023, 'recall': 0.9893557422969188, 'f1': 0.9860413176996092, 'number': 7140}	{'precision': 0.696, 'recall': 0.925531914893617, 'f1': 0.7945205479452053, 'number': 94}	{'precision': 0.9273927392739274, 'recall': 0.9818308874912649, 'f1': 0.9538357094365241, 'number': 5724}	0.8386	0.8948	0.8658	0.8416
0.8374	2.18	11000	0.8823	{'precision': 0.6925305454087417, 'recall': 0.8049780380673499, 'f1': 0.7445324666531248, 'number': 6830}	{'precision': 0.7997152869031976, 'recall': 0.8271604938271605, 'f1': 0.8132063916263014, 'number': 8829}	{'precision': 0.9837115411388, 'recall': 0.9896358543417367, 'f1': 0.9866648048593173, 'number': 7140}	{'precision': 0.6541353383458647, 'recall': 0.925531914893617, 'f1': 0.7665198237885463, 'number': 94}	{'precision': 0.9254637990477754, 'recall': 0.9848008385744235, 'f1': 0.9542107490478207, 'number': 5724}	0.8397	0.8943	0.8661	0.8415
0.8363	2.38	12000	0.8869	{'precision': 0.6901128892196998, 'recall': 0.8144948755490483, 'f1': 0.7471627157343361, 'number': 6830}	{'precision': 0.8069826538504032, 'recall': 0.8272737569373655, 'f1': 0.8170022371364652, 'number': 8829}	{'precision': 0.982363560616581, 'recall': 0.9907563025210084, 'f1': 0.98654208214211, 'number': 7140}	{'precision': 0.6825396825396826, 'recall': 0.9148936170212766, 'f1': 0.7818181818181817, 'number': 94}	{'precision': 0.926649163111257, 'recall': 0.9865478686233403, 'f1': 0.9556608563208665, 'number': 5724}	0.8409	0.8972	0.8681	0.8439
0.8336	2.57	13000	0.8826	{'precision': 0.6855172413793104, 'recall': 0.8004392386530015, 'f1': 0.7385342789598108, 'number': 6830}	{'precision': 0.8088072582429741, 'recall': 0.8279533355985955, 'f1': 0.8182683158896289, 'number': 8829}	{'precision': 0.9803296855520155, 'recall': 0.9911764705882353, 'f1': 0.985723239779929, 'number': 7140}	{'precision': 0.6641221374045801, 'recall': 0.925531914893617, 'f1': 0.7733333333333333, 'number': 94}	{'precision': 0.9256767842493847, 'recall': 0.9856743535988819, 'f1': 0.954733903037482, 'number': 5724}	0.8399	0.8940	0.8661	0.8416
0.8369	2.77	14000	0.8762	{'precision': 0.6956186560452268, 'recall': 0.792679355783309, 'f1': 0.740984055293232, 'number': 6830}	{'precision': 0.8046496253856324, 'recall': 0.8271604938271605, 'f1': 0.8157497905612958, 'number': 8829}	{'precision': 0.980484429065744, 'recall': 0.9921568627450981, 'f1': 0.9862861120779673, 'number': 7140}	{'precision': 0.6666666666666666, 'recall': 0.9361702127659575, 'f1': 0.7787610619469028, 'number': 94}	{'precision': 0.9256225425950196, 'recall': 0.9870719776380154, 'f1': 0.9553601623266824, 'number': 5724}	0.8423	0.8924	0.8667	0.8428
0.8402	2.97	15000	0.8754	{'precision': 0.6976949237939287, 'recall': 0.8109809663250366, 'f1': 0.7500846367391156, 'number': 6830}	{'precision': 0.805531167690957, 'recall': 0.8313512289047458, 'f1': 0.8182375564349813, 'number': 8829}	{'precision': 0.9835905993603115, 'recall': 0.9906162464985995, 'f1': 0.9870909217779638, 'number': 7140}	{'precision': 0.7456140350877193, 'recall': 0.9042553191489362, 'f1': 0.8173076923076923, 'number': 94}	{'precision': 0.9257255287752091, 'recall': 0.9863731656184487, 'f1': 0.9550875412331896, 'number': 5724}	0.8433	0.8975	0.8695	0.8452
0.8036	3.17	16000	0.8847	{'precision': 0.6951158106747231, 'recall': 0.8084919472913616, 'f1': 0.7475294436171653, 'number': 6830}	{'precision': 0.8062272275587943, 'recall': 0.8270472307169555, 'f1': 0.8165045286816504, 'number': 8829}	{'precision': 0.9823660094418217, 'recall': 0.9908963585434174, 'f1': 0.9866127457816204, 'number': 7140}	{'precision': 0.7304347826086957, 'recall': 0.8936170212765957, 'f1': 0.8038277511961722, 'number': 94}	{'precision': 0.9269773547751887, 'recall': 0.9868972746331237, 'f1': 0.9559993230665087, 'number': 5724}	0.8428	0.8957	0.8685	0.8448
0.8037	3.37	17000	0.8834	{'precision': 0.6962266548087918, 'recall': 0.8023426061493412, 'f1': 0.7455275151350249, 'number': 6830}	{'precision': 0.8004607283896445, 'recall': 0.8264809151659305, 'f1': 0.8132627472833659, 'number': 8829}	{'precision': 0.9810328118510314, 'recall': 0.992436974789916, 'f1': 0.9867019424911231, 'number': 7140}	{'precision': 0.7327586206896551, 'recall': 0.9042553191489362, 'f1': 0.8095238095238094, 'number': 94}	{'precision': 0.92678512668641, 'recall': 0.9841020265548568, 'f1': 0.9545839688188443, 'number': 5724}	0.8414	0.8939	0.8668	0.8431
0.8108	3.56	18000	0.8876	{'precision': 0.694210724601281, 'recall': 0.8093704245973645, 'f1': 0.7473805178124789, 'number': 6830}	{'precision': 0.800065803904365, 'recall': 0.8262543889455204, 'f1': 0.8129492394272023, 'number': 8829}	{'precision': 0.9796764827872252, 'recall': 0.992436974789916, 'f1': 0.9860154456272177, 'number': 7140}	{'precision': 0.704, 'recall': 0.9361702127659575, 'f1': 0.8036529680365296, 'number': 94}	{'precision': 0.9266250820748523, 'recall': 0.9861984626135569, 'f1': 0.9554840893703453, 'number': 5724}	0.8399	0.8960	0.8670	0.8441
0.806	3.76	19000	0.8833	{'precision': 0.7025162856048026, 'recall': 0.8052708638360175, 'f1': 0.7503922504945766, 'number': 6830}	{'precision': 0.8099712580145921, 'recall': 0.8298788084720806, 'f1': 0.819804195804196, 'number': 8829}	{'precision': 0.9814353006372957, 'recall': 0.9921568627450981, 'f1': 0.9867669591865162, 'number': 7140}	{'precision': 0.7297297297297297, 'recall': 0.8617021276595744, 'f1': 0.7902439024390245, 'number': 94}	{'precision': 0.9282059745832646, 'recall': 0.9825296995108316, 'f1': 0.9545956038360349, 'number': 5724}	0.8464	0.8951	0.8701	0.8454
0.8118	3.96	20000	0.8805	{'precision': 0.6981322564361434, 'recall': 0.8099560761346999, 'f1': 0.7498983326555511, 'number': 6830}	{'precision': 0.8089974577207915, 'recall': 0.8289727035904406, 'f1': 0.8188632803759232, 'number': 8829}	{'precision': 0.9818282702177833, 'recall': 0.9913165266106443, 'f1': 0.9865495853369572, 'number': 7140}	{'precision': 0.6692307692307692, 'recall': 0.925531914893617, 'f1': 0.7767857142857142, 'number': 94}	{'precision': 0.9277960526315789, 'recall': 0.9854996505939903, 'f1': 0.9557777024737377, 'number': 5724}	0.8443	0.8966	0.8696	0.8433
0.7792	4.16	21000	0.8955	{'precision': 0.6977390425666288, 'recall': 0.8087847730600293, 'f1': 0.7491693225740829, 'number': 6830}	{'precision': 0.8072502210433244, 'recall': 0.8272737569373655, 'f1': 0.8171393410527494, 'number': 8829}	{'precision': 0.981563626282229, 'recall': 0.9917366946778712, 'f1': 0.9866239375783753, 'number': 7140}	{'precision': 0.696, 'recall': 0.925531914893617, 'f1': 0.7945205479452053, 'number': 94}	{'precision': 0.9261447562776958, 'recall': 0.9858490566037735, 'f1': 0.955064737242955, 'number': 5724}	0.8435	0.8959	0.8689	0.8438
0.7844	4.36	22000	0.8965	{'precision': 0.6992586912065439, 'recall': 0.8010248901903367, 'f1': 0.7466903234611709, 'number': 6830}	{'precision': 0.8035242290748899, 'recall': 0.8263676520557255, 'f1': 0.8147858618571668, 'number': 8829}	{'precision': 0.9807559185933823, 'recall': 0.9921568627450981, 'f1': 0.9864234491401519, 'number': 7140}	{'precision': 0.7107438016528925, 'recall': 0.9148936170212766, 'f1': 0.7999999999999999, 'number': 94}	{'precision': 0.9251187940357202, 'recall': 0.9863731656184487, 'f1': 0.9547645218567684, 'number': 5724}	0.8429	0.8940	0.8677	0.8420
0.7783	4.55	23000	0.8986	{'precision': 0.6973415132924335, 'recall': 0.7988286969253294, 'f1': 0.7446431008598335, 'number': 6830}	{'precision': 0.8045444517979263, 'recall': 0.8261411258353154, 'f1': 0.8151997764738753, 'number': 8829}	{'precision': 0.9825169973636743, 'recall': 0.9917366946778712, 'f1': 0.9871053181849865, 'number': 7140}	{'precision': 0.688, 'recall': 0.9148936170212766, 'f1': 0.7853881278538812, 'number': 94}	{'precision': 0.9252948885976409, 'recall': 0.986722571628232, 'f1': 0.9550219817382481, 'number': 5724}	0.8430	0.8934	0.8674	0.8422
0.784	4.75	24000	0.8966	{'precision': 0.6979669631512071, 'recall': 0.8042459736456808, 'f1': 0.7473469387755102, 'number': 6830}	{'precision': 0.8029870415110916, 'recall': 0.8281798618190056, 'f1': 0.8153889043769166, 'number': 8829}	{'precision': 0.9821131447587355, 'recall': 0.9920168067226891, 'f1': 0.9870401337792643, 'number': 7140}	{'precision': 0.6935483870967742, 'recall': 0.9148936170212766, 'f1': 0.7889908256880733, 'number': 94}	{'precision': 0.9265648102513554, 'recall': 0.9853249475890985, 'f1': 0.9550419100838201, 'number': 5724}	0.8426	0.8951	0.8680	0.8443
0.776	4.95	25000	0.8964	{'precision': 0.6977393954787909, 'recall': 0.8043923865300147, 'f1': 0.7472796517954298, 'number': 6830}	{'precision': 0.8060392329733304, 'recall': 0.8284063880394156, 'f1': 0.8170697648438808, 'number': 8829}	{'precision': 0.9822419533851277, 'recall': 0.9915966386554622, 'f1': 0.9868971285196543, 'number': 7140}	{'precision': 0.6991869918699187, 'recall': 0.9148936170212766, 'f1': 0.7926267281105991, 'number': 94}	{'precision': 0.92616899097621, 'recall': 0.9861984626135569, 'f1': 0.9552415601996784, 'number': 5724}	0.8435	0.8952	0.8686	0.8442

Framework versions

• Transformers 4.34.0
• Pytorch 2.0.1+cu118
• Datasets 2.14.5
• Tokenizers 0.14.1