longformer-qmsum-meeting-summarization

This model is a fine-tuned version of allenai/led-base-16384 on the None dataset. It achieves the following results on the evaluation set:

• Loss: 4.2055
• Rouge1: 20.5333
• Rouge2: 7.6756
• Rougel: 16.2531
• Rougelsum: 19.0336
• Gen Len: 20.0

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 3e-07
• train_batch_size: 2
• eval_batch_size: 2
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• lr_scheduler_warmup_steps: 500
• num_epochs: 200
• label_smoothing_factor: 0.1

Training results

Training Loss	Epoch	Step	Validation Loss	Rouge1	Rouge2	Rougel	Rougelsum	Gen Len
5.4071	1.09	100	5.2910	6.012	0.5556	4.936	5.6141	20.0
5.269	2.17	200	5.1446	6.7419	0.9713	5.2774	6.3003	20.0
5.1153	3.26	300	4.9976	8.1369	1.2365	6.391	7.5911	20.0
4.9888	4.35	400	4.8763	9.9113	1.4239	8.0574	9.3442	20.0
4.8687	5.43	500	4.7889	10.504	1.5638	8.1191	9.817	20.0
4.7936	6.52	600	4.7226	12.6475	2.4733	9.968	11.541	20.0
4.713	7.61	700	4.6770	15.2998	3.6209	11.8629	14.2323	20.0
4.6843	8.7	800	4.6428	15.8299	4.4128	12.7301	14.8795	20.0
4.6453	9.78	900	4.6105	16.3702	4.7356	13.1566	15.4497	20.0
4.6212	10.87	1000	4.5849	16.9765	5.1101	13.617	15.9401	20.0
4.5761	11.96	1100	4.5649	17.3024	5.2494	13.79	16.3173	20.0
4.564	13.04	1200	4.5447	18.7699	6.2331	14.8264	17.645	20.0
4.5393	14.13	1300	4.5277	19.1495	6.6082	15.1392	18.2546	20.0
4.5069	15.22	1400	4.5132	20.3648	7.3895	16.018	19.1503	20.0
4.4985	16.3	1500	4.4973	20.165	7.3477	16.1161	18.7585	20.0
4.4476	17.39	1600	4.4859	20.4691	7.5734	16.438	19.1045	20.0
4.4421	18.48	1700	4.4758	20.4402	7.7674	16.3998	19.1045	20.0
4.4554	19.57	1800	4.4648	20.5992	7.3522	16.185	19.2869	20.0
4.4138	20.65	1900	4.4560	20.497	7.1732	16.2177	19.0912	20.0
4.4447	21.74	2000	4.4465	21.2936	7.8856	16.8994	19.7994	20.0
4.3636	22.83	2100	4.4373	21.1015	7.6466	16.787	19.6918	20.0
4.3647	23.91	2200	4.4288	21.3408	7.8052	17.1431	20.1456	20.0
4.3707	25.0	2300	4.4217	21.523	8.017	17.1586	20.2724	20.0
4.3503	26.09	2400	4.4145	21.485	8.015	17.064	20.209	20.0
4.3295	27.17	2500	4.4069	21.5167	7.6749	16.9976	20.265	20.0
4.3444	28.26	2600	4.4004	21.748	7.8808	17.1592	20.4054	20.0
4.3135	29.35	2700	4.3958	21.5523	7.5449	17.2103	20.5405	20.0
4.3028	30.43	2800	4.3880	21.3016	7.6531	17.1515	20.3301	20.0
4.3406	31.52	2900	4.3834	21.4169	7.5647	16.9477	20.3379	20.0
4.286	32.61	3000	4.3760	21.4684	7.4776	17.1018	20.5254	20.0
4.2717	33.7	3100	4.3736	21.596	7.514	17.164	20.6272	20.0
4.285	34.78	3200	4.3666	21.3495	7.676	17.0703	20.3182	20.0
4.2496	35.87	3300	4.3628	21.5539	7.6574	17.1393	20.5116	20.0
4.2618	36.96	3400	4.3591	21.08	7.6814	16.6941	20.2386	20.0
4.255	38.04	3500	4.3522	21.1979	7.7334	16.8281	20.3095	20.0
4.2353	39.13	3600	4.3502	21.1162	8.0427	16.9948	20.3903	20.0
4.2556	40.22	3700	4.3462	21.3417	7.7851	16.6548	20.5316	20.0
4.207	41.3	3800	4.3401	21.4329	7.948	16.944	20.5075	20.0
4.234	42.39	3900	4.3388	21.6109	8.033	16.9375	20.6668	20.0
4.2118	43.48	4000	4.3347	21.5051	7.9239	16.7403	20.6123	20.0
4.1898	44.57	4100	4.3319	21.2644	7.8222	16.7109	20.3999	20.0
4.1951	45.65	4200	4.3265	21.3383	7.997	16.7605	20.4542	20.0
4.1851	46.74	4300	4.3248	21.3509	7.9038	16.9098	20.4593	20.0
4.1674	47.83	4400	4.3223	21.3516	8.0058	17.0061	20.4199	20.0
4.1785	48.91	4500	4.3182	21.4118	8.0755	16.959	20.5154	20.0
4.1599	50.0	4600	4.3175	21.2748	7.8562	16.8107	20.3536	20.0
4.1564	51.09	4700	4.3141	21.1811	7.8563	16.7687	20.2242	20.0
4.1513	52.17	4800	4.3101	21.1557	7.6616	16.8105	19.8191	20.0
4.1234	53.26	4900	4.3083	21.0718	7.8625	16.7849	20.0014	20.0
4.1532	54.35	5000	4.3041	21.4241	7.984	16.6561	20.3073	20.0
4.1371	55.43	5100	4.3035	21.259	7.6476	16.9931	20.3421	20.0
4.1342	56.52	5200	4.3009	21.0745	7.386	16.7976	20.1148	20.0
4.1146	57.61	5300	4.2985	21.0796	7.6743	16.5062	19.8702	20.0
4.0774	58.7	5400	4.2965	21.2129	7.2871	17.0019	20.3176	20.0
4.1726	59.78	5500	4.2930	21.159	7.4045	16.7762	19.9886	20.0
4.0931	60.87	5600	4.2900	20.957	7.2307	16.784	19.8402	20.0
4.0838	61.96	5700	4.2887	21.13	7.2664	16.7837	19.951	20.0
4.0878	63.04	5800	4.2853	21.0281	7.2664	16.6847	19.7843	20.0
4.1067	64.13	5900	4.2848	20.941	7.2307	16.74	19.8262	20.0
4.0743	65.22	6000	4.2817	21.1234	7.4612	16.755	20.027	20.0
4.103	66.3	6100	4.2807	21.2852	7.4802	16.8037	20.2316	20.0
4.0434	67.39	6200	4.2777	21.236	7.3169	16.7967	20.0534	20.0
4.0829	68.48	6300	4.2793	20.947	7.3164	16.8597	19.7938	20.0
4.0619	69.57	6400	4.2736	21.4626	7.7245	16.8395	20.2035	20.0
4.079	70.65	6500	4.2729	21.163	7.6397	16.7826	20.0295	20.0
4.0411	71.74	6600	4.2721	20.8673	7.3841	16.6784	19.6854	20.0
4.046	72.83	6700	4.2697	20.9774	7.3325	16.7779	19.761	20.0
4.0384	73.91	6800	4.2684	21.0736	7.6569	16.7631	19.992	20.0
4.0401	75.0	6900	4.2670	21.2708	7.8224	16.5649	20.2364	20.0
4.0153	76.09	7000	4.2669	21.3638	7.7586	16.765	19.9744	20.0
4.0227	77.17	7100	4.2652	21.0611	7.709	16.3201	20.0516	20.0
4.0264	78.26	7200	4.2634	21.3766	7.7666	16.7508	20.0938	20.0
4.0475	79.35	7300	4.2615	21.2356	7.5533	16.6339	19.9254	20.0
4.0145	80.43	7400	4.2580	20.7689	7.3386	16.287	19.7335	20.0
4.0087	81.52	7500	4.2580	20.9816	7.343	16.4598	19.701	20.0
3.9835	82.61	7600	4.2577	21.1001	7.5887	16.5226	19.714	20.0
4.0029	83.7	7700	4.2562	21.1875	7.7333	16.4799	19.9907	20.0
3.9912	84.78	7800	4.2549	20.8265	7.3897	16.2191	19.4398	20.0
4.008	85.87	7900	4.2541	21.4955	7.7602	16.4989	20.1402	20.0
3.9659	86.96	8000	4.2523	21.687	7.9463	16.5832	20.1598	20.0
3.9923	88.04	8100	4.2505	21.4615	7.817	16.3628	19.9159	20.0
3.9811	89.13	8200	4.2498	21.1917	7.5813	16.3066	19.4905	20.0
3.9819	90.22	8300	4.2488	21.239	7.4585	16.4297	19.5213	20.0
3.9889	91.3	8400	4.2456	21.5052	7.7994	16.3783	19.8739	20.0
3.942	92.39	8500	4.2468	21.3482	7.7517	16.34	19.764	20.0
3.9959	93.48	8600	4.2446	21.4615	7.817	16.3628	19.9159	20.0
3.987	94.57	8700	4.2438	21.1265	7.6497	16.4132	19.5981	20.0
3.9803	95.65	8800	4.2420	21.2956	7.7796	16.3643	19.8607	20.0
3.9415	96.74	8900	4.2410	20.8332	7.5468	16.1678	19.316	20.0
3.97	97.83	9000	4.2407	21.4223	7.8688	16.533	19.8081	20.0
3.9495	98.91	9100	4.2400	21.5678	7.9698	16.5492	19.9404	20.0
3.9489	100.0	9200	4.2391	21.3928	7.8416	16.3595	19.7579	20.0
3.9194	101.09	9300	4.2394	21.2216	7.8416	16.2499	19.5661	20.0
3.966	102.17	9400	4.2372	21.2756	7.8798	16.3124	19.6303	20.0
3.934	103.26	9500	4.2367	21.3106	7.8585	16.3937	19.7289	20.0
3.9316	104.35	9600	4.2349	21.3296	7.9392	16.3574	19.8031	20.0
3.9586	105.43	9700	4.2366	21.0662	7.771	16.2242	19.4813	20.0
3.9189	106.52	9800	4.2338	21.1348	7.8414	16.2757	19.7301	20.0
3.937	107.61	9900	4.2350	21.2434	7.7611	16.4693	19.6923	20.0
3.911	108.7	10000	4.2331	21.2697	7.8282	16.3636	19.6627	20.0
3.8956	109.78	10100	4.2312	21.2697	7.8117	16.3636	19.6321	20.0
3.9396	110.87	10200	4.2303	21.0842	7.7105	16.221	19.4378	20.0
3.9058	111.96	10300	4.2290	21.1633	7.8117	16.3196	19.5575	20.0
3.9198	113.04	10400	4.2278	21.1633	7.8117	16.3196	19.5311	20.0
3.9104	114.13	10500	4.2276	21.0784	7.6899	16.3248	19.5625	20.0
3.915	115.22	10600	4.2282	20.9369	7.6522	16.1615	19.4826	20.0
3.8748	116.3	10700	4.2268	20.9369	7.6522	16.1615	19.4826	20.0
3.9341	117.39	10800	4.2252	21.0067	7.7263	16.3314	19.5589	20.0
3.8713	118.48	10900	4.2253	20.7028	7.5712	16.0398	19.2212	20.0
3.8861	119.57	11000	4.2243	20.7075	7.6844	16.0626	19.2959	20.0
3.8905	120.65	11100	4.2252	20.6546	7.5642	15.9451	19.1838	20.0
3.8682	121.74	11200	4.2238	20.8809	7.6536	16.1667	19.4217	20.0
3.904	122.83	11300	4.2241	20.6916	7.5324	15.9692	19.1791	20.0
3.8577	123.91	11400	4.2231	20.9271	7.6536	16.2314	19.4695	20.0
3.8851	125.0	11500	4.2230	20.8097	7.6891	16.1087	19.3872	20.0
3.8725	126.09	11600	4.2219	20.8965	7.6891	16.197	19.4319	20.0
3.8918	127.17	11700	4.2210	20.8203	7.6562	16.1283	19.388	20.0
3.845	128.26	11800	4.2210	20.7633	7.6883	16.0813	19.3537	20.0
3.8812	129.35	11900	4.2197	20.6605	7.6351	15.9703	19.2425	20.0
3.8734	130.43	12000	4.2208	20.6164	7.601	15.9703	19.1967	20.0
3.8704	131.52	12100	4.2201	20.533	7.5141	15.941	19.1898	20.0
3.8302	132.61	12200	4.2194	20.6164	7.601	15.9703	19.1967	20.0
3.8793	133.7	12300	4.2178	20.5427	7.5674	15.9591	19.2078	20.0
3.8631	134.78	12400	4.2181	20.6953	7.6549	16.0402	19.2734	20.0
3.8565	135.87	12500	4.2173	20.6168	7.5808	16.0402	19.2734	20.0
3.8842	136.96	12600	4.2163	20.6525	7.5782	16.0402	19.3124	20.0
3.8183	138.04	12700	4.2165	20.6168	7.5808	16.0402	19.2734	20.0
3.8482	139.13	12800	4.2155	20.6953	7.6154	16.0402	19.2734	20.0
3.8689	140.22	12900	4.2158	20.8264	7.7844	16.1396	19.4834	20.0
3.8361	141.3	13000	4.2144	20.8264	7.6986	16.2466	19.5192	20.0
3.8336	142.39	13100	4.2148	20.7613	7.7027	16.2516	19.4307	20.0
3.8532	143.48	13200	4.2155	20.6905	7.6695	16.1708	19.3584	20.0
3.8424	144.57	13300	4.2137	20.7613	7.7027	16.2516	19.4307	20.0
3.8781	145.65	13400	4.2128	20.6905	7.6695	16.1708	19.3584	20.0
3.8693	146.74	13500	4.2128	20.5395	7.4561	16.1388	19.1866	20.0
3.8304	147.83	13600	4.2123	20.6345	7.7324	16.1761	19.2764	20.0
3.8434	148.91	13700	4.2123	20.7145	7.6768	16.1729	19.3787	20.0
3.8348	150.0	13800	4.2123	20.7859	7.7023	16.2986	19.4932	20.0
3.8375	151.09	13900	4.2126	20.6319	7.5676	16.325	19.2512	20.0
3.8421	152.17	14000	4.2120	20.6665	7.5619	16.3257	19.2911	20.0
3.831	153.26	14100	4.2110	20.609	7.4912	16.2881	19.2953	20.0
3.8172	154.35	14200	4.2112	20.7352	7.6588	16.2115	19.3408	20.0
3.7853	155.43	14300	4.2107	20.6635	7.5987	16.2131	19.2667	20.0
3.8274	156.52	14400	4.2109	20.7352	7.7559	16.3035	19.3408	20.0
3.8362	157.61	14500	4.2099	20.7559	7.6865	16.325	19.4191	20.0
3.8561	158.7	14600	4.2098	20.6225	7.6943	16.3448	19.1425	20.0
3.7832	159.78	14700	4.2098	20.6307	7.6684	16.2469	19.269	20.0
3.8409	160.87	14800	4.2092	20.683	7.7924	16.2986	19.2414	20.0
3.821	161.96	14900	4.2092	20.5235	7.6721	16.2191	18.9879	20.0
3.8343	163.04	15000	4.2089	20.5235	7.6721	16.2191	18.9879	20.0
3.8279	164.13	15100	4.2087	20.5304	7.5448	16.2106	19.0909	20.0
3.7874	165.22	15200	4.2083	20.6319	7.6145	16.3035	19.2294	20.0
3.8316	166.3	15300	4.2076	20.5759	7.6145	16.2528	19.1508	20.0
3.7817	167.39	15400	4.2084	20.4845	7.5473	16.2067	19.0683	20.0
3.8338	168.48	15500	4.2075	20.5375	7.614	16.2509	19.1047	20.0
3.8515	169.57	15600	4.2069	20.4845	7.5473	16.2067	19.0683	20.0
3.7895	170.65	15700	4.2074	20.4845	7.5473	16.2067	19.0683	20.0
3.8129	171.74	15800	4.2076	20.4845	7.5473	16.2067	19.0683	20.0
3.8582	172.83	15900	4.2073	20.4845	7.5473	16.2067	19.0683	20.0
3.7716	173.91	16000	4.2073	20.5333	7.6756	16.2531	19.0336	20.0
3.8142	175.0	16100	4.2069	20.5333	7.6756	16.2531	19.0336	20.0
3.8186	176.09	16200	4.2068	20.5333	7.6756	16.2531	19.0336	20.0
3.8323	177.17	16300	4.2065	20.5333	7.6281	16.2531	19.0336	20.0
3.774	178.26	16400	4.2064	20.5724	7.677	16.2545	19.0747	20.0
3.8123	179.35	16500	4.2062	20.5333	7.6281	16.2531	19.0336	20.0
3.7914	180.43	16600	4.2066	20.5333	7.6281	16.2531	19.0336	20.0
3.7988	181.52	16700	4.2063	20.5724	7.6287	16.2545	19.0747	20.0
3.8331	182.61	16800	4.2059	20.6225	7.7265	16.3103	19.1036	20.0
3.8125	183.7	16900	4.2061	20.494	7.5897	16.2303	18.9697	20.0
3.8069	184.78	17000	4.2059	20.5333	7.6756	16.2531	19.0336	20.0
3.7933	185.87	17100	4.2058	20.5333	7.6281	16.2531	19.0336	20.0
3.807	186.96	17200	4.2058	20.5333	7.6756	16.2531	19.0336	20.0
3.8	188.04	17300	4.2055	20.5333	7.6756	16.2531	19.0336	20.0
3.776	189.13	17400	4.2057	20.5333	7.6756	16.2531	19.0336	20.0
3.7976	190.22	17500	4.2057	20.5333	7.6756	16.2531	19.0336	20.0
3.8293	191.3	17600	4.2057	20.5333	7.6756	16.2531	19.0336	20.0
3.7807	192.39	17700	4.2057	20.5333	7.6756	16.2531	19.0336	20.0
3.8246	193.48	17800	4.2055	20.5333	7.6756	16.2531	19.0336	20.0
3.7719	194.57	17900	4.2055	20.5333	7.6756	16.2531	19.0336	20.0
3.8055	195.65	18000	4.2055	20.5333	7.6756	16.2531	19.0336	20.0
3.7803	196.74	18100	4.2055	20.5333	7.6756	16.2531	19.0336	20.0
3.8287	197.83	18200	4.2055	20.5333	7.6756	16.2531	19.0336	20.0
3.8066	198.91	18300	4.2055	20.5333	7.6756	16.2531	19.0336	20.0
3.8011	200.0	18400	4.2055	20.5333	7.6756	16.2531	19.0336	20.0

Framework versions

• Transformers 4.18.0
• Pytorch 1.11.0+cu113
• Datasets 2.1.0
• Tokenizers 0.12.1