swin-transformer3

This model is a fine-tuned version of microsoft/swin-large-patch4-window12-384 on the None dataset. It achieves the following results on the evaluation set:

• Loss: 4.1081
• Accuracy: 0.5667
• F1: 0.5667

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 4
• eval_batch_size: 4
• seed: 42
• gradient_accumulation_steps: 4
• total_train_batch_size: 16
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• lr_scheduler_warmup_ratio: 0.1
• num_epochs: 100

Training results

Training Loss	Epoch	Step	Validation Loss	Accuracy	F1
1.6659	0.9925	33	1.0639	0.6333	0.6250
0.7561	1.9850	66	0.7258	0.5167	0.3520
0.7106	2.9774	99	0.7334	0.5	0.3755
0.6749	4.0	133	0.7088	0.4833	0.3661
0.751	4.9925	166	0.7356	0.4833	0.3661
0.7146	5.9850	199	0.7837	0.4833	0.3150
0.6699	6.9774	232	0.7569	0.4833	0.3424
0.6521	8.0	266	0.7255	0.5333	0.4674
0.6885	8.9925	299	0.7253	0.5167	0.4070
0.6407	9.9850	332	0.6506	0.6	0.5909
0.6436	10.9774	365	0.6720	0.55	0.4442
0.7865	12.0	399	0.6606	0.55	0.4792
0.7191	12.9925	432	0.6407	0.65	0.6466
0.5889	13.9850	465	0.8008	0.4833	0.3619
0.5489	14.9774	498	0.7298	0.5333	0.4674
0.596	16.0	532	0.7465	0.6667	0.6591
0.6136	16.9925	565	0.9118	0.5333	0.4692
0.5961	17.9850	598	0.6902	0.65	0.6298
0.6327	18.9774	631	0.8260	0.5667	0.5190
0.6518	20.0	665	0.6919	0.5833	0.5715
0.5551	20.9925	698	1.1780	0.55	0.516
0.511	21.9850	731	0.7414	0.6	0.6
0.4749	22.9774	764	0.7978	0.6167	0.6129
0.4607	24.0	798	0.8087	0.55	0.5420
0.5837	24.9925	831	0.8271	0.5667	0.5456
0.4608	25.9850	864	0.8539	0.6	0.5863
0.536	26.9774	897	0.9802	0.5333	0.5026
0.4225	28.0	931	0.9275	0.6	0.5910
0.4325	28.9925	964	0.8834	0.6167	0.6099
0.4874	29.9850	997	0.8721	0.6167	0.6168
0.4165	30.9774	1030	1.0360	0.6167	0.6163
0.4773	32.0	1064	1.2210	0.5833	0.5759
0.3756	32.9925	1097	1.1291	0.5833	0.5830
0.636	33.9850	1130	1.0178	0.5833	0.5830
0.5474	34.9774	1163	0.9479	0.5667	0.5608
0.3462	36.0	1197	0.9585	0.6167	0.6163
0.3057	36.9925	1230	1.2014	0.6167	0.6163
0.2304	37.9850	1263	1.1975	0.6333	0.6333
0.2628	38.9774	1296	1.5224	0.5833	0.5793
0.3774	40.0	1330	1.2903	0.5667	0.5516
0.2604	40.9925	1363	1.4082	0.5667	0.5608
0.2522	41.9850	1396	1.1783	0.6167	0.6163
0.1925	42.9774	1429	1.3613	0.6167	0.6163
0.3436	44.0	1463	1.6383	0.5333	0.5173
0.1955	44.9925	1496	1.8947	0.5	0.4829
0.2206	45.9850	1529	1.4390	0.6	0.6
0.1912	46.9774	1562	1.5288	0.65	0.6400
0.2794	48.0	1596	1.7393	0.55	0.5420
0.3166	48.9925	1629	2.0414	0.5667	0.5608
0.173	49.9850	1662	1.6377	0.6	0.5991
0.1375	50.9774	1695	1.6228	0.6	0.6
0.2659	52.0	1729	1.6452	0.6333	0.6333
0.2045	52.9925	1762	1.9706	0.5667	0.5608
0.1081	53.9850	1795	1.9546	0.6167	0.6009
0.1782	54.9774	1828	2.1268	0.5667	0.5608
0.244	56.0	1862	1.8301	0.6167	0.6098
0.1783	56.9925	1895	2.5808	0.5667	0.5071
0.2429	57.9850	1928	2.1214	0.6167	0.6059
0.2	58.9774	1961	2.2282	0.5667	0.5657
0.1646	60.0	1995	2.3272	0.5833	0.5662
0.1663	60.9925	2028	2.4723	0.5333	0.5323
0.1935	61.9850	2061	2.3384	0.6	0.5973
0.2079	62.9774	2094	1.9271	0.5833	0.5830
0.1797	64.0	2128	1.8707	0.6167	0.6151
0.173	64.9925	2161	2.6292	0.5167	0.5031
0.1815	65.9850	2194	2.6567	0.6	0.5973
0.0665	66.9774	2227	3.2104	0.5167	0.5031
0.1084	68.0	2261	3.6692	0.5333	0.5228
0.1298	68.9925	2294	3.4104	0.55	0.5373
0.1338	69.9850	2327	2.8215	0.6	0.5973
0.0795	70.9774	2360	2.9208	0.5833	0.5830
0.1138	72.0	2394	3.4277	0.5333	0.5302
0.1644	72.9925	2427	2.8141	0.5833	0.5830
0.1659	73.9850	2460	2.8723	0.6	0.6
0.0453	74.9774	2493	2.8769	0.6333	0.6309
0.0956	76.0	2527	3.2970	0.6167	0.6098
0.1581	76.9925	2560	3.6672	0.5833	0.5816
0.157	77.9850	2593	3.5317	0.55	0.5501
0.0662	78.9774	2626	3.9003	0.55	0.5456
0.1954	80.0	2660	3.3000	0.5833	0.5834
0.0527	80.9925	2693	3.9596	0.5667	0.5638
0.1578	81.9850	2726	3.6724	0.55	0.5481
0.0737	82.9774	2759	4.0222	0.5167	0.5119
0.0617	84.0	2793	3.5510	0.5833	0.5834
0.0531	84.9925	2826	3.5110	0.6	0.6
0.0993	85.9850	2859	4.0699	0.55	0.5481
0.1545	86.9774	2892	3.6860	0.5667	0.5667
0.0554	88.0	2926	3.4409	0.6	0.6
0.0641	88.9925	2959	3.8304	0.55	0.5496
0.0633	89.9850	2992	4.0899	0.55	0.5456
0.0991	90.9774	3025	3.7344	0.6	0.6
0.0772	92.0	3059	3.8448	0.6	0.5991
0.0646	92.9925	3092	3.7794	0.6	0.5991
0.0562	93.9850	3125	3.9340	0.5833	0.5830
0.0475	94.9774	3158	4.2388	0.55	0.5481
0.0715	96.0	3192	4.2732	0.5333	0.5302
0.0875	96.9925	3225	4.1521	0.5667	0.5657
0.0253	97.9850	3258	4.0813	0.5667	0.5667
0.1037	98.9774	3291	4.1074	0.5667	0.5667
0.1094	99.2481	3300	4.1081	0.5667	0.5667

Framework versions

• Transformers 4.45.2
• Pytorch 2.4.0+cu121
• Datasets 3.1.0
• Tokenizers 0.20.1