snowflake_en_llama3_70

This model was trained from scratch on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 0.2847
• Precision: 0.5140
• Recall: 0.4098
• F1 Macro: 0.4397
• Accuracy: 0.6854

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0003
• train_batch_size: 256
• eval_batch_size: 128
• seed: 0
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 20

Training results

Training Loss	Epoch	Step	Validation Loss	Precision	Recall	F1 Macro	Accuracy
No log	0	0	1.9416	0.1293	0.1671	0.0404	0.1279
0.3479	0.6083	1000	0.3381	0.4895	0.3523	0.3731	0.6456
0.3265	1.2165	2000	0.3177	0.5076	0.3831	0.4118	0.6655
0.3188	1.8248	3000	0.3147	0.5156	0.3880	0.4142	0.6624
0.3108	2.4331	4000	0.3030	0.5149	0.3921	0.4230	0.6773
0.3012	3.0414	5000	0.3035	0.5180	0.3857	0.4134	0.6793
0.2996	3.6496	6000	0.2990	0.5079	0.3974	0.4249	0.6787
0.2923	4.2579	7000	0.2951	0.5135	0.3997	0.4311	0.6791
0.2857	4.8662	8000	0.2972	0.5093	0.4040	0.4322	0.6741
0.2854	5.4745	9000	0.2907	0.5185	0.3968	0.4285	0.6842
0.2821	6.0827	10000	0.2891	0.5362	0.3834	0.4148	0.6832
0.2823	6.6910	11000	0.2907	0.5141	0.4021	0.4316	0.6796
0.2715	7.2993	12000	0.2899	0.5226	0.3928	0.4233	0.6866
0.2714	7.9075	13000	0.2907	0.5337	0.3827	0.4141	0.6814
0.2709	8.5158	14000	0.2877	0.5141	0.4048	0.4341	0.6839
0.2683	9.1241	15000	0.2867	0.5085	0.4080	0.4381	0.6837
0.2658	9.7324	16000	0.2871	0.5078	0.4101	0.4377	0.6840
0.2669	10.3406	17000	0.2901	0.5278	0.3824	0.4133	0.6857
0.2576	10.9489	18000	0.2861	0.5161	0.4143	0.4448	0.6853
0.2579	11.5572	19000	0.2880	0.5066	0.4088	0.4371	0.6843
0.2532	12.1655	20000	0.2898	0.5096	0.4147	0.4419	0.6801
0.2545	12.7737	21000	0.2872	0.5167	0.3992	0.4293	0.6849
0.2513	13.3820	22000	0.2858	0.5122	0.4079	0.4370	0.6841
0.2485	13.9903	23000	0.2857	0.5193	0.4027	0.4338	0.6871
0.2477	14.5985	24000	0.2854	0.5239	0.4087	0.4394	0.6855
0.2429	15.2068	25000	0.2879	0.5114	0.4076	0.4361	0.6808
0.2438	15.8151	26000	0.2855	0.5131	0.4047	0.4343	0.6860
0.2423	16.4234	27000	0.2867	0.5082	0.4138	0.4418	0.6832
0.2361	17.0316	28000	0.2866	0.5214	0.4017	0.4319	0.6830
0.246	17.6399	29000	0.2848	0.5116	0.4075	0.4365	0.6853
0.237	18.2482	30000	0.2853	0.5120	0.4149	0.4439	0.6847
0.2364	18.8564	31000	0.2847	0.5212	0.4036	0.4342	0.6858
0.2368	19.4647	32000	0.2847	0.5140	0.4098	0.4397	0.6854

Framework versions

• Transformers 4.43.3
• Pytorch 2.4.0+cu121
• Datasets 2.20.0
• Tokenizers 0.19.1