unaligned

This model is a fine-tuned version of meta-llama/Meta-Llama-3-8B-Instruct on the None dataset. It achieves the following results on the evaluation set:

• Loss: 0.0709

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0002
• train_batch_size: 8
• eval_batch_size: 8
• seed: 42
• gradient_accumulation_steps: 8
• total_train_batch_size: 64
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• lr_scheduler_warmup_steps: 100
• training_steps: 2048

Training results

Training Loss	Epoch	Step	Validation Loss
0.1154	0.1110	100	0.1172
0.092	0.2220	200	0.1028
0.0462	0.3330	300	0.0992
0.0482	0.4440	400	0.0755
0.043	0.5550	500	0.0794
0.0476	0.6660	600	0.0628
0.0482	0.7770	700	0.0821
0.0484	0.8880	800	0.0691
0.0448	0.9990	900	0.0829
0.0214	1.1100	1000	0.0720
0.0439	1.2210	1100	0.0635
0.0364	1.3320	1200	0.0713
0.0497	1.4430	1300	0.0669
0.0455	1.5540	1400	0.0672
0.0614	1.6650	1500	0.0805
0.0416	1.7761	1600	0.0669
0.0367	1.8871	1700	0.0716
0.0578	1.9981	1800	0.0684
0.0358	2.1091	1900	0.0705
0.0326	2.2201	2000	0.0709

Framework versions

• PEFT 0.11.1
• Transformers 4.41.1
• Pytorch 2.3.0+cu121
• Datasets 2.19.1
• Tokenizers 0.19.1