emotions_google_gemma

This model is a fine-tuned version of google/gemma-1.1-2b-it on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 0.4858
• F1 Micro: 0.6961
• F1 Macro: 0.6067
• Accuracy: 0.2369

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0001
• train_batch_size: 16
• eval_batch_size: 16
• seed: 42
• gradient_accumulation_steps: 4
• total_train_batch_size: 64
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 5

Training results

Training Loss	Epoch	Step	Validation Loss	F1 Micro	F1 Macro	Accuracy
0.7307	0.2067	20	0.6081	0.6151	0.4711	0.1709
0.5276	0.4134	40	0.5106	0.6760	0.5729	0.1676
0.5081	0.6202	60	0.4978	0.6922	0.5857	0.2006
0.5091	0.8269	80	0.4853	0.6908	0.6038	0.2052
0.4823	1.0336	100	0.4858	0.6961	0.6067	0.2369
0.3346	1.2403	120	0.5500	0.6663	0.5728	0.1702
0.3245	1.4470	140	0.5440	0.6816	0.5819	0.2052
0.3375	1.6537	160	0.5541	0.6808	0.5853	0.1987
0.3407	1.8605	180	0.5275	0.6875	0.5872	0.1916
0.2745	2.0672	200	0.6019	0.6928	0.5784	0.2311
0.1312	2.2739	220	0.7540	0.6750	0.5614	0.2097
0.1363	2.4806	240	0.7292	0.6805	0.5712	0.2078
0.1302	2.6873	260	0.7316	0.6846	0.5795	0.2045
0.125	2.8941	280	0.7491	0.6819	0.5711	0.2104
0.0877	3.1008	300	0.8069	0.6805	0.5651	0.2330
0.0457	3.3075	320	0.8849	0.6867	0.5592	0.2356
0.0453	3.5142	340	0.8583	0.6774	0.5626	0.2246
0.0429	3.7209	360	0.8338	0.6812	0.5675	0.2227
0.0463	3.9276	380	0.8497	0.6823	0.5735	0.2272
0.0256	4.1344	400	0.9236	0.6759	0.5607	0.2278
0.0155	4.3411	420	0.9380	0.6871	0.5651	0.2421
0.0127	4.5478	440	0.9505	0.6852	0.5646	0.2414
0.0129	4.7545	460	0.9438	0.6837	0.5658	0.2382
0.0123	4.9612	480	0.9431	0.6843	0.5666	0.2401

Framework versions

• PEFT 0.10.0
• Transformers 4.40.2
• Pytorch 2.2.1+cu121
• Datasets 2.19.1
• Tokenizers 0.19.1