Hugging Face's logo

imdatta0
/
gemma-2-9b_pct_default_r32

PEFT
Safetensors
unsloth
Generated from Trainer
Model card Files Community
gemma-2-9b_pct_default_r32 / README.md
imdatta0's picture
imdatta0
End of training
0ee87d9 verified
preview code
|
raw
history blame contribute delete
3.88 kB
metadata
base_model: google/gemma-2-9b
library_name: peft
license: gemma
tags:
  - unsloth
  - generated_from_trainer
model-index:
  - name: gemma-2-9b_pct_default_r32
    results: []

gemma-2-9b_pct_default_r32

This model is a fine-tuned version of google/gemma-2-9b on an unknown dataset. It achieves the following results on the evaluation set:

  • Loss: 9.9020

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

  • learning_rate: 0.0001
  • train_batch_size: 1
  • eval_batch_size: 1
  • seed: 42
  • gradient_accumulation_steps: 64
  • total_train_batch_size: 64
  • optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
  • lr_scheduler_type: cosine
  • lr_scheduler_warmup_ratio: 0.02
  • num_epochs: 1

Training results

Training Loss Epoch Step Validation Loss
2.5978 0.0206 8 4.9799
11.0056 0.0412 16 11.4640
11.7215 0.0618 24 11.8765
11.8793 0.0824 32 11.9059
11.8739 0.1030 40 11.9781
11.8763 0.1236 48 11.9487
11.8231 0.1442 56 11.8282
11.7758 0.1648 64 11.7664
11.8011 0.1854 72 11.4372
11.6991 0.2060 80 11.7334
11.8108 0.2266 88 11.5005
11.6519 0.2472 96 11.5944
11.6905 0.2678 104 11.5944
11.6003 0.2885 112 11.5914
11.5813 0.3091 120 11.5684
11.5493 0.3297 128 11.7560
11.5458 0.3503 136 11.4566
11.5838 0.3709 144 11.4331
11.4815 0.3915 152 11.5174
11.5369 0.4121 160 11.5271
11.4617 0.4327 168 11.5392
11.4399 0.4533 176 11.3691
11.3199 0.4739 184 10.9823
10.6547 0.4945 192 10.0666
8.8163 0.5151 200 8.7638
9.5635 0.5357 208 8.5153
8.7862 0.5563 216 9.2186
10.2774 0.5769 224 10.3834
9.7932 0.5975 232 9.7972
9.5421 0.6181 240 9.6845
9.5401 0.6387 248 9.3438
10.9001 0.6593 256 10.6070
9.959 0.6799 264 9.6172
9.5409 0.7005 272 10.4762
10.8074 0.7211 280 10.4872
9.1645 0.7417 288 7.6567
8.1072 0.7623 296 8.7911
9.7069 0.7829 304 10.0043
10.0752 0.8035 312 10.1655
9.9734 0.8241 320 9.9644
9.6722 0.8447 328 9.7803
9.8279 0.8654 336 9.5826
9.6714 0.8860 344 9.5533
9.655 0.9066 352 9.6339
9.7184 0.9272 360 9.7651
9.6142 0.9478 368 9.8536
9.9249 0.9684 376 9.8906
9.8654 0.9890 384 9.9020

Framework versions

  • PEFT 0.12.0
  • Transformers 4.44.2
  • Pytorch 2.3.0+cu121
  • Datasets 2.21.0
  • Tokenizers 0.19.1