fununun / README.md
mecoaoge2's picture
End of training
5c5c6c1
metadata
license: apache-2.0
library_name: peft
tags:
  - trl
  - dpo
  - generated_from_trainer
base_model: TinyLlama/TinyLlama-1.1B-Chat-v1.0
model-index:
  - name: fununun
    results: []

fununun

This model is a fine-tuned version of TinyLlama/TinyLlama-1.1B-Chat-v1.0 on an unknown dataset. It achieves the following results on the evaluation set:

  • Loss: 0.6908
  • Rewards/chosen: 0.0015
  • Rewards/rejected: -0.0032
  • Rewards/accuracies: 0.7176
  • Rewards/margins: 0.0047
  • Logps/rejected: -197.2385
  • Logps/chosen: -235.0630
  • Logits/rejected: -3.0691
  • Logits/chosen: -3.1037

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

  • learning_rate: 5e-07
  • train_batch_size: 2
  • eval_batch_size: 2
  • seed: 42
  • gradient_accumulation_steps: 16
  • total_train_batch_size: 32
  • optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
  • lr_scheduler_type: linear
  • lr_scheduler_warmup_steps: 20
  • training_steps: 100

Training results

Training Loss Epoch Step Validation Loss Rewards/chosen Rewards/rejected Rewards/accuracies Rewards/margins Logps/rejected Logps/chosen Logits/rejected Logits/chosen
0.693 0.04 20 0.6927 0.0001 -0.0007 0.5614 0.0009 -197.2139 -235.0765 -3.0688 -3.1035
0.6922 0.07 40 0.6919 0.0007 -0.0017 0.6440 0.0024 -197.2236 -235.0704 -3.0690 -3.1036
0.6913 0.11 60 0.6913 0.0011 -0.0025 0.6886 0.0037 -197.2319 -235.0664 -3.0691 -3.1037
0.6909 0.15 80 0.6909 0.0014 -0.0030 0.7098 0.0044 -197.2367 -235.0639 -3.0691 -3.1037
0.6906 0.19 100 0.6908 0.0015 -0.0032 0.7176 0.0047 -197.2385 -235.0630 -3.0691 -3.1037

Framework versions

  • PEFT 0.7.1
  • Transformers 4.36.2
  • Pytorch 2.1.0+cu121
  • Datasets 2.16.1
  • Tokenizers 0.15.0