Model description
The llama2-7B-finetuned-chat-guanaco model is a fine-tuned version of the NousResearch/Llama-2-7b-chat-hf base model. This base model is a variant of LLaMA (Large Language Model Meta AI) designed for chat applications, optimized for conversational understanding and generation.
Dataset used
Intended uses & limitations
More information needed
Training results
The training loss over steps is as follows:
| Step | Training Loss |
|---|---|
| 25 | 1.823 |
| 50 | 2.056 |
| 75 | 1.829 |
| 100 | 1.744 |
| 125 | 1.717 |
| 150 | 1.412 |
| 175 | 1.506 |
| 200 | 1.446 |
| 225 | 1.499 |
| 250 | 1.432 |
| 275 | 1.281 |
| 300 | 1.341 |
| 325 | 1.345 |
| 350 | 1.391 |
| 375 | 1.388 |
Training procedure
Training hyperparameters
The following hyperparameters were used during training:
- learning_rate: 2e-05
- train_batch_size: 8
- eval_batch_size: 8
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: constant
- lr_scheduler_warmup_ratio: 0.03
- num_epochs: 3
- mixed_precision_training: Native AMP
Framework versions
- PEFT 0.12.0
- Transformers 4.43.3
- Pytorch 2.3.1+cu121
- Datasets 2.20.0
- Tokenizers 0.19.1
- Downloads last month
- 3
Inference API (serverless) does not yet support peft models for this pipeline type.
Model tree for luluw/llama2-7B-finetuned-chat-guanaco
Base model
NousResearch/Llama-2-7b-chat-hf