---
base_model:
- NousResearch/Llama-2-7b-hf
pipeline_tag: text-generation
library_name: peft
license: apache-2.0
datasets:
- mlabonne/guanaco-llama2-1k
metrics:
- bleu
- accuracy
language:
- en
tags:
- 'nlp '
- genetation
---
## Abstract

Llama 2 is a state-of-the-art large language model (LLM) developed by Meta AI, designed for a variety of natural language processing (NLP) tasks. Its architecture builds upon transformer-based models, leveraging massive text corpora during pretraining to develop rich language understanding capabilities. Fine-tuning Llama 2 can be customized to a specific task by using a smaller, task-specific dataset, often resulting in a specialized model that outperforms the general-purpose base model on that task. In this project, we fine-tune the Llama-2-7b-hf model using a subset of the Guanaco dataset, focusing on developing a highly efficient model called "MiniGuanaco."

## Project Overview

This repository demonstrates the steps and code required to fine-tune Llama 2 for specific tasks. Using the Hugging Face model **NousResearch/Llama-2-7b-hf** as the base, the model is fine-tuned with the dataset **mlabonne/guanaco-llama2-1k**. The fine-tuned model is saved under the name **llama2-miniguanaco**.

# You can access more information through: 
  - [Github](https://github.com/zeyadusf/FineTune-Llama2)
  - [Kaggle](https://www.kaggle.com/code/zeyadusf/finetune-llama2/notebook)