Model Card: Llama-7b with LoRA Fine-tuning on QACR data
Model Overview
- Model Name: Llama-7b
- Model Architecture: Transformer-based Language Model
- Fine-tuning Method: LoRA
- Training Datasets:
- Educational Question Generation Dataset (described in the dataset chart)
- Alpaca GPT-4 french dataset (chat instruction task)
- Dolly_fr dataset (chat instruction task)
Model Details
- Base Model: decapoda-research/llama-7b-hf
- Fine-tuning Approach: LoRA fine-tuning method, which combines pre-training on a large corpus with additional task-specific fine-tuning.
- Training Objective: The model is trained to generate relevant and useful questions based on educational texts and to handle chat instruction tasks from the Alpaca GPT-4 and Dolly datasets.
- Training Procedure: The base Llama-7b model is first pretrained on a large corpus to learn general language patterns and representations. It is then fine-tuned using a combination of the aforementioned datasets to specialize in educational question generation and chat instruction tasks.
Intended Use
- Primary Task: Question generation for educational purposes and chat instruction tasks.
- Potential Use Cases:
- Automated question generation for educational platforms and tutoring systems.
- Chat-based instruction and assistance in various domains.