File size: 1,353 Bytes
1ce2b6a |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 |
---
language:
- es
---
## EQASpa 7b
This model is a fine-tuned version of [Llama-2-7b-hf](https://huggingface.co/meta-llama/Llama-2-7b-hf) for the Extractive Question Answering task in spanish.
It was fine-tuned on the [QuALES 2022](https://www.fing.edu.uy/inco/grupos/pln/quales/) dataset training partition using LoRA for one epoch.
## Prompt format
To use the model, the following prompting format should be applied:
### TEXTO:
{{Context document}}
### PREGUNTA:
{{Question}}
### RESPUESTA:
## Evaluation
We evaluate the model on the test partition of the QuALES dataset, and compare it with one-shot prompting as a baseline.
Prompt | Model | Acc_exact | F_bertscore
--- | --- | --- | ---
one-shot prompting | [zephyr-7b-beta](HuggingFaceH4/zephyr-7b-beta) | 0.025 | 0.614
one-shot prompting | [Llama-2-13b-chat-hf](meta-llama/Llama-2-13b-chat-hf) | 0.192 | 0.700
default | EQASpa 7b | **0.225** | **0.713**
## Training procedure
The following `bitsandbytes` quantization config was used during training:
- load_in_8bit: False
- load_in_4bit: True
- llm_int8_threshold: 6.0
- llm_int8_skip_modules: None
- llm_int8_enable_fp32_cpu_offload: False
- llm_int8_has_fp16_weight: False
- bnb_4bit_quant_type: nf4
- bnb_4bit_use_double_quant: False
- bnb_4bit_compute_dtype: float16
### Framework versions
- PEFT 0.4.0 |