README.md

## 📖 Introduction

**Qwen2-7B-Instruct-Response-Exp** is a powerful large language model that can automatically expand datasets and annotations like self-instruct.

We fine-tuned **Qwen2-7B-Instruct** to obtain **Qwen2-7B-Instruct-Response-Exp**.
We sampled the dataset from OpenHermes and the LCCD dataset, ensuring a balanced task distribution. For training set annotations, we used Qwen-max with incorporated our handwritten examples as in-context prompts.

#### Example Input
>\#\#\# Instruction
>
>Here are some concepts: car, driving, travel Can you make a sentence with these concepts?
>
>\#\#\# Response
>
>Of course you can! Here is a sentence: "We drove the car and started an unforgettable journey.
>
>\#\#\# Instruction
>
>Why does the sun rise in the east every day?
>
>\#\#\# Response
>
> 
>The sun rises in the east every day because of the direction of the earth's rotation and geographical features. The earth rotates from west to east, so an observer on the surface of the earth will see the sun rise from the eastern horizon, pass through the sky, and finally set on the western horizon. Specifically, the earth's rotation axis refers to an imaginary axis that passes through the north and south poles of the earth, and the earth rotates from west to east around this axis. Because of this rotation, any place on the earth will experience day and night. When a place rotates to the side facing the sun, it will usher in daytime; when it rotates to the side facing away from the sun, it will usher in night. Since the direction of the earth's rotation is from west to east, the sun appears to rise from the east and set in the west. This phenomenon is a direct result of the earth's rotation, not the movement of the sun itself.
>
>\#\#\# Instruction
>
#### Example Output
>\#\#\# Instruction
>
>Here are some concepts: books, learning, knowledge Can you make a sentence with these concepts?
>
>\#\#\# Response
>
>Of course you can! Here is a sentence that contains these concepts: By reading books, we can continue to learn and enrich our knowledge. 



## 🚀 Quick Start

Here provides a code snippet with `apply_chat_template` to show you how to load the tokenizer and model and how to generate contents.

```python
from transformers import AutoModelForCausalLM, AutoTokenizer
device = "cuda" # the device to load the model onto

model = AutoModelForCausalLM.from_pretrained(
    "alibaba-pai/Qwen2-7B-Instruct-Response-Exp",
    torch_dtype="auto",
    device_map="auto"
)
tokenizer = AutoTokenizer.from_pretrained("alibaba-pai/Qwen2-7B-Instruct-Response-Exp")

prompt = "Give me a short introduction to large language model."
messages = [
    {"role": "user", "content": prompt}
]
text = tokenizer.apply_chat_template(
    messages,
    tokenize=False,
    add_generation_prompt=True
)
model_inputs = tokenizer([text], return_tensors="pt").to(device)

generated_ids = model.generate(
    model_inputs.input_ids,
    max_new_tokens=2048，
    eos_token_id=151645，
)
generated_ids = [
    output_ids[len(input_ids):] for input_ids, output_ids in zip(model_inputs.input_ids, generated_ids)
]

response = tokenizer.batch_decode(generated_ids, skip_special_tokens=True)[0]
```

## 🔍 Evaluation

|                 | Diversity | Length | Complexity | Factuality |
|-----------------|-----------|--------|------------|------------|
| Self-Instruct   | 9.6       | 15.8   | 0.32       | 5.0        |
| Qwen2-7B-Instruct-Response-Exp | 17.2      | 26.3   | 4.97       | 4.9        |