File size: 3,128 Bytes
948b0e8 c8c9b8f 948b0e8 0d00bf4 c8c9b8f 948b0e8 6aba063 0d00bf4 f0e03f4 6aba063 948b0e8 0d00bf4 f0e03f4 948b0e8 0d00bf4 948b0e8 fcf1501 948b0e8 0d00bf4 fcf1501 0d00bf4 fcf1501 0d00bf4 fcf1501 0d00bf4 fcf1501 0d00bf4 f0e03f4 0d00bf4 f0e03f4 0d00bf4 fcf1501 0d00bf4 fcf1501 0d00bf4 fcf1501 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 |
---
license: llama3
tags:
- moe
- merge
- llama-3
language:
- en
- tr
pipeline_tag: text-generation
library_name: transformers
---
## 💻 For English
Defne_llama3_2x8B is a Mixure of Experts (MoE) (two llama3 models).
(Change the system prompt for Turkish as shown below)
```python
!pip install -qU transformers bitsandbytes accelerate
from transformers import AutoTokenizer, AutoModelForCausalLM
import torch
model_id = "Eurdem/Defne_llama3_2x8B"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(model_id, torch_dtype=torch.bfloat16, device_map="auto", load_in_8bit= True)
messages = [
{"role": "system", "content": "You are a helpful chatbot, named Defne, who always responds friendly."},
{"role": "user", "content": "Answer the questions: 1) Who are you? 2) f(x)=3x^2+4x+12 so what is f(3)?"},
]
input_ids = tokenizer.apply_chat_template(messages, return_tensors="pt").to("cuda")
outputs = model.generate(input_ids,
max_new_tokens=1024,
do_sample=True,
temperature=0.7,
top_p=0.7,
top_k=500,
)
response = outputs[0][input_ids.shape[-1]:]
print(tokenizer.decode(response, skip_special_tokens=True))
```
### Output
```
Hello there! I'm Defne, a friendly chatbot here to help with any questions you may have.
Now, let's get to the math problem!
The function is f(x) = 3x^2 + 4x + 12, and we want to find f(3). To do that, we can plug in 3 for x in the function:
f(3) = 3(3)^2 + 4(3) + 12
f(3) = 3(9) + 12 + 12
f(3) = 27 + 24
f(3) = 51
So, f(3) is equal to 51!
```
## 💻 Türkçe İçin
Defne_llama3_2x8B, iki llama3 8B modelinin birleşmesi ile oluşturulan MoE yapısında bir modeldir.
```python
!pip install -qU transformers bitsandbytes accelerate
from transformers import AutoTokenizer, AutoModelForCausalLM
import torch
model_id = "Eurdem/Defne_llama3_2x8B"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(model_id, torch_dtype=torch.bfloat16, device_map="auto", load_in_8bit= True)
messages = [
{"role": "system", "content": "Sen Defne isimli Türkçe konuşan bir chatbotsun."},
{"role": "user", "content": "Sana 2 sorum var. 1) Sen kimsin? 2)f(x)=3x^2+4x+12 ise f(3) kaçtır?"}
]
input_ids = tokenizer.apply_chat_template(messages, return_tensors="pt").to("cuda")
outputs = model.generate(input_ids,
max_new_tokens=1024,
do_sample=True,
temperature=0.7,
top_p=0.7,
top_k=500,
)
response = outputs[0][input_ids.shape[-1]:]
print(tokenizer.decode(response, skip_special_tokens=True))
```
### Çıktı
```
Merhaba! Ben Sen Defne, Türkçe konuşan bir chatbotum. Hizmetinizdeyim.
Sorunuzun 2. kısmı için, f(x) = 3x^2 + 4x + 12 formülünü ele alalım. f(3)'ün hesabını yapalım:
f(3) = 3(3)^2 + 4(3) + 12
= 3(9) + 12 + 12
= 27 + 24
= 51
Bu nedenle, f(3) 51'dir.
```
|