Breezeblossom-v4-mistral-2x7B
Breezeblossom-v4-mistral-2x7B is a Mixure of Experts (MoE) made with the following models using LazyMergekit:
🧩 Configuration
base_model: MediaTek-Research/Breeze-7B-Instruct-v0_1
gate_mode: hidden
dtype: float16
experts:
- source_model: MediaTek-Research/Breeze-7B-Instruct-v0_1
positive_prompts: [ "<s>You are a helpful AI assistant built by MediaTek Research. The user you are helping speaks Traditional Chinese and comes from Taiwan. [INST] 你好,請問你可以完成什麼任務? [/INST] "]
- source_model: Azure99/blossom-v4-mistral-7b
positive_prompts: ["A chat between a human and an artificial intelligence bot. The bot gives helpful, detailed, and polite answers to the human's questions. \n|Human|: hello\n|Bot|: "]
💻 Usage
!pip install -qU transformers bitsandbytes accelerate
from transformers import AutoTokenizer
import transformers
import torch
model = "sam-ezai/Breezeblossom-v4-mistral-2x7B"
tokenizer = AutoTokenizer.from_pretrained(model)
pipeline = transformers.pipeline(
"text-generation",
model=model,
model_kwargs={"torch_dtype": torch.float16, "load_in_4bit": True},
)
messages = [{"role": "user", "content": "Explain what a Mixture of Experts is in less than 100 words."}]
prompt = pipeline.tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
outputs = pipeline(prompt, max_new_tokens=256, do_sample=True, temperature=0.7, top_k=50, top_p=0.95)
print(outputs[0]["generated_text"])
- Downloads last month
- 4
This model does not have enough activity to be deployed to Inference API (serverless) yet. Increase its social
visibility and check back later, or deploy to Inference Endpoints (dedicated)
instead.
Model tree for sam-ezai/Breezeblossom-v4-mistral-2x7B
Merge model
this model