Quantization made by Richard Erkhov.

Helion-4x34B - GGUF

Model creator: https://huggingface.co/Weyaxi/
Original model: https://huggingface.co/Weyaxi/Helion-4x34B/

Name	Quant method	Size
Helion-4x34B.Q2_K.gguf	Q2_K	38.68GB
Helion-4x34B.Q3_K_S.gguf	Q3_K_S	45.66GB
Helion-4x34B.Q3_K.gguf	Q3_K	50.66GB
Helion-4x34B.Q3_K_M.gguf	Q3_K_M	50.66GB
Helion-4x34B.Q3_K_L.gguf	Q3_K_L	54.97GB
Helion-4x34B.IQ4_XS.gguf	IQ4_XS	57.03GB
Helion-4x34B.Q4_0.gguf	Q4_0	59.66GB
Helion-4x34B.IQ4_NL.gguf	IQ4_NL	60.19GB
Helion-4x34B.Q4_K_S.gguf	Q4_K_S	60.15GB
Helion-4x34B.Q4_K.gguf	Q4_K	63.95GB
Helion-4x34B.Q4_K_M.gguf	Q4_K_M	63.95GB
Helion-4x34B.Q4_1.gguf	Q4_1	66.25GB
Helion-4x34B.Q5_0.gguf	Q5_0	72.84GB
Helion-4x34B.Q5_K_S.gguf	Q5_K_S	72.84GB
Helion-4x34B.Q5_K.gguf	Q5_K	75.05GB
Helion-4x34B.Q5_K_M.gguf	Q5_K_M	75.05GB
Helion-4x34B.Q5_1.gguf	Q5_1	79.43GB
Helion-4x34B.Q6_K.gguf	Q6_K	86.84GB
Helion-4x34B.Q8_0.gguf	Q8_0	112.48GB

Original model description:

license: other tags: - yi - moe license_name: yi-license license_link: https://huggingface.co/01-ai/Yi-34B-200K/blob/main/LICENSE model-index: - name: Helion-4x34B results: - task: type: text-generation name: Text Generation dataset: name: AI2 Reasoning Challenge (25-Shot) type: ai2_arc config: ARC-Challenge split: test args: num_few_shot: 25 metrics: - type: acc_norm value: 69.71 name: normalized accuracy source: url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=Weyaxi/Helion-4x34B name: Open LLM Leaderboard - task: type: text-generation name: Text Generation dataset: name: HellaSwag (10-Shot) type: hellaswag split: validation args: num_few_shot: 10 metrics: - type: acc_norm value: 85.28 name: normalized accuracy source: url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=Weyaxi/Helion-4x34B name: Open LLM Leaderboard - task: type: text-generation name: Text Generation dataset: name: MMLU (5-Shot) type: cais/mmlu config: all split: test args: num_few_shot: 5 metrics: - type: acc value: 77.33 name: accuracy source: url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=Weyaxi/Helion-4x34B name: Open LLM Leaderboard - task: type: text-generation name: Text Generation dataset: name: TruthfulQA (0-shot) type: truthful_qa config: multiple_choice split: validation args: num_few_shot: 0 metrics: - type: mc2 value: 63.91 source: url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=Weyaxi/Helion-4x34B name: Open LLM Leaderboard - task: type: text-generation name: Text Generation dataset: name: Winogrande (5-shot) type: winogrande config: winogrande_xl split: validation args: num_few_shot: 5 metrics: - type: acc value: 84.37 name: accuracy source: url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=Weyaxi/Helion-4x34B name: Open LLM Leaderboard - task: type: text-generation name: Text Generation dataset: name: GSM8k (5-shot) type: gsm8k config: main split: test args: num_few_shot: 5 metrics: - type: acc value: 72.25 name: accuracy source: url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=Weyaxi/Helion-4x34B name: Open LLM Leaderboard

Helion-4x34B

This is the model for Helion-4x34B. I used this repo to make this MOE model.

Prompt Template(s):

Since bagel-dpo-34b-v0.2 uses many prompt templates, you can utilize prompt templates provided by bagel and other expert's prompt templates.

Note: I currently do not know which prompt template is best.

ChatML:

<|im_start|>system
{system}<|im_end|>
<|im_start|>user
{user}<|im_end|>
<|im_start|>assistant
{asistant}<|im_end|>

Human Asistant

Human: {user}

### Assistant: {asistant}

Alpaca (sort of)

Below is an instruction that describes a task.  Write a response that appropriately completes the request.

### Instruction:
{system}
{instruction}

### Response:

Vicuna

{system}
USER: {instruction}
ASSISTANT:

Visit bagel-dpo-34b-v0.2 to try more prompt templates.

Yaml Config to reproduce

base_model: nontoxic-bagel-34b-v0.2
gate_mode: hidden
dtype: bfloat16

experts:
  - source_model: bagel-dpo-34b-v0.2
    positive_prompts: ["question answering", "Q:", science", "biology", "chemistry", "physics"]
    negative_prompts: ["math", "reason", "mathematics", "solve", "count", "code", "python", "javascript", "programming", "algorithm"]

  - source_model: Nous-Hermes-2-Yi-34B
    positive_prompts: ["chat", "math", "reason", "mathematics", "solve", "count", "python", "javascript", "programming", "algorithm", "tell me", "assistant"]

  - source_model: SUS-Chat-34B
    positive_prompts: ["math", "reason", "mathematics", "solve", "count", "assistant"]

  - source_model: platypus-yi-34b
    positive_prompts: [""]
    negative_prompts: ["math", "reason", "mathematics", "solve", "count"]

Quantizationed versions

Quantizationed versions of this model is available thanks to TheBloke.

Open LLM Leaderboard Evaluation Results

Detailed results can be found here

Metric	Value
Avg.	75.48
AI2 Reasoning Challenge (25-Shot)	69.71
HellaSwag (10-Shot)	85.28
MMLU (5-Shot)	77.33
TruthfulQA (0-shot)	63.91
Winogrande (5-shot)	84.37
GSM8k (5-shot)	72.25

If you would like to support me:

☕ Buy Me a Coffee