Hugging Face's logo

bartowski
/
zephyr_7b_norobots-exl2

PEFT
code
instruct
zephyr
Model card Files Community
zephyr_7b_norobots-exl2 / README.md
bartowski's picture
bartowski
Quant for 4.0
2c9cc54
preview code
|
raw
history blame
1.59 kB
metadata
base_model: qblocks/zephyr_7b_norobots
datasets:
  - HuggingFaceH4/no_robots
inference: false
library_name: peft
license: apache-2.0
model_creator: MonsterAPI
model_name: Zephyr 7B Norobots
model_type: mistral
prompt_template: |
  <|im_start|>system
  {system_message}<|im_end|>
  <|im_start|>user
  {prompt}<|im_end|>
  <|im_start|>assistant
quantized_by: TheBloke
tags:
  - code
  - instruct
  - zephyr

Prompt Structure 

<|system|> <|endoftext|> <|user|> [USER PROMPT]<|endoftext|> <|assistant|> [ASSISTANT ANSWER] <|endoftext|>

Train loss :

training loss quantized_by: bartowski

Exllama v2 Quantizations of zephyr_7b_norobots at 4.0 bits per weight

Using turboderp's ExLlamaV2 v0.0.8 for quantization.

Conversion was done using wikitext-103-raw-v1-test.parquet as calibration dataset.

Original model: https://huggingface.co/TheBloke/zephyr_7b_norobots-fp16

Download instructions

With git:

git clone --single-branch --branch 4.0 https://huggingface.co/bartowski/zephyr_7b_norobots-exl2

With huggingface hub (credit to TheBloke for instructions):

pip3 install huggingface-hub

To download from a different branch, add the --revision parameter:

mkdir zephyr_7b_norobots-exl2
huggingface-cli download bartowski/zephyr_7b_norobots-exl2 --revision 4_0 --local-dir zephyr_7b_norobots-exl2 --local-dir-use-symlinks False