TheBloke

/

OpenHermes-2.5-Mistral-7B-GGUF

like 230

I tried running the model on kali linux using ctransformers

tokenizer = AutoTokenizer.from_pretrained("teknium/OpenHermes-2.5-Mistral-7B")
model_path = "openhermes-2.5-mistral-7b.Q4_K_M.gguf"

model = AutoModelForCausalLM.from_pretrained(model_path, gpu_layers=16, threads=1, context_length=4096, max_new_tokens=-1)

and i always get this error:
Special tokens have been added in the vocabulary, make sure the associated word embeddings are fine-tuned or trained.
Special tokens have been added in the vocabulary, make sure the associated word embeddings are fine-tuned or trained.
zsh: illegal hardware instruction python PentestAI.py