how to convert the model to gguf?

by rinoa - opened Nov 29, 2023

Nov 29, 2023

•

edited Nov 29, 2023

Hi, Could you please share the steps to convert this model into GGUF format? I tried to the convert.py from llama.cpp, but got an error:

Exception: Vocab size mismatch (model has 102400, but tokenizer.model has 32000).

deleted

Nov 30, 2023

It crashes when using GPT4ALL, so I'm not sure the Bloke was able to convert it to GGUF while maintaining full compatibility.

All the Chinese models (Yi, Qwen & Deepseek) are having an issue with GGUF, likely because of the gigantic token vocabulary required by the Chinese language.

Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.

Tap or paste here to upload images

· Sign up or log in to comment