我想把这个模型转换成 gguf 格式,然后导入到 ollama 中,测试下效果,转换时候出错,请求帮助。
MiniCPM-Llama3-V 2.5 can run with llama.cpp now! See our fork of llama.cpp for more detail.
and here is the MiniCPM-Llama3-V-2_5-gguf
https://huggingface.co/openbmb/MiniCPM-Llama3-V-2_5-gguf
@changingshow
Try to import GGUF format model directly into ollama, it can not recognize the image, I don't know what is the problem
Hi! Thanks for the hardwork! I tried to create a Modelfile in ollama as followed:
FROM ./ggml-model-Q4_K_M.gguf
FROM ./mmproj-model-f16.gguf
TEMPLATE """{{ if .System }}<|start_header_id|>system<|end_header_id|>
{{ .System }}<|eot_id|>{{ end }}{{ if .Prompt }}<|start_header_id|>user<|end_header_id|>
{{ .Prompt }}<|eot_id|>{{ end }}<|start_header_id|>assistant<|end_header_id|>
{{ .Response }}<|eot_id|>"""
TEMPLATE "{{ .System }} USER: {{ .Prompt }} ASSISTANT:"
PARAMETER stop "<|start_header_id|>"
PARAMETER stop "<|end_header_id|>"
PARAMETER stop "<|eot_id|>"
PARAMETER stop "<|reserved_special_token"
If I remove the vision projector, the q4 gguf model works! Ollama core dump though if I combine the projector model. I see that you have forked llama.cpp. It seems that you're providing your own version of CLIP implementation/format for the visual encoder?
https://github.com/OpenBMB/ollama
@y22ma
Here is ollama fork for minicpm-v 2.5
For CLIP, we do make our own adaptations because we have dynamic input for the image
https://ollama.com/ghyghoo8/minicpm-llama3-2_5
I found this, does this support vision?