Please consider support for GGUF
#5
by
ThiloteE
- opened
See llama.cpp issue https://github.com/ggerganov/llama.cpp/issues/9380.
Many projects are built on top of llama.cpp. Models in GGUF format are small and fast, which is probably one of the main advantages of OLMoE-1B-7B-0924-Instruct, compared to other (larger) models.
GGUF support has been added.
shanearora
changed discussion status to
closed