Benchmarks worse than Qwen2.5-7B-Instruct on MMLU-Pro Computer Science in limited testing.
#1
by
ubergarm
- opened
Did a more detailed write-up here on r/localllama
| Model | MMLU-Pro Computer Science Score | Generation tok/sec |
|---|---|---|
bartowski/qwen2.5-7b-ins-v3-GGUF |
51.71 | 104.76 |
bartowski/Qwen2.5-7B-Instruct-GGUF |
59.51 | 170.51 |
Might be system prompt effecting it or something else, YMMV...