Description
This is an instruction following model (based on Mistral v0.1 Instruct) optimized for Russian language. It was trained using kolibrify on a multitude of instruction datasets.
The model uses ChatML template. Adding a system prompt will likely improve the model's performance on your task (experiment with it). I recommend using the model with LMStudio.
Currently in pre-alpha, later releases will include more details regarding training procedure and data mix.
Instruction following evals
The model was tested using the following benchmarks:
| Eval name | Strict Value | Loose Value |
|---|---|---|
| Avg. | 44.99 | 47.47 |
| ifeval-prompt-level | 45.84 | 48.05 |
| ifeval-instruction-level | 56.84 | 58.99 |
| ru-ifeval-prompt-level | 32.71 | 35.48 |
| ru-ifeval-instruction-level | 44.60 | 47.36 |
- Downloads last month
- 27