---
license: cc-by-sa-4.0
language:
- pl
---
# Model Card for Krakowiak-v2-7b

Krakowiak-v2-7b is a state of the art 7.3 billion parameters LLM based on Mistral-7B. It was finetuned for Polish text generation using custom created large corpus of 100K Polish instructions. It uses novel techniques e.g. LORA, adding noise to the embeddings for great preformance. For full details of this model please read our [paper to be released soon](www.example.come)

## Model Architecture

Krakowiak-v2-7b is a huge update to [Krakowiak-7b](https://huggingface.co/szymonrucinski/krakowiak-7b) bringing the following improvements:
- Grouped-Query Attention
- Sliding-Window Attention
- Byte-fallback BPE tokenizer
- Significantly larger training corpus of better quality
- Improved training pipeline
- Faster inference
- No random token generation (generating Russian or Czech text alongside Polish)
- Significantly higher quality of generated text

Mistral 7B is a pretrained base model and therefore does not have any moderation mechanisms.

## Krakowiak team
 
[Szymon Franciszek Ruciński](https://szymonrucinski.pl)