Quantization made by Richard Erkhov.

Poro-34B-chat - GGUF

Model creator: https://huggingface.co/LumiOpen/
Original model: https://huggingface.co/LumiOpen/Poro-34B-chat/

Name	Quant method	Size
Poro-34B-chat.Q2_K.gguf	Q2_K	12.49GB
Poro-34B-chat.IQ3_XS.gguf	IQ3_XS	14.05GB
Poro-34B-chat.IQ3_S.gguf	IQ3_S	14.42GB
Poro-34B-chat.Q3_K_S.gguf	Q3_K_S	14.42GB
Poro-34B-chat.IQ3_M.gguf	IQ3_M	15.94GB
Poro-34B-chat.Q3_K.gguf	Q3_K	17.23GB
Poro-34B-chat.Q3_K_M.gguf	Q3_K_M	17.23GB
Poro-34B-chat.Q3_K_L.gguf	Q3_K_L	18.78GB
Poro-34B-chat.IQ4_XS.gguf	IQ4_XS	17.83GB
Poro-34B-chat.Q4_0.gguf	Q4_0	18.65GB
Poro-34B-chat.IQ4_NL.gguf	IQ4_NL	18.79GB
Poro-34B-chat.Q4_K_S.gguf	Q4_K_S	18.79GB
Poro-34B-chat.Q4_K.gguf	Q4_K	20.9GB
Poro-34B-chat.Q4_K_M.gguf	Q4_K_M	20.9GB
Poro-34B-chat.Q4_1.gguf	Q4_1	20.64GB
Poro-34B-chat.Q5_0.gguf	Q5_0	22.63GB
Poro-34B-chat.Q5_K_S.gguf	Q5_K_S	22.63GB
Poro-34B-chat.Q5_K.gguf	Q5_K	24.32GB
Poro-34B-chat.Q5_K_M.gguf	Q5_K_M	24.32GB
Poro-34B-chat.Q5_1.gguf	Q5_1	24.62GB
Poro-34B-chat.Q6_K.gguf	Q6_K	26.86GB
Poro-34B-chat.Q8_0.gguf	Q8_0	34.78GB

Original model description:

license: apache-2.0 datasets: - LumiOpen/instruction-collection-fin language: - fi - en

Poro 34B Chat

Poro 34b chat is a chat-tuned version of Poro 34B trained to follow instructions in both Finnish and English. Quantized versions are available on Poro 34B-chat-GGUF.

Because of the limited amount of instruction tuning available for Finnish, documents from the English datasets were machine-translated by the Poro 34B base model into Finnish, then used to train this chat version. We selected only datasets that are available for commercial use and only contain synthetic data if it was gathered in ToS-compliant fashion.

More information about the data selection and translation process for our Finnish dataset are available on the LumiOpen/instruction-collection-fin page.

Poro was created in a collaboration between SiloGen from Silo AI, the TurkuNLP group of the University of Turku, and High Performance Language Technologies (HPLT). Training was conducted on the LUMI supercomputer, using compute resources generously provided by CSC - IT Center for Science, Finland.

This project is part of an ongoing effort to create open source large language models for non-English and especially low resource languages like Finnish. Through the combination of English and Finnish training data we get a model that outperforms previous Finnish only models, while also being fluent in English and code, and capable of basic translation between English and Finnish.

Fine Tuning

Poro-34b-Chat is an SFT finetune of Poro-34b on a collection of Finnish and English instruction datasets. The collection is made up of roughly of 40% English, 40% Finnish, and 20% cross-lingual entries.

We finetuned the base model for 3 epochs with a learning rate of 2e-05, warmup ratio of 0.1, and a global batch size of 48. For full-parameter finetuning, we used 3 nodes (8 GPUs per node). We used the Alignment Handbook code for finetuning.

Datasets

Finnish and Cross-lingual

LumiOpen/instruction-collection-fin

English

Chat template

We use the ChatML chat template. For example:

<|im_start|>system 
You can add an optional system prompt here.<|im_end|> 
<|im_start|>user 
Miten rakennan tietokoneen?<|im_end|>
<|im_start|>assistant

Evaluations

We relied on the popular MTBench benchmark to evaluate multi-turn performance.

Since MTBench is an English only benchmark, we also release this fork of MTBench Finnish with multilingual support and machine translated Finnish prompts. Our scores for both benchmarks follow.

Note: Updated on 18 June 2024

Eval	Overall	Coding	Extraction	Humanities	Math	Reasoning	Roleplay	STEM	Writing
MTBench English	6.13	4.25	6.65	9.60	2.30	4.30	7.05	7.55	7.35
MTBench Finnish	6.06	3.70	6.37	9.25	1.20	4.35	7.35	7.80	8.50

License

Poro 34B chat is released under the Apache 2.0 license.

Citation

@misc{luukkonen2024poro,
      title={Poro 34B and the Blessing of Multilinguality},
      author={Risto Luukkonen and Jonathan Burdge and Elaine Zosa and Aarne
Talman and Ville Komulainen and Väinö Hatanpää and Peter Sarlin and Sampo
Pyysalo},
      year={2024},
      eprint={2404.01856},
      archivePrefix={arXiv},
      primaryClass={cs.CL}
}